2024-02-14T14:41:35.086 INFO:root:teuthology version: 0.0.1.dev258+g9e44c16 2024-02-14T14:41:35.087 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621 --owner scheduled_vshankar@teuthology --name vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/orig.config.yaml 2024-02-14T14:41:35.143 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T14:41:35.234 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621 branch: wip-vshankar-testing-20240206.051942 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7559621' kernel: client: sha1: distro flavor: default kdb: true sha1: 8d59ab0bd93c806231018a6729893bf309624262 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240206.051942 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240206.051942 sha1: a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 962 sha1: a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 sleep_before_teardown: 0 subset: 960/1024 suite: fs suite_branch: wip-vshankar-testing-20240206.051942 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi017.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAa8ezSIdIQKqK0GdLjLoauCo8pQ9mm8cyzgjq2oN8+LPnPb3qENtPalaXvAKivFrPGUKg4fVB7Wtr8yMbGOD04= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgqXvaELHhVlHUxeAN7QyRbJFFiggi2/+PXlYCDY0LypmslC2KbdwnKVNsBInMbxDfU31Z5HKm2oEdEyghJdhk= smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK7bWvqByIQoaYmNpH83DOBveIfzmZbDyd9okIeP4/1goaE2n/hmJNOEqbaruhiJqDad6bqkxwnecLP2avm6QtA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9e44c16382dfc4c612f7437cbb344c76223137cc timestamp: 2024-02-14_05:24:17 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-14T14:41:35.234 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa; will attempt to use it 2024-02-14T14:41:35.234 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks 2024-02-14T14:41:35.234 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-14T14:41:35.236 INFO:teuthology.task.internal:Checking packages... 2024-02-14T14:41:35.256 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0' 2024-02-14T14:41:35.256 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-14T14:41:35.256 INFO:teuthology.packaging:ref: None 2024-02-14T14:41:35.256 INFO:teuthology.packaging:tag: None 2024-02-14T14:41:35.256 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240206.051942 2024-02-14T14:41:35.256 INFO:teuthology.packaging:sha1: a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:41:35.256 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240206.051942 2024-02-14T14:41:35.570 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1280.ga6fe3fa3 2024-02-14T14:41:35.571 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-14T14:41:35.579 INFO:teuthology.task.internal:no buildpackages task found 2024-02-14T14:41:35.579 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-14T14:41:35.598 INFO:teuthology.task.internal:Saving configuration 2024-02-14T14:41:35.618 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-14T14:41:35.627 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T14:41:35.649 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 14:35:57.969843', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAa8ezSIdIQKqK0GdLjLoauCo8pQ9mm8cyzgjq2oN8+LPnPb3qENtPalaXvAKivFrPGUKg4fVB7Wtr8yMbGOD04='} 2024-02-14T14:41:35.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 14:35:57.971559', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgqXvaELHhVlHUxeAN7QyRbJFFiggi2/+PXlYCDY0LypmslC2KbdwnKVNsBInMbxDfU31Z5HKm2oEdEyghJdhk='} 2024-02-14T14:41:35.691 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 14:35:57.967825', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK7bWvqByIQoaYmNpH83DOBveIfzmZbDyd9okIeP4/1goaE2n/hmJNOEqbaruhiJqDad6bqkxwnecLP2avm6QtA='} 2024-02-14T14:41:35.691 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-14T14:41:35.699 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-14T14:41:35.699 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-14T14:41:35.699 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-14T14:41:35.699 INFO:teuthology.run_tasks:Running task console_log... 2024-02-14T14:41:35.799 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f63a3d50160>, signals=[15]) 2024-02-14T14:41:35.799 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-14T14:41:35.805 INFO:teuthology.task.internal:Opening connections... 2024-02-14T14:41:35.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T14:41:35.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:41:35.911 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-02-14T14:41:35.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:41:36.013 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-02-14T14:41:36.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:41:36.096 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-14T14:41:36.104 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-02-14T14:41:36.122 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-02-14T14:41:36.122 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-02-14T14:41:36.178 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream" 2024-02-14T14:41:36.178 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="9" 2024-02-14T14:41:36.178 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2024-02-14T14:41:36.178 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="9" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el9" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:LOGO="fedora-logo-icon" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T14:41:36.179 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T14:41:36.180 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-02-14T14:41:36.203 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-02-14T14:41:36.221 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-02-14T14:41:36.221 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-02-14T14:41:36.277 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T14:41:36.278 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T14:41:36.279 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T14:41:36.279 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-02-14T14:41:36.303 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-02-14T14:41:36.321 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-02-14T14:41:36.321 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-02-14T14:41:36.378 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T14:41:36.379 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T14:41:36.380 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T14:41:36.380 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-02-14T14:41:36.403 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-14T14:41:36.413 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-14T14:41:36.422 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-14T14:41:36.422 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2024-02-14T14:41:36.425 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-02-14T14:41:36.427 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-02-14T14:41:36.443 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-14T14:41:36.452 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-14T14:41:36.453 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2024-02-14T14:41:36.482 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-02-14T14:41:36.485 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-02-14T14:41:36.518 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-14T14:41:36.604 INFO:teuthology.run_tasks:Running task kernel... 2024-02-14T14:41:36.621 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8d59ab0bd93c806231018a6729893bf309624262'} 2024-02-14T14:41:36.622 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-02-14T14:41:36.622 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-02-14T14:41:36.622 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T14:41:36.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T14:41:36.639 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-02-14T14:41:36.698 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-361.el9.x86_64 2024-02-14T14:41:36.698 INFO:teuthology.task.kernel:Running kernel on smithi017: 5.14.0-361.el9.x86_64 2024-02-14T14:41:36.698 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum install -y kernel 2024-02-14T14:41:39.199 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - AppStream 8.2 MB/s | 6.7 MB 00:00 2024-02-14T14:41:40.882 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-02-14T14:41:41.806 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-02-14T14:41:47.341 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 19 MB 00:04 2024-02-14T14:41:53.499 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-02-14T14:41:55.258 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2024-02-14T14:41:57.488 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 20 MB 00:02 2024-02-14T14:42:02.534 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-02-14T14:42:04.967 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T14:42:04.967 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T14:42:05.055 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-02-14T14:42:05.060 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T14:42:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:42:05.062 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-02-14T14:42:05.062 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-02-14T14:42:05.062 INFO:teuthology.orchestra.run.smithi017.stdout:Install 4 Packages 2024-02-14T14:42:05.062 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:42:05.063 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 99 M 2024-02-14T14:42:05.063 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 122 M 2024-02-14T14:42:05.063 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-02-14T14:42:06.094 INFO:teuthology.orchestra.run.smithi017.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 5.8 MB/s | 6.0 MB 00:01 2024-02-14T14:42:07.661 INFO:teuthology.orchestra.run.smithi017.stdout:(2/4): kernel-core-5.14.0-420.el9.x86_64.rpm 8.0 MB/s | 21 MB 00:02 2024-02-14T14:42:08.636 INFO:teuthology.orchestra.run.smithi017.stdout:(3/4): kernel-modules-core-5.14.0-420.el9.x86_6 13 MB/s | 33 MB 00:02 2024-02-14T14:42:09.387 INFO:teuthology.orchestra.run.smithi017.stdout:(4/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 9.0 MB/s | 39 MB 00:04 2024-02-14T14:42:09.390 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-02-14T14:42:09.390 INFO:teuthology.orchestra.run.smithi017.stdout:Total 23 MB/s | 99 MB 00:04 2024-02-14T14:42:09.390 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-02-14T14:42:09.843 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-02-14T14:42:09.844 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-02-14T14:42:10.933 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-02-14T14:42:10.934 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-02-14T14:42:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-02-14T14:42:14.136 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T14:42:14.261 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T14:42:17.226 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T14:42:17.406 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T14:42:22.509 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T14:42:22.619 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T14:42:27.731 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T14:42:58.587 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T14:42:58.587 INFO:teuthology.orchestra.run.smithi017.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T14:42:58.587 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:42:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T14:42:59.090 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T14:42:59.090 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T14:42:59.091 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T14:42:59.091 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T14:42:59.525 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T14:42:59.525 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:42:59.526 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-02-14T14:42:59.526 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T14:42:59.526 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T14:42:59.526 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T14:42:59.526 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T14:42:59.526 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:42:59.526 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-02-14T14:43:00.024 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T14:43:00.208 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T14:43:00.208 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T14:43:00.208 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T14:43:00.208 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T14:43:00.209 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T14:43:00.209 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-02-14T14:43:00.209 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T14:43:00.209 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T14:43:00.321 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T14:43:00.322 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T14:43:00.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo rpm -qi grub2-tools 2024-02-14T14:43:00.371 INFO:teuthology.orchestra.run.smithi017.stdout:Name : grub2-tools 2024-02-14T14:43:00.371 INFO:teuthology.orchestra.run.smithi017.stdout:Epoch : 1 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Version : 2.06 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Release : 68.el9 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: x86_64 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Group : Unspecified 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Size : 8264254 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:License : GPLv3+ 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T14:43:00.372 INFO:teuthology.orchestra.run.smithi017.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:Packager : builder@centos.org 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:Vendor : CentOS 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:Summary : Support tools for GRUB. 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:Description : 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T14:43:00.373 INFO:teuthology.orchestra.run.smithi017.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T14:43:00.374 INFO:teuthology.orchestra.run.smithi017.stdout:hardware devices. 2024-02-14T14:43:00.374 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:43:00.374 INFO:teuthology.orchestra.run.smithi017.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T14:43:00.375 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T14:43:00.375 INFO:teuthology.task.kernel:Updating grub on smithi017 to boot 5.14.0-420.el9.x86_64 2024-02-14T14:43:00.375 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T14:43:01.036 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-02-14T14:43:02.533 INFO:teuthology.orchestra.run.smithi017.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T14:43:02.559 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-02-14T14:43:02.562 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-02-14T14:43:02.578 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.z0NqXs1rXC 2024-02-14T14:43:02.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z0NqXs1rXC 2024-02-14T14:43:02.656 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.z0NqXs1rXC 2024-02-14T14:43:02.805 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.z0NqXs1rXC is 6KB 2024-02-14T14:43:02.857 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.z0NqXs1rXC 2024-02-14T14:43:02.872 DEBUG:teuthology.orchestra.run.smithi017:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T14:43:02.941 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T14:43:02.941 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T14:43:02.941 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T14:43:02.941 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T14:43:02.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T14:43:03.086 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2024-02-14T14:43:33.089 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-14T14:43:33.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-02-14T14:43:33.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:44:33.093 DEBUG:teuthology.orchestra.remote:timed out 2024-02-14T14:44:42.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-02-14T14:44:42.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:44:42.507 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-02-14T14:44:43.004 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-02-14T14:44:43.004 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-420.el9.x86_64"... 2024-02-14T14:44:43.005 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-02-14T14:44:43.018 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-420.el9.x86_64 2024-02-14T14:44:43.019 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T14:44:43.019 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T14:44:43.019 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T14:44:44.020 DEBUG:teuthology.parallel:result is None 2024-02-14T14:44:44.021 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-14T14:44:44.028 INFO:teuthology.task.internal:Creating test directory... 2024-02-14T14:44:44.028 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T14:44:44.030 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T14:44:44.033 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T14:44:44.051 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-14T14:44:44.057 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-14T14:44:44.064 INFO:teuthology.task.internal:Creating archive directory... 2024-02-14T14:44:44.064 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T14:44:44.087 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T14:44:44.092 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T14:44:44.202 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-14T14:44:44.208 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-14T14:44:44.208 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T14:44:44.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T14:44:44.226 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T14:44:44.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T14:44:44.242 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T14:44:44.244 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T14:44:44.269 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T14:44:44.308 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T14:44:44.332 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T14:44:44.334 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T14:44:44.344 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T14:44:44.357 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T14:44:44.728 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T14:44:44.730 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-14T14:44:44.738 INFO:teuthology.task.internal:Configuring sudo... 2024-02-14T14:44:44.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T14:44:44.741 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T14:44:44.743 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T14:44:44.781 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-14T14:44:44.790 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-14T14:44:44.790 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T14:44:44.824 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T14:44:44.827 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T14:44:44.845 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T14:44:44.941 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T14:44:45.011 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T14:44:45.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T14:44:45.120 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:44:45.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T14:44:45.209 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T14:44:45.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T14:44:45.318 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T14:44:45.360 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T14:44:45.409 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:44:45.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T14:44:45.482 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T14:44:45.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T14:44:45.586 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T14:44:45.629 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T14:44:45.676 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:44:45.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T14:44:45.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2024-02-14T14:44:45.750 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-02-14T14:44:45.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-02-14T14:44:45.791 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T14:44:45.800 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T14:44:45.831 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T14:44:46.169 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-14T14:44:46.177 INFO:teuthology.task.internal:Starting timer... 2024-02-14T14:44:46.177 INFO:teuthology.run_tasks:Running task pcp... 2024-02-14T14:44:46.186 INFO:teuthology.run_tasks:Running task selinux... 2024-02-14T14:44:46.194 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-14T14:44:46.194 DEBUG:teuthology.orchestra.run.smithi017:> sudo service auditd rotate 2024-02-14T14:44:46.359 INFO:teuthology.orchestra.run.smithi017.stdout:Rotating logs: 2024-02-14T14:44:46.360 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-02-14T14:44:46.428 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-02-14T14:44:46.430 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-02-14T14:44:46.488 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-02-14T14:44:46.489 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-14T14:44:46.489 DEBUG:teuthology.orchestra.run.smithi017:> /usr/sbin/getenforce 2024-02-14T14:44:46.510 INFO:teuthology.orchestra.run.smithi017.stdout:Permissive 2024-02-14T14:44:46.511 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-02-14T14:44:46.533 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-02-14T14:44:46.533 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-02-14T14:44:46.566 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-02-14T14:44:46.566 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi017.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi076.front.sepia.ceph.com': 'permissive'} 2024-02-14T14:44:46.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T14:44:46.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T14:44:46.593 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T14:44:46.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T14:44:46.625 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T14:44:46.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T14:44:46.654 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-14T14:44:46.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo /usr/sbin/setenforce permissive 2024-02-14T14:44:46.720 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-02-14T14:44:46.760 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-02-14T14:44:46.793 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-14T14:44:46.803 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-14T14:44:46.808 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-14T14:44:46.927 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-14T14:44:46.940 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-14T14:44:46.941 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi050.front.sepia.ceph.com,smithi076.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-14T14:49:24.014 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi076.front.sepia.ceph.com')] 2024-02-14T14:49:24.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-02-14T14:49:24.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:49:24.111 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-02-14T14:49:24.186 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-02-14T14:49:24.187 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-14T14:49:24.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:49:24.292 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-02-14T14:49:24.367 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-14T14:49:24.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-14T14:49:24.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T14:49:24.496 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-02-14T14:49:24.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-02-14T14:49:24.563 INFO:teuthology.run_tasks:Running task clock... 2024-02-14T14:49:24.572 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-14T14:49:24.572 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T14:49:24.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T14:49:24.575 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T14:49:24.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T14:49:24.577 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T14:49:24.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T14:49:24.608 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T14:49:24.614 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T14:49:24.626 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T14:49:24.633 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T14:49:24.635 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T14:49:24.650 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T14:49:24.657 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: ntpd: command not found 2024-02-14T14:49:24.672 INFO:teuthology.orchestra.run.smithi017.stdout:506 Cannot talk to daemon 2024-02-14T14:49:24.673 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-02-14T14:49:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-02-14T14:49:24.689 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T14:49:24.690 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-02-14T14:49:24.695 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-02-14T14:49:24.706 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T14:49:24.709 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T14:49:24.710 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T14:49:24.726 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T14:49:24.730 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T14:49:24.763 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-02-14T14:49:24.767 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T14:49:24.767 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-02-14T14:49:24.767 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.767 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.767 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.768 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.770 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-02-14T14:49:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T14:49:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-02-14T14:49:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.778 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-02-14T14:49:24.782 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T14:49:24.782 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-02-14T14:49:24.782 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.782 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.782 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.782 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T14:49:24.783 INFO:teuthology.run_tasks:Running task install... 2024-02-14T14:49:24.790 DEBUG:teuthology.task.install:project ceph 2024-02-14T14:49:24.790 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0'}} 2024-02-14T14:49:24.790 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0'} 2024-02-14T14:49:24.790 INFO:teuthology.task.install:Using flavor: default 2024-02-14T14:49:24.795 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-14T14:49:24.796 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-14T14:49:24.796 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0', 'tag': None, 'wait_for_package': False} 2024-02-14T14:49:24.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:49:24.799 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0', 'tag': None, 'wait_for_package': False} 2024-02-14T14:49:24.799 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:49:24.800 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0', 'tag': None, 'wait_for_package': False} 2024-02-14T14:49:24.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:49:24.943 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/ 2024-02-14T14:49:24.943 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1280.ga6fe3fa3 2024-02-14T14:49:24.956 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/ 2024-02-14T14:49:24.956 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1280.ga6fe3fa3 2024-02-14T14:49:24.957 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/ 2024-02-14T14:49:24.957 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1280.ga6fe3fa3 2024-02-14T14:49:25.058 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-14T14:49:25.059 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:49:25.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-14T14:49:25.064 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-14T14:49:25.064 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:49:25.064 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-14T14:49:25.086 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-14T14:49:25.087 DEBUG:teuthology.orchestra.run.smithi017:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-14T14:49:25.096 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-14T14:49:25.096 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-14T14:49:25.123 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-14T14:49:25.123 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:49:25.123 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-14T14:49:25.151 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-14T14:49:25.151 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-14T14:49:25.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-14T14:49:25.168 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-14T14:49:25.219 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-14T14:49:25.233 DEBUG:teuthology.orchestra.run.smithi017:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-14T14:49:25.255 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-14T14:49:25.285 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-02-14T14:49:25.286 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-02-14T14:49:25.298 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-14T14:49:25.299 INFO:teuthology.orchestra.run.smithi017.stdout:check_obsoletes = 1 2024-02-14T14:49:25.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum clean all 2024-02-14T14:49:25.364 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-02-14T14:49:25.366 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-02-14T14:49:25.534 INFO:teuthology.orchestra.run.smithi050.stdout:74 files removed 2024-02-14T14:49:25.550 INFO:teuthology.orchestra.run.smithi017.stdout:74 files removed 2024-02-14T14:49:25.561 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-14T14:49:25.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-14T14:49:25.602 INFO:teuthology.orchestra.run.smithi076.stdout:74 files removed 2024-02-14T14:49:25.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-14T14:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 216 kB/s | 80 kB 00:00 2024-02-14T14:49:26.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 170 kB/s | 80 kB 00:00 2024-02-14T14:49:26.413 INFO:teuthology.orchestra.run.smithi017.stdout:ceph packages for x86_64 157 kB/s | 80 kB 00:00 2024-02-14T14:49:26.607 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-02-14T14:49:26.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-02-14T14:49:26.822 INFO:teuthology.orchestra.run.smithi017.stdout:ceph noarch packages 28 kB/s | 10 kB 00:00 2024-02-14T14:49:26.949 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-02-14T14:49:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-02-14T14:49:27.239 INFO:teuthology.orchestra.run.smithi017.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-02-14T14:49:27.452 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-02-14T14:49:27.515 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-02-14T14:49:27.716 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-14T14:49:29.010 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2024-02-14T14:49:29.016 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2024-02-14T14:49:29.167 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-02-14T14:49:29.949 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-02-14T14:49:30.098 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-02-14T14:49:30.268 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-02-14T14:49:32.083 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-02-14T14:49:33.542 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-02-14T14:49:34.544 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-02-14T14:49:36.984 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-02-14T14:49:38.826 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.1 MB 00:01 2024-02-14T14:49:39.475 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2024-02-14T14:49:39.644 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-02-14T14:49:41.079 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2024-02-14T14:49:41.251 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 15 kB 00:01 2024-02-14T14:49:41.287 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-02-14T14:49:43.680 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 20 MB 00:02 2024-02-14T14:49:43.880 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 20 MB 00:02 2024-02-14T14:49:46.010 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-02-14T14:49:48.651 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-14T14:49:48.652 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-14T14:49:48.666 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-14T14:49:48.667 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-14T14:49:48.668 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-14T14:49:48.669 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-14T14:49:48.671 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-14T14:49:48.674 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-14T14:49:48.675 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-14T14:49:48.694 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-02-14T14:49:48.755 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 6.6 k 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 5.1 M 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 886 k 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 145 k 2024-02-14T14:49:48.775 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 1.5 M 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 138 k 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 1.8 M 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 7.4 M 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 52 k 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 9.7 M 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 47 M 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 263 k 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 748 k 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 217 k 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 27 k 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 18 k 2024-02-14T14:49:48.776 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 31 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 743 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 127 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 161 k 2024-02-14T14:49:48.777 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 321 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 297 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 99 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 85 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.1 M 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 171 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-14T14:49:48.778 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.4 M 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.1 M 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-14T14:49:48.779 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 20 M 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 28 k 2024-02-14T14:49:48.780 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 2.1 M 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 247 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 4.7 M 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 17 M 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 15 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 25 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-14T14:49:48.781 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 163 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 499 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 5.1 M 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-14T14:49:48.782 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-14T14:49:48.783 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 45 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 131 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-14T14:49:48.784 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-14T14:49:48.785 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-14T14:49:48.786 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-14T14:49:48.787 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout:Install 147 Packages 2024-02-14T14:49:48.788 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 26 Packages 2024-02-14T14:49:48.789 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:49:48.789 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 222 M 2024-02-14T14:49:48.789 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-02-14T14:49:48.930 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-02-14T14:49:49.603 INFO:teuthology.orchestra.run.smithi076.stdout:(1/173): ceph-19.0.0-1280.ga6fe3fa3.el9.x86_64. 42 kB/s | 6.6 kB 00:00 2024-02-14T14:49:49.812 INFO:teuthology.orchestra.run.smithi076.stdout:(2/173): ceph-fuse-19.0.0-1280.ga6fe3fa3.el9.x8 4.2 MB/s | 886 kB 00:00 2024-02-14T14:49:49.878 INFO:teuthology.orchestra.run.smithi076.stdout:(3/173): ceph-immutable-object-cache-19.0.0-128 2.1 MB/s | 145 kB 00:00 2024-02-14T14:49:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:(4/173): ceph-base-19.0.0-1280.ga6fe3fa3.el9.x8 9.3 MB/s | 5.1 MB 00:00 2024-02-14T14:49:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:(5/173): ceph-mds-19.0.0-1280.ga6fe3fa3.el9.x86 11 MB/s | 2.1 MB 00:00 2024-02-14T14:49:50.121 INFO:teuthology.orchestra.run.smithi076.stdout:(6/173): ceph-mgr-19.0.0-1280.ga6fe3fa3.el9.x86 12 MB/s | 1.5 MB 00:00 2024-02-14T14:49:50.538 INFO:teuthology.orchestra.run.smithi076.stdout:(7/173): ceph-mon-19.0.0-1280.ga6fe3fa3.el9.x86 10 MB/s | 4.7 MB 00:00 2024-02-14T14:49:51.272 INFO:teuthology.orchestra.run.smithi076.stdout:(8/173): ceph-osd-19.0.0-1280.ga6fe3fa3.el9.x86 15 MB/s | 17 MB 00:01 2024-02-14T14:49:51.313 INFO:teuthology.orchestra.run.smithi076.stdout:(9/173): ceph-selinux-19.0.0-1280.ga6fe3fa3.el9 610 kB/s | 25 kB 00:00 2024-02-14T14:49:51.370 INFO:teuthology.orchestra.run.smithi017.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-14T14:49:51.371 INFO:teuthology.orchestra.run.smithi017.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-14T14:49:51.383 INFO:teuthology.orchestra.run.smithi017.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-14T14:49:51.384 INFO:teuthology.orchestra.run.smithi017.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-14T14:49:51.385 INFO:teuthology.orchestra.run.smithi017.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-14T14:49:51.386 INFO:teuthology.orchestra.run.smithi017.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-14T14:49:51.387 INFO:teuthology.orchestra.run.smithi017.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-14T14:49:51.390 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-14T14:49:51.391 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-14T14:49:51.447 INFO:teuthology.orchestra.run.smithi076.stdout:(10/173): ceph-radosgw-19.0.0-1280.ga6fe3fa3.el 11 MB/s | 9.7 MB 00:00 2024-02-14T14:49:51.477 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-02-14T14:49:51.497 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-02-14T14:49:51.497 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 6.6 k 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 5.1 M 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 886 k 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 145 k 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 1.5 M 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 138 k 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 1.8 M 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 7.4 M 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 52 k 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 9.7 M 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 47 M 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 263 k 2024-02-14T14:49:51.498 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 748 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 217 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-shell x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 27 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 18 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 31 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 743 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 127 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-14T14:49:51.499 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 161 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 321 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 297 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 99 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 85 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.1 M 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 171 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrading: 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-14T14:49:51.500 INFO:teuthology.orchestra.run.smithi017.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.4 M 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.1 M 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-14T14:49:51.501 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 20 M 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 28 k 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 2.1 M 2024-02-14T14:49:51.502 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 247 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 4.7 M 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 17 M 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 15 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 25 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-14T14:49:51.503 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 163 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 499 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 5.1 M 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-14T14:49:51.504 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 45 k 2024-02-14T14:49:51.505 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 131 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-14T14:49:51.506 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-14T14:49:51.507 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-14T14:49:51.508 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-14T14:49:51.509 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-14T14:49:51.510 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:49:51.510 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-02-14T14:49:51.510 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-02-14T14:49:51.510 INFO:teuthology.orchestra.run.smithi017.stdout:Install 147 Packages 2024-02-14T14:49:51.510 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrade 26 Packages 2024-02-14T14:49:51.510 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:49:51.510 INFO:teuthology.orchestra.run.smithi076.stdout:(11/173): cephfs-mirror-19.0.0-1280.ga6fe3fa3.e 3.6 MB/s | 217 kB 00:00 2024-02-14T14:49:51.511 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 222 M 2024-02-14T14:49:51.511 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-02-14T14:49:51.556 INFO:teuthology.orchestra.run.smithi076.stdout:(12/173): cephfs-shell-19.0.0-1280.ga6fe3fa3.el 543 kB/s | 27 kB 00:00 2024-02-14T14:49:51.588 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-14T14:49:51.589 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-14T14:49:51.604 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-14T14:49:51.605 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-14T14:49:51.606 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-14T14:49:51.607 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-14T14:49:51.609 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-14T14:49:51.612 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-14T14:49:51.614 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-14T14:49:51.614 INFO:teuthology.orchestra.run.smithi076.stdout:(13/173): libcephfs-devel-19.0.0-1280.ga6fe3fa3 537 kB/s | 31 kB 00:00 2024-02-14T14:49:51.698 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 6.6 k 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 5.1 M 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 886 k 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 145 k 2024-02-14T14:49:51.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 1.5 M 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 138 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 1.8 M 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 7.4 M 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 52 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 9.7 M 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 47 M 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 263 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 748 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 217 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 27 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 18 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-14T14:49:51.717 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 31 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 743 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 127 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 161 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 321 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 297 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 99 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 85 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.1 M 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 171 k 2024-02-14T14:49:51.718 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-14T14:49:51.719 INFO:teuthology.orchestra.run.smithi050.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.4 M 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 3.1 M 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-14T14:49:51.720 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 20 M 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 28 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 2.1 M 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 247 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 4.7 M 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 17 M 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1280.ga6fe3fa3.el9 ceph-noarch 15 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 25 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-14T14:49:51.721 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 163 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 499 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 5.1 M 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-14T14:49:51.722 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-14T14:49:51.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 45 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-1280.ga6fe3fa3.el9 ceph 131 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-14T14:49:51.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-14T14:49:51.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-14T14:49:51.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-14T14:49:51.727 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout:Install 147 Packages 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 26 Packages 2024-02-14T14:49:51.728 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:49:51.729 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 222 M 2024-02-14T14:49:51.729 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-14T14:49:51.739 INFO:teuthology.orchestra.run.smithi076.stdout:(14/173): libcephfs2-19.0.0-1280.ga6fe3fa3.el9. 5.8 MB/s | 743 kB 00:00 2024-02-14T14:49:51.806 INFO:teuthology.orchestra.run.smithi076.stdout:(15/173): libcephsqlite-19.0.0-1280.ga6fe3fa3.e 2.4 MB/s | 163 kB 00:00 2024-02-14T14:49:51.865 INFO:teuthology.orchestra.run.smithi076.stdout:(16/173): librados-devel-19.0.0-1280.ga6fe3fa3. 2.1 MB/s | 127 kB 00:00 2024-02-14T14:49:51.957 INFO:teuthology.orchestra.run.smithi076.stdout:(17/173): libradosstriper1-19.0.0-1280.ga6fe3fa 5.3 MB/s | 499 kB 00:00 2024-02-14T14:49:52.457 INFO:teuthology.orchestra.run.smithi076.stdout:(18/173): librgw2-19.0.0-1280.ga6fe3fa3.el9.x86 10 MB/s | 5.1 MB 00:00 2024-02-14T14:49:52.516 INFO:teuthology.orchestra.run.smithi076.stdout:(19/173): python3-ceph-argparse-19.0.0-1280.ga6 778 kB/s | 45 kB 00:00 2024-02-14T14:49:52.583 INFO:teuthology.orchestra.run.smithi076.stdout:(20/173): python3-ceph-common-19.0.0-1280.ga6fe 1.9 MB/s | 131 kB 00:00 2024-02-14T14:49:52.649 INFO:teuthology.orchestra.run.smithi076.stdout:(21/173): python3-cephfs-19.0.0-1280.ga6fe3fa3. 2.4 MB/s | 161 kB 00:00 2024-02-14T14:49:52.716 INFO:teuthology.orchestra.run.smithi076.stdout:(22/173): python3-rados-19.0.0-1280.ga6fe3fa3.e 4.7 MB/s | 321 kB 00:00 2024-02-14T14:49:52.737 INFO:teuthology.orchestra.run.smithi017.stdout:(1/173): ceph-19.0.0-1280.ga6fe3fa3.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-02-14T14:49:52.783 INFO:teuthology.orchestra.run.smithi076.stdout:(23/173): python3-rbd-19.0.0-1280.ga6fe3fa3.el9 4.4 MB/s | 297 kB 00:00 2024-02-14T14:49:52.834 INFO:teuthology.orchestra.run.smithi076.stdout:(24/173): python3-rgw-19.0.0-1280.ga6fe3fa3.el9 1.9 MB/s | 99 kB 00:00 2024-02-14T14:49:52.901 INFO:teuthology.orchestra.run.smithi076.stdout:(25/173): rbd-fuse-19.0.0-1280.ga6fe3fa3.el9.x8 1.3 MB/s | 85 kB 00:00 2024-02-14T14:49:53.082 INFO:teuthology.orchestra.run.smithi050.stdout:(1/173): ceph-19.0.0-1280.ga6fe3fa3.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-02-14T14:49:53.259 INFO:teuthology.orchestra.run.smithi076.stdout:(26/173): rbd-mirror-19.0.0-1280.ga6fe3fa3.el9. 8.6 MB/s | 3.1 MB 00:00 2024-02-14T14:49:53.334 INFO:teuthology.orchestra.run.smithi076.stdout:(27/173): rbd-nbd-19.0.0-1280.ga6fe3fa3.el9.x86 2.2 MB/s | 171 kB 00:00 2024-02-14T14:49:53.435 INFO:teuthology.orchestra.run.smithi076.stdout:(28/173): ceph-grafana-dashboards-19.0.0-1280.g 285 kB/s | 28 kB 00:00 2024-02-14T14:49:53.510 INFO:teuthology.orchestra.run.smithi076.stdout:(29/173): ceph-mgr-cephadm-19.0.0-1280.ga6fe3fa 1.8 MB/s | 138 kB 00:00 2024-02-14T14:49:53.769 INFO:teuthology.orchestra.run.smithi076.stdout:(30/173): ceph-mgr-dashboard-19.0.0-1280.ga6fe3 6.8 MB/s | 1.8 MB 00:00 2024-02-14T14:49:54.199 INFO:teuthology.orchestra.run.smithi050.stdout:(2/173): ceph-fuse-19.0.0-1280.ga6fe3fa3.el9.x8 794 kB/s | 886 kB 00:01 2024-02-14T14:49:54.382 INFO:teuthology.orchestra.run.smithi050.stdout:(3/173): ceph-immutable-object-cache-19.0.0-128 792 kB/s | 145 kB 00:00 2024-02-14T14:49:54.429 INFO:teuthology.orchestra.run.smithi017.stdout:(2/173): ceph-base-19.0.0-1280.ga6fe3fa3.el9.x8 2.5 MB/s | 5.1 MB 00:02 2024-02-14T14:49:54.496 INFO:teuthology.orchestra.run.smithi017.stdout:(3/173): ceph-immutable-object-cache-19.0.0-128 2.1 MB/s | 145 kB 00:00 2024-02-14T14:49:54.546 INFO:teuthology.orchestra.run.smithi017.stdout:(4/173): ceph-fuse-19.0.0-1280.ga6fe3fa3.el9.x8 490 kB/s | 886 kB 00:01 2024-02-14T14:49:54.761 INFO:teuthology.orchestra.run.smithi076.stdout:(31/173): ceph-test-19.0.0-1280.ga6fe3fa3.el9.x 14 MB/s | 47 MB 00:03 2024-02-14T14:49:54.869 INFO:teuthology.orchestra.run.smithi076.stdout:(32/173): ceph-mgr-diskprediction-local-19.0.0- 6.7 MB/s | 7.4 MB 00:01 2024-02-14T14:49:54.894 INFO:teuthology.orchestra.run.smithi076.stdout:(33/173): ceph-mgr-modules-core-19.0.0-1280.ga6 1.8 MB/s | 247 kB 00:00 2024-02-14T14:49:54.896 INFO:teuthology.orchestra.run.smithi017.stdout:(5/173): ceph-mds-19.0.0-1280.ga6fe3fa3.el9.x86 5.3 MB/s | 2.1 MB 00:00 2024-02-14T14:49:54.920 INFO:teuthology.orchestra.run.smithi076.stdout:(34/173): ceph-mgr-rook-19.0.0-1280.ga6fe3fa3.e 1.0 MB/s | 52 kB 00:00 2024-02-14T14:49:54.944 INFO:teuthology.orchestra.run.smithi076.stdout:(35/173): ceph-prometheus-alerts-19.0.0-1280.ga 306 kB/s | 15 kB 00:00 2024-02-14T14:49:54.978 INFO:teuthology.orchestra.run.smithi076.stdout:(36/173): ceph-volume-19.0.0-1280.ga6fe3fa3.el9 4.4 MB/s | 263 kB 00:00 2024-02-14T14:49:55.032 INFO:teuthology.orchestra.run.smithi050.stdout:(4/173): ceph-base-19.0.0-1280.ga6fe3fa3.el9.x8 2.2 MB/s | 5.1 MB 00:02 2024-02-14T14:49:55.053 INFO:teuthology.orchestra.run.smithi076.stdout:(37/173): cephadm-19.0.0-1280.ga6fe3fa3.el9.noa 6.7 MB/s | 748 kB 00:00 2024-02-14T14:49:55.070 INFO:teuthology.orchestra.run.smithi076.stdout:(38/173): cephfs-top-19.0.0-1280.ga6fe3fa3.el9. 199 kB/s | 18 kB 00:00 2024-02-14T14:49:55.191 INFO:teuthology.orchestra.run.smithi050.stdout:(5/173): ceph-mds-19.0.0-1280.ga6fe3fa3.el9.x86 2.6 MB/s | 2.1 MB 00:00 2024-02-14T14:49:55.312 INFO:teuthology.orchestra.run.smithi076.stdout:(39/173): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2024-02-14T14:49:55.388 INFO:teuthology.orchestra.run.smithi076.stdout:(40/173): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-02-14T14:49:55.455 INFO:teuthology.orchestra.run.smithi076.stdout:(41/173): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 920 kB 00:00 2024-02-14T14:49:55.472 INFO:teuthology.orchestra.run.smithi076.stdout:(42/173): elfutils-libelf-devel-0.190-2.el9.x86 245 kB/s | 20 kB 00:00 2024-02-14T14:49:55.474 INFO:teuthology.orchestra.run.smithi050.stdout:(6/173): ceph-mgr-19.0.0-1280.ga6fe3fa3.el9.x86 3.4 MB/s | 1.5 MB 00:00 2024-02-14T14:49:55.522 INFO:teuthology.orchestra.run.smithi076.stdout:(43/173): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-02-14T14:49:55.539 INFO:teuthology.orchestra.run.smithi076.stdout:(44/173): flexiblas-3.0.4-8.el9.x86_64.rpm 473 kB/s | 32 kB 00:00 2024-02-14T14:49:55.631 INFO:teuthology.orchestra.run.smithi076.stdout:(45/173): flexiblas-openblas-openmp-3.0.4-8.el9 187 kB/s | 17 kB 00:00 2024-02-14T14:49:55.713 INFO:teuthology.orchestra.run.smithi017.stdout:(6/173): ceph-mon-19.0.0-1280.ga6fe3fa3.el9.x86 5.7 MB/s | 4.7 MB 00:00 2024-02-14T14:49:55.739 INFO:teuthology.orchestra.run.smithi076.stdout:(46/173): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-02-14T14:49:55.756 INFO:teuthology.orchestra.run.smithi076.stdout:(47/173): libaio-devel-0.3.111-13.el9.x86_64.rp 81 kB/s | 10 kB 00:00 2024-02-14T14:49:55.807 INFO:teuthology.orchestra.run.smithi076.stdout:(48/173): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-02-14T14:49:55.823 INFO:teuthology.orchestra.run.smithi076.stdout:(49/173): librabbitmq-0.11.0-7.el9.x86_64.rpm 660 kB/s | 44 kB 00:00 2024-02-14T14:49:55.890 INFO:teuthology.orchestra.run.smithi076.stdout:(50/173): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-02-14T14:49:55.924 INFO:teuthology.orchestra.run.smithi076.stdout:(51/173): libtool-2.4.6-45.el9.x86_64.rpm 5.6 MB/s | 578 kB 00:00 2024-02-14T14:49:55.958 INFO:teuthology.orchestra.run.smithi076.stdout:(52/173): libuuid-devel-2.37.4-18.el9.x86_64.rp 370 kB/s | 24 kB 00:00 2024-02-14T14:49:55.983 INFO:teuthology.orchestra.run.smithi076.stdout:(53/173): libzstd-devel-1.5.1-2.el9.x86_64.rpm 793 kB/s | 46 kB 00:00 2024-02-14T14:49:56.016 INFO:teuthology.orchestra.run.smithi076.stdout:(54/173): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-02-14T14:49:56.042 INFO:teuthology.orchestra.run.smithi076.stdout:(55/173): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-02-14T14:49:56.067 INFO:teuthology.orchestra.run.smithi076.stdout:(56/173): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-02-14T14:49:56.351 INFO:teuthology.orchestra.run.smithi076.stdout:(57/173): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-02-14T14:49:56.468 INFO:teuthology.orchestra.run.smithi076.stdout:(58/173): openssl-devel-3.0.7-25.el9.x86_64.rpm 7.4 MB/s | 3.0 MB 00:00 2024-02-14T14:49:56.588 INFO:teuthology.orchestra.run.smithi017.stdout:(7/173): ceph-mgr-19.0.0-1280.ga6fe3fa3.el9.x86 745 kB/s | 1.5 MB 00:02 2024-02-14T14:49:56.601 INFO:teuthology.orchestra.run.smithi076.stdout:(59/173): postgresql-13.11-1.el9.x86_64.rpm 6.1 MB/s | 1.5 MB 00:00 2024-02-14T14:49:56.625 INFO:teuthology.orchestra.run.smithi050.stdout:(7/173): ceph-mon-19.0.0-1280.ga6fe3fa3.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-02-14T14:49:56.634 INFO:teuthology.orchestra.run.smithi076.stdout:(60/173): postgresql-contrib-13.11-1.el9.x86_64 4.8 MB/s | 811 kB 00:00 2024-02-14T14:49:56.660 INFO:teuthology.orchestra.run.smithi076.stdout:(61/173): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-02-14T14:49:56.977 INFO:teuthology.orchestra.run.smithi076.stdout:(62/173): postgresql-server-13.11-1.el9.x86_64. 17 MB/s | 5.7 MB 00:00 2024-02-14T14:49:57.010 INFO:teuthology.orchestra.run.smithi076.stdout:(63/173): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-02-14T14:49:57.036 INFO:teuthology.orchestra.run.smithi076.stdout:(64/173): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-02-14T14:49:57.386 INFO:teuthology.orchestra.run.smithi076.stdout:(65/173): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2024-02-14T14:49:57.411 INFO:teuthology.orchestra.run.smithi076.stdout:(66/173): python3-jinja2-2.11.3-5.el9.noarch.rp 605 kB/s | 227 kB 00:00 2024-02-14T14:49:57.453 INFO:teuthology.orchestra.run.smithi076.stdout:(67/173): python3-jsonpointer-2.0-4.el9.noarch. 437 kB/s | 18 kB 00:00 2024-02-14T14:49:57.478 INFO:teuthology.orchestra.run.smithi076.stdout:(68/173): python3-jsonpatch-1.21-16.el9.noarch. 366 kB/s | 24 kB 00:00 2024-02-14T14:49:57.503 INFO:teuthology.orchestra.run.smithi076.stdout:(69/173): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-14T14:49:57.513 INFO:teuthology.orchestra.run.smithi017.stdout:(8/173): ceph-common-19.0.0-1280.ga6fe3fa3.el9. 3.9 MB/s | 20 MB 00:05 2024-02-14T14:49:57.528 INFO:teuthology.orchestra.run.smithi076.stdout:(70/173): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-02-14T14:49:57.572 INFO:teuthology.orchestra.run.smithi017.stdout:(9/173): ceph-selinux-19.0.0-1280.ga6fe3fa3.el9 434 kB/s | 25 kB 00:00 2024-02-14T14:49:57.763 INFO:teuthology.orchestra.run.smithi076.stdout:(71/173): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-02-14T14:49:57.788 INFO:teuthology.orchestra.run.smithi076.stdout:(72/173): python3-numpy-f2py-1.20.1-5.el9.x86_6 842 kB/s | 217 kB 00:00 2024-02-14T14:49:57.813 INFO:teuthology.orchestra.run.smithi076.stdout:(73/173): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-14T14:49:57.838 INFO:teuthology.orchestra.run.smithi076.stdout:(74/173): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-14T14:49:57.856 INFO:teuthology.orchestra.run.smithi017.stdout:(10/173): ceph-osd-19.0.0-1280.ga6fe3fa3.el9.x8 7.8 MB/s | 17 MB 00:02 2024-02-14T14:49:57.864 INFO:teuthology.orchestra.run.smithi076.stdout:(75/173): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-02-14T14:49:57.889 INFO:teuthology.orchestra.run.smithi076.stdout:(76/173): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-02-14T14:49:57.906 INFO:teuthology.orchestra.run.smithi017.stdout:(11/173): cephfs-mirror-19.0.0-1280.ga6fe3fa3.e 4.3 MB/s | 217 kB 00:00 2024-02-14T14:49:57.914 INFO:teuthology.orchestra.run.smithi076.stdout:(77/173): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-14T14:49:57.940 INFO:teuthology.orchestra.run.smithi076.stdout:(78/173): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-14T14:49:57.956 INFO:teuthology.orchestra.run.smithi017.stdout:(12/173): cephfs-shell-19.0.0-1280.ga6fe3fa3.el 544 kB/s | 27 kB 00:00 2024-02-14T14:49:57.956 INFO:teuthology.orchestra.run.smithi076.stdout:(79/173): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-02-14T14:49:57.982 INFO:teuthology.orchestra.run.smithi076.stdout:(80/173): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-02-14T14:49:57.998 INFO:teuthology.orchestra.run.smithi017.stdout:(13/173): libcephfs-devel-19.0.0-1280.ga6fe3fa3 748 kB/s | 31 kB 00:00 2024-02-14T14:49:58.107 INFO:teuthology.orchestra.run.smithi017.stdout:(14/173): libcephfs2-19.0.0-1280.ga6fe3fa3.el9. 6.7 MB/s | 743 kB 00:00 2024-02-14T14:49:58.157 INFO:teuthology.orchestra.run.smithi017.stdout:(15/173): libcephsqlite-19.0.0-1280.ga6fe3fa3.e 3.2 MB/s | 163 kB 00:00 2024-02-14T14:49:58.183 INFO:teuthology.orchestra.run.smithi050.stdout:(8/173): ceph-osd-19.0.0-1280.ga6fe3fa3.el9.x86 6.2 MB/s | 17 MB 00:02 2024-02-14T14:49:58.207 INFO:teuthology.orchestra.run.smithi017.stdout:(16/173): librados-devel-19.0.0-1280.ga6fe3fa3. 2.5 MB/s | 127 kB 00:00 2024-02-14T14:49:58.225 INFO:teuthology.orchestra.run.smithi050.stdout:(9/173): ceph-selinux-19.0.0-1280.ga6fe3fa3.el9 609 kB/s | 25 kB 00:00 2024-02-14T14:49:58.248 INFO:teuthology.orchestra.run.smithi076.stdout:(81/173): python3-toml-0.10.2-6.el9.noarch.rpm 145 kB/s | 39 kB 00:00 2024-02-14T14:49:58.291 INFO:teuthology.orchestra.run.smithi017.stdout:(17/173): libradosstriper1-19.0.0-1280.ga6fe3fa 5.9 MB/s | 499 kB 00:00 2024-02-14T14:49:58.441 INFO:teuthology.orchestra.run.smithi076.stdout:(82/173): python3-wcwidth-0.2.5-8.el9.noarch.rp 215 kB/s | 41 kB 00:00 2024-02-14T14:49:58.607 INFO:teuthology.orchestra.run.smithi076.stdout:(83/173): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-02-14T14:49:58.633 INFO:teuthology.orchestra.run.smithi076.stdout:(84/173): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-02-14T14:49:58.650 INFO:teuthology.orchestra.run.smithi076.stdout:(85/173): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-02-14T14:49:58.675 INFO:teuthology.orchestra.run.smithi076.stdout:(86/173): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-02-14T14:49:58.692 INFO:teuthology.orchestra.run.smithi076.stdout:(87/173): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-14T14:49:58.708 INFO:teuthology.orchestra.run.smithi076.stdout:(88/173): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-02-14T14:49:58.734 INFO:teuthology.orchestra.run.smithi076.stdout:(89/173): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-02-14T14:49:58.764 INFO:teuthology.orchestra.run.smithi076.stdout:(90/173): jq-1.6-16.el9.x86_64.rpm 3.3 MB/s | 186 kB 00:00 2024-02-14T14:49:58.814 INFO:teuthology.orchestra.run.smithi076.stdout:(91/173): libgfortran-11.4.1-3.el9.x86_64.rpm 9.9 MB/s | 802 kB 00:00 2024-02-14T14:49:58.840 INFO:teuthology.orchestra.run.smithi076.stdout:(92/173): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-02-14T14:49:58.858 INFO:teuthology.orchestra.run.smithi017.stdout:(18/173): librgw2-19.0.0-1280.ga6fe3fa3.el9.x86 9.0 MB/s | 5.1 MB 00:00 2024-02-14T14:49:58.865 INFO:teuthology.orchestra.run.smithi076.stdout:(93/173): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-02-14T14:49:58.890 INFO:teuthology.orchestra.run.smithi076.stdout:(94/173): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-14T14:49:58.900 INFO:teuthology.orchestra.run.smithi017.stdout:(19/173): python3-ceph-argparse-19.0.0-1280.ga6 1.1 MB/s | 45 kB 00:00 2024-02-14T14:49:58.941 INFO:teuthology.orchestra.run.smithi017.stdout:(20/173): python3-ceph-common-19.0.0-1280.ga6fe 3.1 MB/s | 131 kB 00:00 2024-02-14T14:49:58.966 INFO:teuthology.orchestra.run.smithi076.stdout:(95/173): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-02-14T14:49:58.991 INFO:teuthology.orchestra.run.smithi076.stdout:(96/173): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-02-14T14:49:58.992 INFO:teuthology.orchestra.run.smithi017.stdout:(21/173): python3-cephfs-19.0.0-1280.ga6fe3fa3. 3.2 MB/s | 161 kB 00:00 2024-02-14T14:49:59.024 INFO:teuthology.orchestra.run.smithi076.stdout:(97/173): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-02-14T14:49:59.050 INFO:teuthology.orchestra.run.smithi076.stdout:(98/173): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 113 kB 00:00 2024-02-14T14:49:59.059 INFO:teuthology.orchestra.run.smithi017.stdout:(22/173): python3-rados-19.0.0-1280.ga6fe3fa3.e 4.7 MB/s | 321 kB 00:00 2024-02-14T14:49:59.083 INFO:teuthology.orchestra.run.smithi076.stdout:(99/173): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-02-14T14:49:59.126 INFO:teuthology.orchestra.run.smithi017.stdout:(23/173): python3-rbd-19.0.0-1280.ga6fe3fa3.el9 4.4 MB/s | 297 kB 00:00 2024-02-14T14:49:59.176 INFO:teuthology.orchestra.run.smithi017.stdout:(24/173): python3-rgw-19.0.0-1280.ga6fe3fa3.el9 1.9 MB/s | 99 kB 00:00 2024-02-14T14:49:59.217 INFO:teuthology.orchestra.run.smithi076.stdout:(100/173): python3-devel-3.9.17-2.el9.x86_64.rp 1.4 MB/s | 245 kB 00:00 2024-02-14T14:49:59.226 INFO:teuthology.orchestra.run.smithi017.stdout:(25/173): rbd-fuse-19.0.0-1280.ga6fe3fa3.el9.x8 1.7 MB/s | 85 kB 00:00 2024-02-14T14:49:59.610 INFO:teuthology.orchestra.run.smithi017.stdout:(26/173): rbd-mirror-19.0.0-1280.ga6fe3fa3.el9. 8.1 MB/s | 3.1 MB 00:00 2024-02-14T14:49:59.660 INFO:teuthology.orchestra.run.smithi017.stdout:(27/173): rbd-nbd-19.0.0-1280.ga6fe3fa3.el9.x86 3.4 MB/s | 171 kB 00:00 2024-02-14T14:49:59.702 INFO:teuthology.orchestra.run.smithi017.stdout:(28/173): ceph-grafana-dashboards-19.0.0-1280.g 683 kB/s | 28 kB 00:00 2024-02-14T14:49:59.717 INFO:teuthology.orchestra.run.smithi076.stdout:(101/173): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-02-14T14:49:59.744 INFO:teuthology.orchestra.run.smithi017.stdout:(29/173): ceph-mgr-cephadm-19.0.0-1280.ga6fe3fa 3.2 MB/s | 138 kB 00:00 2024-02-14T14:49:59.969 INFO:teuthology.orchestra.run.smithi017.stdout:(30/173): ceph-mgr-dashboard-19.0.0-1280.ga6fe3 7.8 MB/s | 1.8 MB 00:00 2024-02-14T14:50:00.367 INFO:teuthology.orchestra.run.smithi076.stdout:(102/173): protobuf-compiler-3.14.0-13.el9.x86_ 751 kB/s | 863 kB 00:01 2024-02-14T14:50:00.492 INFO:teuthology.orchestra.run.smithi050.stdout:(10/173): ceph-common-19.0.0-1280.ga6fe3fa3.el9 2.5 MB/s | 20 MB 00:07 2024-02-14T14:50:00.642 INFO:teuthology.orchestra.run.smithi050.stdout:(11/173): cephfs-mirror-19.0.0-1280.ga6fe3fa3.e 1.4 MB/s | 217 kB 00:00 2024-02-14T14:50:00.736 INFO:teuthology.orchestra.run.smithi017.stdout:(31/173): ceph-mgr-diskprediction-local-19.0.0- 9.7 MB/s | 7.4 MB 00:00 2024-02-14T14:50:00.742 INFO:teuthology.orchestra.run.smithi050.stdout:(12/173): cephfs-shell-19.0.0-1280.ga6fe3fa3.el 270 kB/s | 27 kB 00:00 2024-02-14T14:50:00.786 INFO:teuthology.orchestra.run.smithi017.stdout:(32/173): ceph-mgr-modules-core-19.0.0-1280.ga6 4.9 MB/s | 247 kB 00:00 2024-02-14T14:50:00.801 INFO:teuthology.orchestra.run.smithi076.stdout:(103/173): abseil-cpp-20211102.0-3.el9.x86_64.r 506 kB/s | 548 kB 00:01 2024-02-14T14:50:00.836 INFO:teuthology.orchestra.run.smithi017.stdout:(33/173): ceph-mgr-rook-19.0.0-1280.ga6fe3fa3.e 1.0 MB/s | 52 kB 00:00 2024-02-14T14:50:00.851 INFO:teuthology.orchestra.run.smithi050.stdout:(13/173): libcephfs-devel-19.0.0-1280.ga6fe3fa3 288 kB/s | 31 kB 00:00 2024-02-14T14:50:00.878 INFO:teuthology.orchestra.run.smithi017.stdout:(34/173): ceph-prometheus-alerts-19.0.0-1280.ga 368 kB/s | 15 kB 00:00 2024-02-14T14:50:00.928 INFO:teuthology.orchestra.run.smithi017.stdout:(35/173): ceph-volume-19.0.0-1280.ga6fe3fa3.el9 5.2 MB/s | 263 kB 00:00 2024-02-14T14:50:00.959 INFO:teuthology.orchestra.run.smithi076.stdout:(104/173): grpc-data-1.46.7-10.el9.noarch.rpm 123 kB/s | 19 kB 00:00 2024-02-14T14:50:01.018 INFO:teuthology.orchestra.run.smithi050.stdout:(14/173): ceph-radosgw-19.0.0-1280.ga6fe3fa3.el 2.2 MB/s | 9.7 MB 00:04 2024-02-14T14:50:01.020 INFO:teuthology.orchestra.run.smithi017.stdout:(36/173): cephadm-19.0.0-1280.ga6fe3fa3.el9.noa 8.0 MB/s | 748 kB 00:00 2024-02-14T14:50:01.071 INFO:teuthology.orchestra.run.smithi017.stdout:(37/173): cephfs-top-19.0.0-1280.ga6fe3fa3.el9. 367 kB/s | 18 kB 00:00 2024-02-14T14:50:01.168 INFO:teuthology.orchestra.run.smithi076.stdout:(105/173): gperftools-libs-2.9.1-2.el9.x86_64.r 386 kB/s | 309 kB 00:00 2024-02-14T14:50:01.285 INFO:teuthology.orchestra.run.smithi050.stdout:(15/173): libcephsqlite-19.0.0-1280.ga6fe3fa3.e 612 kB/s | 163 kB 00:00 2024-02-14T14:50:01.302 INFO:teuthology.orchestra.run.smithi076.stdout:(106/173): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-02-14T14:50:01.377 INFO:teuthology.orchestra.run.smithi050.stdout:(16/173): libcephfs2-19.0.0-1280.ga6fe3fa3.el9. 1.4 MB/s | 743 kB 00:00 2024-02-14T14:50:01.411 INFO:teuthology.orchestra.run.smithi050.stdout:(17/173): librados-devel-19.0.0-1280.ga6fe3fa3. 1.0 MB/s | 127 kB 00:00 2024-02-14T14:50:01.435 INFO:teuthology.orchestra.run.smithi076.stdout:(107/173): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-02-14T14:50:01.488 INFO:teuthology.orchestra.run.smithi017.stdout:(38/173): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-02-14T14:50:01.663 INFO:teuthology.orchestra.run.smithi017.stdout:(39/173): bison-3.7.4-5.el9.x86_64.rpm 5.1 MB/s | 920 kB 00:00 2024-02-14T14:50:01.670 INFO:teuthology.orchestra.run.smithi050.stdout:(18/173): libradosstriper1-19.0.0-1280.ga6fe3fa 1.7 MB/s | 499 kB 00:00 2024-02-14T14:50:01.722 INFO:teuthology.orchestra.run.smithi017.stdout:(40/173): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-02-14T14:50:01.745 INFO:teuthology.orchestra.run.smithi050.stdout:(19/173): python3-ceph-argparse-19.0.0-1280.ga6 603 kB/s | 45 kB 00:00 2024-02-14T14:50:01.772 INFO:teuthology.orchestra.run.smithi017.stdout:(41/173): elfutils-libelf-devel-0.190-2.el9.x86 409 kB/s | 20 kB 00:00 2024-02-14T14:50:01.839 INFO:teuthology.orchestra.run.smithi017.stdout:(42/173): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-02-14T14:50:01.878 INFO:teuthology.orchestra.run.smithi050.stdout:(20/173): python3-ceph-common-19.0.0-1280.ga6fe 985 kB/s | 131 kB 00:00 2024-02-14T14:50:01.889 INFO:teuthology.orchestra.run.smithi017.stdout:(43/173): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2024-02-14T14:50:01.919 INFO:teuthology.orchestra.run.smithi076.stdout:(108/173): libunwind-1.6.2-1.el9.x86_64.rpm 139 kB/s | 67 kB 00:00 2024-02-14T14:50:02.037 INFO:teuthology.orchestra.run.smithi050.stdout:(21/173): python3-cephfs-19.0.0-1280.ga6fe3fa3. 1.0 MB/s | 161 kB 00:00 2024-02-14T14:50:02.073 INFO:teuthology.orchestra.run.smithi017.stdout:(44/173): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-02-14T14:50:02.123 INFO:teuthology.orchestra.run.smithi017.stdout:(45/173): flexiblas-openblas-openmp-3.0.4-8.el9 343 kB/s | 17 kB 00:00 2024-02-14T14:50:02.152 INFO:teuthology.orchestra.run.smithi076.stdout:(109/173): luarocks-3.9.2-1.el9.noarch.rpm 648 kB/s | 151 kB 00:00 2024-02-14T14:50:02.173 INFO:teuthology.orchestra.run.smithi017.stdout:(46/173): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2024-02-14T14:50:02.232 INFO:teuthology.orchestra.run.smithi017.stdout:(47/173): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-02-14T14:50:02.237 INFO:teuthology.orchestra.run.smithi050.stdout:(22/173): python3-rados-19.0.0-1280.ga6fe3fa3.e 1.6 MB/s | 321 kB 00:00 2024-02-14T14:50:02.307 INFO:teuthology.orchestra.run.smithi017.stdout:(48/173): librabbitmq-0.11.0-7.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-02-14T14:50:02.374 INFO:teuthology.orchestra.run.smithi017.stdout:(49/173): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-02-14T14:50:02.429 INFO:teuthology.orchestra.run.smithi050.stdout:(23/173): python3-rbd-19.0.0-1280.ga6fe3fa3.el9 1.5 MB/s | 297 kB 00:00 2024-02-14T14:50:02.441 INFO:teuthology.orchestra.run.smithi017.stdout:(50/173): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-02-14T14:50:02.492 INFO:teuthology.orchestra.run.smithi017.stdout:(51/173): libuuid-devel-2.37.4-18.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-02-14T14:50:02.513 INFO:teuthology.orchestra.run.smithi050.stdout:(24/173): python3-rgw-19.0.0-1280.ga6fe3fa3.el9 1.2 MB/s | 99 kB 00:00 2024-02-14T14:50:02.550 INFO:teuthology.orchestra.run.smithi017.stdout:(52/173): libzstd-devel-1.5.1-2.el9.x86_64.rpm 796 kB/s | 46 kB 00:00 2024-02-14T14:50:02.597 INFO:teuthology.orchestra.run.smithi050.stdout:(25/173): rbd-fuse-19.0.0-1280.ga6fe3fa3.el9.x8 1.0 MB/s | 85 kB 00:00 2024-02-14T14:50:02.608 INFO:teuthology.orchestra.run.smithi017.stdout:(53/173): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-02-14T14:50:02.659 INFO:teuthology.orchestra.run.smithi017.stdout:(54/173): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-02-14T14:50:02.717 INFO:teuthology.orchestra.run.smithi017.stdout:(55/173): openblas-0.3.21-2.el9.x86_64.rpm 586 kB/s | 34 kB 00:00 2024-02-14T14:50:02.976 INFO:teuthology.orchestra.run.smithi017.stdout:(56/173): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-02-14T14:50:03.181 INFO:teuthology.orchestra.run.smithi050.stdout:(26/173): ceph-test-19.0.0-1280.ga6fe3fa3.el9.x 9.5 MB/s | 47 MB 00:04 2024-02-14T14:50:03.185 INFO:teuthology.orchestra.run.smithi017.stdout:(57/173): openssl-devel-3.0.7-25.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-02-14T14:50:03.247 INFO:teuthology.orchestra.run.smithi050.stdout:(27/173): rbd-nbd-19.0.0-1280.ga6fe3fa3.el9.x86 2.5 MB/s | 171 kB 00:00 2024-02-14T14:50:03.281 INFO:teuthology.orchestra.run.smithi050.stdout:(28/173): ceph-grafana-dashboards-19.0.0-1280.g 854 kB/s | 28 kB 00:00 2024-02-14T14:50:03.293 INFO:teuthology.orchestra.run.smithi017.stdout:(58/173): postgresql-13.11-1.el9.x86_64.rpm 14 MB/s | 1.5 MB 00:00 2024-02-14T14:50:03.331 INFO:teuthology.orchestra.run.smithi050.stdout:(29/173): ceph-mgr-cephadm-19.0.0-1280.ga6fe3fa 2.7 MB/s | 138 kB 00:00 2024-02-14T14:50:03.385 INFO:teuthology.orchestra.run.smithi017.stdout:(59/173): postgresql-contrib-13.11-1.el9.x86_64 8.7 MB/s | 811 kB 00:00 2024-02-14T14:50:03.461 INFO:teuthology.orchestra.run.smithi017.stdout:(60/173): postgresql-private-libs-13.11-1.el9.x 1.7 MB/s | 134 kB 00:00 2024-02-14T14:50:03.499 INFO:teuthology.orchestra.run.smithi050.stdout:(30/173): ceph-mgr-dashboard-19.0.0-1280.ga6fe3 11 MB/s | 1.8 MB 00:00 2024-02-14T14:50:03.553 INFO:teuthology.orchestra.run.smithi076.stdout:(110/173): parquet-libs-9.0.0-10.el9.x86_64.rpm 599 kB/s | 838 kB 00:01 2024-02-14T14:50:03.744 INFO:teuthology.orchestra.run.smithi017.stdout:(61/173): postgresql-server-13.11-1.el9.x86_64. 20 MB/s | 5.7 MB 00:00 2024-02-14T14:50:03.853 INFO:teuthology.orchestra.run.smithi017.stdout:(62/173): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-02-14T14:50:03.928 INFO:teuthology.orchestra.run.smithi017.stdout:(63/173): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 78 kB 00:00 2024-02-14T14:50:04.082 INFO:teuthology.orchestra.run.smithi050.stdout:(31/173): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-02-14T14:50:04.140 INFO:teuthology.orchestra.run.smithi050.stdout:(32/173): ceph-mgr-modules-core-19.0.0-1280.ga6 4.2 MB/s | 247 kB 00:00 2024-02-14T14:50:04.182 INFO:teuthology.orchestra.run.smithi050.stdout:(33/173): ceph-mgr-rook-19.0.0-1280.ga6fe3fa3.e 1.2 MB/s | 52 kB 00:00 2024-02-14T14:50:04.203 INFO:teuthology.orchestra.run.smithi017.stdout:(64/173): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-02-14T14:50:04.224 INFO:teuthology.orchestra.run.smithi050.stdout:(34/173): ceph-prometheus-alerts-19.0.0-1280.ga 367 kB/s | 15 kB 00:00 2024-02-14T14:50:04.253 INFO:teuthology.orchestra.run.smithi017.stdout:(65/173): python3-jinja2-2.11.3-5.el9.noarch.rp 4.5 MB/s | 227 kB 00:00 2024-02-14T14:50:04.283 INFO:teuthology.orchestra.run.smithi050.stdout:(35/173): ceph-volume-19.0.0-1280.ga6fe3fa3.el9 4.4 MB/s | 263 kB 00:00 2024-02-14T14:50:04.295 INFO:teuthology.orchestra.run.smithi017.stdout:(66/173): python3-jsonpatch-1.21-16.el9.noarch. 587 kB/s | 24 kB 00:00 2024-02-14T14:50:04.337 INFO:teuthology.orchestra.run.smithi017.stdout:(67/173): python3-jsonpointer-2.0-4.el9.noarch. 434 kB/s | 18 kB 00:00 2024-02-14T14:50:04.383 INFO:teuthology.orchestra.run.smithi050.stdout:(36/173): cephadm-19.0.0-1280.ga6fe3fa3.el9.noa 7.3 MB/s | 748 kB 00:00 2024-02-14T14:50:04.387 INFO:teuthology.orchestra.run.smithi017.stdout:(68/173): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-14T14:50:04.425 INFO:teuthology.orchestra.run.smithi050.stdout:(37/173): cephfs-top-19.0.0-1280.ga6fe3fa3.el9. 437 kB/s | 18 kB 00:00 2024-02-14T14:50:04.438 INFO:teuthology.orchestra.run.smithi017.stdout:(69/173): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-02-14T14:50:04.617 INFO:teuthology.orchestra.run.smithi050.stdout:(38/173): librgw2-19.0.0-1280.ga6fe3fa3.el9.x86 1.6 MB/s | 5.1 MB 00:03 2024-02-14T14:50:04.722 INFO:teuthology.orchestra.run.smithi017.stdout:(70/173): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-02-14T14:50:04.788 INFO:teuthology.orchestra.run.smithi017.stdout:(71/173): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 217 kB 00:00 2024-02-14T14:50:04.792 INFO:teuthology.orchestra.run.smithi050.stdout:(39/173): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-02-14T14:50:04.855 INFO:teuthology.orchestra.run.smithi017.stdout:(72/173): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-02-14T14:50:04.860 INFO:teuthology.orchestra.run.smithi050.stdout:(40/173): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2024-02-14T14:50:04.893 INFO:teuthology.orchestra.run.smithi050.stdout:(41/173): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 106 kB 00:00 2024-02-14T14:50:04.904 INFO:teuthology.orchestra.run.smithi076.stdout:(111/173): python3-asyncssh-2.13.2-3.el9.noarch 406 kB/s | 548 kB 00:01 2024-02-14T14:50:04.914 INFO:teuthology.orchestra.run.smithi017.stdout:(73/173): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-02-14T14:50:04.919 INFO:teuthology.orchestra.run.smithi050.stdout:(42/173): elfutils-libelf-devel-0.190-2.el9.x86 349 kB/s | 20 kB 00:00 2024-02-14T14:50:04.969 INFO:teuthology.orchestra.run.smithi050.stdout:(43/173): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-02-14T14:50:04.973 INFO:teuthology.orchestra.run.smithi017.stdout:(74/173): python3-prettytable-0.7.2-27.el9.noar 707 kB/s | 41 kB 00:00 2024-02-14T14:50:04.994 INFO:teuthology.orchestra.run.smithi050.stdout:(44/173): flexiblas-3.0.4-8.el9.x86_64.rpm 419 kB/s | 32 kB 00:00 2024-02-14T14:50:05.031 INFO:teuthology.orchestra.run.smithi017.stdout:(75/173): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-02-14T14:50:05.098 INFO:teuthology.orchestra.run.smithi017.stdout:(76/173): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-02-14T14:50:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:(112/173): python3-bcrypt-3.2.2-1.el9.x86_64.rp 209 kB/s | 43 kB 00:00 2024-02-14T14:50:05.111 INFO:teuthology.orchestra.run.smithi050.stdout:(45/173): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-02-14T14:50:05.137 INFO:teuthology.orchestra.run.smithi050.stdout:(46/173): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-02-14T14:50:05.187 INFO:teuthology.orchestra.run.smithi050.stdout:(47/173): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2024-02-14T14:50:05.198 INFO:teuthology.orchestra.run.smithi017.stdout:(77/173): ceph-radosgw-19.0.0-1280.ga6fe3fa3.el 1.1 MB/s | 9.7 MB 00:08 2024-02-14T14:50:05.212 INFO:teuthology.orchestra.run.smithi050.stdout:(48/173): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-02-14T14:50:05.224 INFO:teuthology.orchestra.run.smithi017.stdout:(78/173): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 211 kB 00:00 2024-02-14T14:50:05.237 INFO:teuthology.orchestra.run.smithi050.stdout:(49/173): librabbitmq-0.11.0-7.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-02-14T14:50:05.249 INFO:teuthology.orchestra.run.smithi017.stdout:(79/173): python3-pytz-2021.1-5.el9.noarch.rpm 958 kB/s | 47 kB 00:00 2024-02-14T14:50:05.261 INFO:teuthology.orchestra.run.smithi076.stdout:(113/173): python3-cachetools-4.2.4-1.el9.noarc 214 kB/s | 32 kB 00:00 2024-02-14T14:50:05.271 INFO:teuthology.orchestra.run.smithi050.stdout:(50/173): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-02-14T14:50:05.274 INFO:teuthology.orchestra.run.smithi017.stdout:(80/173): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-02-14T14:50:05.305 INFO:teuthology.orchestra.run.smithi050.stdout:(51/173): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-02-14T14:50:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:(52/173): libuuid-devel-2.37.4-18.el9.x86_64.rp 368 kB/s | 24 kB 00:00 2024-02-14T14:50:05.372 INFO:teuthology.orchestra.run.smithi050.stdout:(53/173): libzstd-devel-1.5.1-2.el9.x86_64.rpm 692 kB/s | 46 kB 00:00 2024-02-14T14:50:05.414 INFO:teuthology.orchestra.run.smithi050.stdout:(54/173): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-14T14:50:05.428 INFO:teuthology.orchestra.run.smithi076.stdout:(114/173): python3-certifi-2021.10.8-2.el9.noar 90 kB/s | 15 kB 00:00 2024-02-14T14:50:05.442 INFO:teuthology.orchestra.run.smithi050.stdout:(55/173): lua-5.4.4-4.el9.x86_64.rpm 2.6 MB/s | 187 kB 00:00 2024-02-14T14:50:05.476 INFO:teuthology.orchestra.run.smithi050.stdout:(56/173): openblas-0.3.21-2.el9.x86_64.rpm 556 kB/s | 34 kB 00:00 2024-02-14T14:50:05.677 INFO:teuthology.orchestra.run.smithi050.stdout:(57/173): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-02-14T14:50:05.727 INFO:teuthology.orchestra.run.smithi050.stdout:(58/173): openssl-devel-3.0.7-25.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-02-14T14:50:05.775 INFO:teuthology.orchestra.run.smithi017.stdout:(81/173): python3-toml-0.10.2-6.el9.noarch.rpm 77 kB/s | 39 kB 00:00 2024-02-14T14:50:05.836 INFO:teuthology.orchestra.run.smithi050.stdout:(59/173): postgresql-13.11-1.el9.x86_64.rpm 9.8 MB/s | 1.5 MB 00:00 2024-02-14T14:50:05.869 INFO:teuthology.orchestra.run.smithi050.stdout:(60/173): postgresql-contrib-13.11-1.el9.x86_64 5.6 MB/s | 811 kB 00:00 2024-02-14T14:50:05.887 INFO:teuthology.orchestra.run.smithi076.stdout:(115/173): libarrow-9.0.0-10.el9.x86_64.rpm 918 kB/s | 4.4 MB 00:04 2024-02-14T14:50:05.928 INFO:teuthology.orchestra.run.smithi050.stdout:(61/173): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-02-14T14:50:06.029 INFO:teuthology.orchestra.run.smithi076.stdout:(116/173): python3-cheroot-8.6.0-4.el9.noarch.r 286 kB/s | 172 kB 00:00 2024-02-14T14:50:06.128 INFO:teuthology.orchestra.run.smithi050.stdout:(62/173): postgresql-server-13.11-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-02-14T14:50:06.162 INFO:teuthology.orchestra.run.smithi050.stdout:(63/173): protobuf-3.14.0-13.el9.x86_64.rpm 4.4 MB/s | 1.0 MB 00:00 2024-02-14T14:50:06.187 INFO:teuthology.orchestra.run.smithi050.stdout:(64/173): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-02-14T14:50:06.200 INFO:teuthology.orchestra.run.smithi017.stdout:(82/173): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-02-14T14:50:06.225 INFO:teuthology.orchestra.run.smithi017.stdout:(83/173): python3-wcwidth-0.2.5-8.el9.noarch.rp 91 kB/s | 41 kB 00:00 2024-02-14T14:50:06.317 INFO:teuthology.orchestra.run.smithi017.stdout:(84/173): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 300 kB 00:00 2024-02-14T14:50:06.342 INFO:teuthology.orchestra.run.smithi017.stdout:(85/173): uuid-1.6.2-55.el9.x86_64.rpm 484 kB/s | 56 kB 00:00 2024-02-14T14:50:06.384 INFO:teuthology.orchestra.run.smithi017.stdout:(86/173): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2024-02-14T14:50:06.409 INFO:teuthology.orchestra.run.smithi017.stdout:(87/173): xmlstarlet-1.6.1-20.el9.x86_64.rpm 935 kB/s | 63 kB 00:00 2024-02-14T14:50:06.446 INFO:teuthology.orchestra.run.smithi050.stdout:(65/173): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-02-14T14:50:06.468 INFO:teuthology.orchestra.run.smithi017.stdout:(88/173): zlib-devel-1.2.11-40.el9.x86_64.rpm 528 kB/s | 44 kB 00:00 2024-02-14T14:50:06.471 INFO:teuthology.orchestra.run.smithi050.stdout:(66/173): python3-jinja2-2.11.3-5.el9.noarch.rp 802 kB/s | 227 kB 00:00 2024-02-14T14:50:06.488 INFO:teuthology.orchestra.run.smithi050.stdout:(67/173): python3-jsonpatch-1.21-16.el9.noarch. 594 kB/s | 24 kB 00:00 2024-02-14T14:50:06.493 INFO:teuthology.orchestra.run.smithi017.stdout:(89/173): fuse-2.9.9-15.el9.x86_64.rpm 945 kB/s | 79 kB 00:00 2024-02-14T14:50:06.512 INFO:teuthology.orchestra.run.smithi076.stdout:(117/173): python3-cherrypy-18.6.1-2.el9.noarch 573 kB/s | 358 kB 00:00 2024-02-14T14:50:06.513 INFO:teuthology.orchestra.run.smithi050.stdout:(68/173): python3-jsonpointer-2.0-4.el9.noarch. 432 kB/s | 18 kB 00:00 2024-02-14T14:50:06.518 INFO:teuthology.orchestra.run.smithi017.stdout:(90/173): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-14T14:50:06.539 INFO:teuthology.orchestra.run.smithi050.stdout:(69/173): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-14T14:50:06.564 INFO:teuthology.orchestra.run.smithi050.stdout:(70/173): python3-markupsafe-1.1.1-12.el9.x86_6 625 kB/s | 32 kB 00:00 2024-02-14T14:50:06.585 INFO:teuthology.orchestra.run.smithi017.stdout:(91/173): libgfortran-11.4.1-3.el9.x86_64.rpm 8.5 MB/s | 802 kB 00:00 2024-02-14T14:50:06.611 INFO:teuthology.orchestra.run.smithi017.stdout:(92/173): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 194 kB 00:00 2024-02-14T14:50:06.646 INFO:teuthology.orchestra.run.smithi076.stdout:(118/173): python3-colorama-0.4.4-12.el9.noarch 265 kB/s | 35 kB 00:00 2024-02-14T14:50:06.669 INFO:teuthology.orchestra.run.smithi017.stdout:(93/173): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-02-14T14:50:06.698 INFO:teuthology.orchestra.run.smithi050.stdout:(71/173): rbd-mirror-19.0.0-1280.ga6fe3fa3.el9. 771 kB/s | 3.1 MB 00:04 2024-02-14T14:50:06.761 INFO:teuthology.orchestra.run.smithi017.stdout:(94/173): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 241 kB 00:00 2024-02-14T14:50:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:(72/173): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-02-14T14:50:06.824 INFO:teuthology.orchestra.run.smithi050.stdout:(73/173): python3-numpy-f2py-1.20.1-5.el9.x86_6 840 kB/s | 217 kB 00:00 2024-02-14T14:50:06.836 INFO:teuthology.orchestra.run.smithi017.stdout:(95/173): python3-cryptography-36.0.1-4.el9.x86 6.9 MB/s | 1.1 MB 00:00 2024-02-14T14:50:06.846 INFO:teuthology.orchestra.run.smithi076.stdout:(119/173): python3-cmd2-2.4.2-2.el9.noarch.rpm 371 kB/s | 303 kB 00:00 2024-02-14T14:50:06.849 INFO:teuthology.orchestra.run.smithi050.stdout:(74/173): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-02-14T14:50:06.862 INFO:teuthology.orchestra.run.smithi017.stdout:(96/173): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-02-14T14:50:06.874 INFO:teuthology.orchestra.run.smithi050.stdout:(75/173): python3-prettytable-0.7.2-27.el9.noar 833 kB/s | 41 kB 00:00 2024-02-14T14:50:06.887 INFO:teuthology.orchestra.run.smithi017.stdout:(97/173): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-02-14T14:50:06.908 INFO:teuthology.orchestra.run.smithi050.stdout:(76/173): python3-packaging-20.9-5.el9.noarch.r 825 kB/s | 69 kB 00:00 2024-02-14T14:50:06.912 INFO:teuthology.orchestra.run.smithi017.stdout:(98/173): python3-requests-2.25.1-8.el9.noarch. 2.2 MB/s | 113 kB 00:00 2024-02-14T14:50:06.933 INFO:teuthology.orchestra.run.smithi050.stdout:(77/173): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-02-14T14:50:06.937 INFO:teuthology.orchestra.run.smithi017.stdout:(99/173): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-02-14T14:50:06.958 INFO:teuthology.orchestra.run.smithi050.stdout:(78/173): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-02-14T14:50:06.984 INFO:teuthology.orchestra.run.smithi050.stdout:(79/173): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-02-14T14:50:07.009 INFO:teuthology.orchestra.run.smithi050.stdout:(80/173): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-02-14T14:50:07.029 INFO:teuthology.orchestra.run.smithi017.stdout:(100/173): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-02-14T14:50:07.034 INFO:teuthology.orchestra.run.smithi050.stdout:(81/173): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2024-02-14T14:50:07.038 INFO:teuthology.orchestra.run.smithi076.stdout:(120/173): python3-google-auth-2.27.0-1.el9.noa 517 kB/s | 203 kB 00:00 2024-02-14T14:50:07.239 INFO:teuthology.orchestra.run.smithi017.stdout:(101/173): python3-devel-3.9.17-2.el9.x86_64.rp 755 kB/s | 245 kB 00:00 2024-02-14T14:50:07.372 INFO:teuthology.orchestra.run.smithi076.stdout:(121/173): python3-grpcio-tools-1.46.7-10.el9.x 434 kB/s | 144 kB 00:00 2024-02-14T14:50:07.380 INFO:teuthology.orchestra.run.smithi017.stdout:(102/173): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-02-14T14:50:07.472 INFO:teuthology.orchestra.run.smithi076.stdout:(122/173): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-02-14T14:50:07.526 INFO:teuthology.orchestra.run.smithi050.stdout:(82/173): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-02-14T14:50:07.551 INFO:teuthology.orchestra.run.smithi050.stdout:(83/173): python3-toml-0.10.2-6.el9.noarch.rpm 75 kB/s | 39 kB 00:00 2024-02-14T14:50:07.577 INFO:teuthology.orchestra.run.smithi050.stdout:(84/173): python3-wcwidth-0.2.5-8.el9.noarch.rp 76 kB/s | 41 kB 00:00 2024-02-14T14:50:07.602 INFO:teuthology.orchestra.run.smithi050.stdout:(85/173): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-02-14T14:50:07.614 INFO:teuthology.orchestra.run.smithi076.stdout:(123/173): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2024-02-14T14:50:07.627 INFO:teuthology.orchestra.run.smithi050.stdout:(86/173): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-02-14T14:50:07.653 INFO:teuthology.orchestra.run.smithi050.stdout:(87/173): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2024-02-14T14:50:07.678 INFO:teuthology.orchestra.run.smithi050.stdout:(88/173): xmlstarlet-1.6.1-20.el9.x86_64.rpm 833 kB/s | 63 kB 00:00 2024-02-14T14:50:07.703 INFO:teuthology.orchestra.run.smithi050.stdout:(89/173): zlib-devel-1.2.11-40.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-02-14T14:50:07.723 INFO:teuthology.orchestra.run.smithi076.stdout:(124/173): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-02-14T14:50:07.728 INFO:teuthology.orchestra.run.smithi050.stdout:(90/173): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-02-14T14:50:07.753 INFO:teuthology.orchestra.run.smithi050.stdout:(91/173): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-02-14T14:50:07.779 INFO:teuthology.orchestra.run.smithi050.stdout:(92/173): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 802 kB 00:00 2024-02-14T14:50:07.804 INFO:teuthology.orchestra.run.smithi050.stdout:(93/173): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-02-14T14:50:07.829 INFO:teuthology.orchestra.run.smithi050.stdout:(94/173): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-02-14T14:50:07.840 INFO:teuthology.orchestra.run.smithi076.stdout:(125/173): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-02-14T14:50:07.854 INFO:teuthology.orchestra.run.smithi050.stdout:(95/173): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-02-14T14:50:07.897 INFO:teuthology.orchestra.run.smithi050.stdout:(96/173): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-02-14T14:50:07.922 INFO:teuthology.orchestra.run.smithi050.stdout:(97/173): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-02-14T14:50:07.947 INFO:teuthology.orchestra.run.smithi050.stdout:(98/173): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-02-14T14:50:07.957 INFO:teuthology.orchestra.run.smithi076.stdout:(126/173): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-02-14T14:50:07.973 INFO:teuthology.orchestra.run.smithi050.stdout:(99/173): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 113 kB 00:00 2024-02-14T14:50:07.998 INFO:teuthology.orchestra.run.smithi050.stdout:(100/173): python3-urllib3-1.26.5-5.el9.noarch. 3.6 MB/s | 187 kB 00:00 2024-02-14T14:50:08.164 INFO:teuthology.orchestra.run.smithi017.stdout:(103/173): abseil-cpp-20211102.0-3.el9.x86_64.r 592 kB/s | 548 kB 00:00 2024-02-14T14:50:08.189 INFO:teuthology.orchestra.run.smithi017.stdout:(104/173): gperftools-libs-2.9.1-2.el9.x86_64.r 395 kB/s | 309 kB 00:00 2024-02-14T14:50:08.297 INFO:teuthology.orchestra.run.smithi017.stdout:(105/173): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-02-14T14:50:08.414 INFO:teuthology.orchestra.run.smithi017.stdout:(106/173): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-02-14T14:50:08.540 INFO:teuthology.orchestra.run.smithi017.stdout:(107/173): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-02-14T14:50:08.674 INFO:teuthology.orchestra.run.smithi017.stdout:(108/173): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-02-14T14:50:08.698 INFO:teuthology.orchestra.run.smithi050.stdout:(101/173): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-02-14T14:50:08.815 INFO:teuthology.orchestra.run.smithi017.stdout:(109/173): libarrow-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-02-14T14:50:08.990 INFO:teuthology.orchestra.run.smithi050.stdout:(102/173): python3-devel-3.9.17-2.el9.x86_64.rp 235 kB/s | 245 kB 00:01 2024-02-14T14:50:09.024 INFO:teuthology.orchestra.run.smithi017.stdout:(110/173): luarocks-3.9.2-1.el9.noarch.rpm 432 kB/s | 151 kB 00:00 2024-02-14T14:50:09.057 INFO:teuthology.orchestra.run.smithi017.stdout:(111/173): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-02-14T14:50:09.199 INFO:teuthology.orchestra.run.smithi017.stdout:(112/173): python3-bcrypt-3.2.2-1.el9.x86_64.rp 306 kB/s | 43 kB 00:00 2024-02-14T14:50:09.224 INFO:teuthology.orchestra.run.smithi076.stdout:(127/173): ceph-common-19.0.0-1280.ga6fe3fa3.el 1.0 MB/s | 20 MB 00:19 2024-02-14T14:50:09.241 INFO:teuthology.orchestra.run.smithi017.stdout:(113/173): python3-asyncssh-2.13.2-3.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-02-14T14:50:09.249 INFO:teuthology.orchestra.run.smithi050.stdout:(103/173): protobuf-compiler-3.14.0-13.el9.x86_ 690 kB/s | 863 kB 00:01 2024-02-14T14:50:09.317 INFO:teuthology.orchestra.run.smithi017.stdout:(114/173): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-02-14T14:50:09.350 INFO:teuthology.orchestra.run.smithi017.stdout:(115/173): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-02-14T14:50:09.415 INFO:teuthology.orchestra.run.smithi076.stdout:(128/173): python3-kubernetes-26.1.0-2.el9.noar 717 kB/s | 1.0 MB 00:01 2024-02-14T14:50:09.466 INFO:teuthology.orchestra.run.smithi050.stdout:(104/173): abseil-cpp-20211102.0-3.el9.x86_64.r 715 kB/s | 548 kB 00:00 2024-02-14T14:50:09.533 INFO:teuthology.orchestra.run.smithi050.stdout:(105/173): grpc-data-1.46.7-10.el9.noarch.rpm 69 kB/s | 19 kB 00:00 2024-02-14T14:50:09.557 INFO:teuthology.orchestra.run.smithi076.stdout:(129/173): python3-more-itertools-8.12.0-2.el9. 557 kB/s | 79 kB 00:00 2024-02-14T14:50:09.558 INFO:teuthology.orchestra.run.smithi017.stdout:(116/173): python3-cheroot-8.6.0-4.el9.noarch.r 710 kB/s | 172 kB 00:00 2024-02-14T14:50:09.584 INFO:teuthology.orchestra.run.smithi017.stdout:(117/173): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-02-14T14:50:09.591 INFO:teuthology.orchestra.run.smithi076.stdout:(130/173): python3-logutils-0.3.5-21.el9.noarch 126 kB/s | 46 kB 00:00 2024-02-14T14:50:09.666 INFO:teuthology.orchestra.run.smithi050.stdout:(106/173): gperftools-libs-2.9.1-2.el9.x86_64.r 457 kB/s | 309 kB 00:00 2024-02-14T14:50:09.683 INFO:teuthology.orchestra.run.smithi050.stdout:(107/173): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-02-14T14:50:09.699 INFO:teuthology.orchestra.run.smithi076.stdout:(131/173): python3-natsort-7.1.1-5.el9.noarch.r 406 kB/s | 58 kB 00:00 2024-02-14T14:50:09.734 INFO:teuthology.orchestra.run.smithi017.stdout:(118/173): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-02-14T14:50:09.759 INFO:teuthology.orchestra.run.smithi017.stdout:(119/173): python3-colorama-0.4.4-12.el9.noarch 201 kB/s | 35 kB 00:00 2024-02-14T14:50:09.805 INFO:teuthology.orchestra.run.smithi050.stdout:(108/173): liboath-2.6.7-2.el9.x86_64.rpm 350 kB/s | 49 kB 00:00 2024-02-14T14:50:09.816 INFO:teuthology.orchestra.run.smithi076.stdout:(132/173): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-02-14T14:50:09.885 INFO:teuthology.orchestra.run.smithi017.stdout:(120/173): python3-google-auth-2.27.0-1.el9.noa 1.3 MB/s | 203 kB 00:00 2024-02-14T14:50:09.939 INFO:teuthology.orchestra.run.smithi050.stdout:(109/173): libunwind-1.6.2-1.el9.x86_64.rpm 263 kB/s | 67 kB 00:00 2024-02-14T14:50:10.006 INFO:teuthology.orchestra.run.smithi050.stdout:(110/173): libarrow-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-02-14T14:50:10.031 INFO:teuthology.orchestra.run.smithi050.stdout:(111/173): luarocks-3.9.2-1.el9.noarch.rpm 670 kB/s | 151 kB 00:00 2024-02-14T14:50:10.033 INFO:teuthology.orchestra.run.smithi076.stdout:(133/173): python3-pyOpenSSL-21.0.0-1.el9.noarc 415 kB/s | 90 kB 00:00 2024-02-14T14:50:10.110 INFO:teuthology.orchestra.run.smithi017.stdout:(121/173): python3-grpcio-tools-1.46.7-10.el9.x 643 kB/s | 144 kB 00:00 2024-02-14T14:50:10.159 INFO:teuthology.orchestra.run.smithi076.stdout:(134/173): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-02-14T14:50:10.173 INFO:teuthology.orchestra.run.smithi050.stdout:(112/173): python3-asyncssh-2.13.2-3.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-02-14T14:50:10.190 INFO:teuthology.orchestra.run.smithi050.stdout:(113/173): python3-bcrypt-3.2.2-1.el9.x86_64.rp 274 kB/s | 43 kB 00:00 2024-02-14T14:50:10.192 INFO:teuthology.orchestra.run.smithi076.stdout:(135/173): python3-pecan-1.4.2-3.el9.noarch.rpm 453 kB/s | 272 kB 00:00 2024-02-14T14:50:10.193 INFO:teuthology.orchestra.run.smithi017.stdout:(122/173): python3-grpcio-1.46.7-10.el9.x86_64. 4.7 MB/s | 2.0 MB 00:00 2024-02-14T14:50:10.236 INFO:teuthology.orchestra.run.smithi017.stdout:(123/173): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-02-14T14:50:10.309 INFO:teuthology.orchestra.run.smithi076.stdout:(136/173): python3-repoze-lru-0.7-16.el9.noarch 205 kB/s | 31 kB 00:00 2024-02-14T14:50:10.311 INFO:teuthology.orchestra.run.smithi017.stdout:(124/173): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-02-14T14:50:10.361 INFO:teuthology.orchestra.run.smithi017.stdout:(125/173): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-02-14T14:50:10.391 INFO:teuthology.orchestra.run.smithi050.stdout:(114/173): python3-cachetools-4.2.4-1.el9.noarc 148 kB/s | 32 kB 00:00 2024-02-14T14:50:10.407 INFO:teuthology.orchestra.run.smithi050.stdout:(115/173): python3-certifi-2021.10.8-2.el9.noar 69 kB/s | 15 kB 00:00 2024-02-14T14:50:10.418 INFO:teuthology.orchestra.run.smithi076.stdout:(137/173): python3-grpcio-1.46.7-10.el9.x86_64. 584 kB/s | 2.0 MB 00:03 2024-02-14T14:50:10.436 INFO:teuthology.orchestra.run.smithi017.stdout:(126/173): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-02-14T14:50:10.449 INFO:teuthology.orchestra.run.smithi050.stdout:(116/173): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-02-14T14:50:10.460 INFO:teuthology.orchestra.run.smithi076.stdout:(138/173): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2024-02-14T14:50:10.487 INFO:teuthology.orchestra.run.smithi017.stdout:(127/173): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-02-14T14:50:10.525 INFO:teuthology.orchestra.run.smithi050.stdout:(117/173): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-02-14T14:50:10.552 INFO:teuthology.orchestra.run.smithi076.stdout:(139/173): python3-tempora-5.0.0-2.el9.noarch.r 268 kB/s | 36 kB 00:00 2024-02-14T14:50:10.604 INFO:teuthology.orchestra.run.smithi076.stdout:(140/173): python3-typing-extensions-4.4.0-2.el 355 kB/s | 51 kB 00:00 2024-02-14T14:50:10.617 INFO:teuthology.orchestra.run.smithi050.stdout:(118/173): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-02-14T14:50:10.629 INFO:teuthology.orchestra.run.smithi017.stdout:(128/173): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-02-14T14:50:10.629 INFO:teuthology.orchestra.run.smithi076.stdout:(141/173): python3-routes-2.5.1-5.el9.noarch.rp 432 kB/s | 188 kB 00:00 2024-02-14T14:50:10.634 INFO:teuthology.orchestra.run.smithi050.stdout:(119/173): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-02-14T14:50:10.654 INFO:teuthology.orchestra.run.smithi017.stdout:(129/173): python3-logutils-0.3.5-21.el9.noarch 277 kB/s | 46 kB 00:00 2024-02-14T14:50:10.659 INFO:teuthology.orchestra.run.smithi050.stdout:(120/173): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-02-14T14:50:10.759 INFO:teuthology.orchestra.run.smithi050.stdout:(121/173): python3-google-auth-2.27.0-1.el9.noa 1.4 MB/s | 203 kB 00:00 2024-02-14T14:50:10.771 INFO:teuthology.orchestra.run.smithi017.stdout:(130/173): python3-more-itertools-8.12.0-2.el9. 556 kB/s | 79 kB 00:00 2024-02-14T14:50:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:(131/173): python3-natsort-7.1.1-5.el9.noarch.r 384 kB/s | 58 kB 00:00 2024-02-14T14:50:10.826 INFO:teuthology.orchestra.run.smithi050.stdout:(122/173): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-02-14T14:50:10.846 INFO:teuthology.orchestra.run.smithi076.stdout:(142/173): python3-websocket-client-1.2.3-2.el9 370 kB/s | 90 kB 00:00 2024-02-14T14:50:10.902 INFO:teuthology.orchestra.run.smithi050.stdout:(123/173): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-02-14T14:50:10.921 INFO:teuthology.orchestra.run.smithi017.stdout:(132/173): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-02-14T14:50:10.927 INFO:teuthology.orchestra.run.smithi050.stdout:(124/173): python3-grpcio-tools-1.46.7-10.el9.x 539 kB/s | 144 kB 00:00 2024-02-14T14:50:10.938 INFO:teuthology.orchestra.run.smithi017.stdout:(133/173): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-02-14T14:50:10.952 INFO:teuthology.orchestra.run.smithi050.stdout:(125/173): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-02-14T14:50:10.979 INFO:teuthology.orchestra.run.smithi076.stdout:(143/173): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-02-14T14:50:11.005 INFO:teuthology.orchestra.run.smithi076.stdout:(144/173): python3-webob-1.8.7-6.el9.noarch.rpm 507 kB/s | 230 kB 00:00 2024-02-14T14:50:11.019 INFO:teuthology.orchestra.run.smithi050.stdout:(126/173): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-02-14T14:50:11.044 INFO:teuthology.orchestra.run.smithi050.stdout:(127/173): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-02-14T14:50:11.047 INFO:teuthology.orchestra.run.smithi017.stdout:(134/173): python3-pyOpenSSL-21.0.0-1.el9.noarc 718 kB/s | 90 kB 00:00 2024-02-14T14:50:11.074 INFO:teuthology.orchestra.run.smithi017.stdout:(135/173): python3-pyperclip-1.8.2-2.el9.noarch 211 kB/s | 28 kB 00:00 2024-02-14T14:50:11.111 INFO:teuthology.orchestra.run.smithi050.stdout:(128/173): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-02-14T14:50:11.164 INFO:teuthology.orchestra.run.smithi017.stdout:(136/173): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-02-14T14:50:11.186 INFO:teuthology.orchestra.run.smithi050.stdout:(129/173): python3-logutils-0.3.5-21.el9.noarch 326 kB/s | 46 kB 00:00 2024-02-14T14:50:11.206 INFO:teuthology.orchestra.run.smithi017.stdout:(137/173): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-02-14T14:50:11.245 INFO:teuthology.orchestra.run.smithi050.stdout:(130/173): python3-more-itertools-8.12.0-2.el9. 591 kB/s | 79 kB 00:00 2024-02-14T14:50:11.280 INFO:teuthology.orchestra.run.smithi076.stdout:(145/173): python3-werkzeug-2.0.3-3.el9.1.noarc 656 kB/s | 427 kB 00:00 2024-02-14T14:50:11.298 INFO:teuthology.orchestra.run.smithi017.stdout:(138/173): python3-rsa-4.9-2.el9.noarch.rpm 442 kB/s | 59 kB 00:00 2024-02-14T14:50:11.304 INFO:teuthology.orchestra.run.smithi050.stdout:(131/173): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-02-14T14:50:11.322 INFO:teuthology.orchestra.run.smithi076.stdout:(146/173): re2-20211101-3.el9.x86_64.rpm 560 kB/s | 192 kB 00:00 2024-02-14T14:50:11.332 INFO:teuthology.orchestra.run.smithi017.stdout:(139/173): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-02-14T14:50:11.345 INFO:teuthology.orchestra.run.smithi050.stdout:(132/173): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-02-14T14:50:11.388 INFO:teuthology.orchestra.run.smithi050.stdout:(133/173): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-02-14T14:50:11.424 INFO:teuthology.orchestra.run.smithi017.stdout:(140/173): python3-typing-extensions-4.4.0-2.el 407 kB/s | 51 kB 00:00 2024-02-14T14:50:11.465 INFO:teuthology.orchestra.run.smithi017.stdout:(141/173): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-02-14T14:50:11.572 INFO:teuthology.orchestra.run.smithi050.stdout:(134/173): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-02-14T14:50:11.597 INFO:teuthology.orchestra.run.smithi050.stdout:(135/173): python3-pyperclip-1.8.2-2.el9.noarch 135 kB/s | 28 kB 00:00 2024-02-14T14:50:11.622 INFO:teuthology.orchestra.run.smithi050.stdout:(136/173): python3-pyOpenSSL-21.0.0-1.el9.noarc 326 kB/s | 90 kB 00:00 2024-02-14T14:50:11.639 INFO:teuthology.orchestra.run.smithi076.stdout:(147/173): librados2-19.0.0-1280.ga6fe3fa3.el9. 9.4 MB/s | 3.4 MB 00:00 2024-02-14T14:50:11.651 INFO:teuthology.orchestra.run.smithi017.stdout:(142/173): python3-websocket-client-1.2.3-2.el9 394 kB/s | 90 kB 00:00 2024-02-14T14:50:11.689 INFO:teuthology.orchestra.run.smithi076.stdout:(148/173): librbd1-19.0.0-1280.ga6fe3fa3.el9.x8 8.5 MB/s | 3.1 MB 00:00 2024-02-14T14:50:11.705 INFO:teuthology.orchestra.run.smithi050.stdout:(137/173): python3-repoze-lru-0.7-16.el9.noarch 230 kB/s | 31 kB 00:00 2024-02-14T14:50:11.714 INFO:teuthology.orchestra.run.smithi076.stdout:(149/173): libacl-devel-2.3.1-4.el9.x86_64.rpm 900 kB/s | 68 kB 00:00 2024-02-14T14:50:11.731 INFO:teuthology.orchestra.run.smithi050.stdout:(138/173): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-02-14T14:50:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:(143/173): python3-werkzeug-2.0.3-3.el9.1.noarc 1.5 MB/s | 427 kB 00:00 2024-02-14T14:50:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:(139/173): python3-rsa-4.9-2.el9.noarch.rpm 442 kB/s | 59 kB 00:00 2024-02-14T14:50:11.766 INFO:teuthology.orchestra.run.smithi017.stdout:(144/173): python3-zc-lockfile-2.0-10.el9.noarc 174 kB/s | 20 kB 00:00 2024-02-14T14:50:11.848 INFO:teuthology.orchestra.run.smithi076.stdout:(150/173): NetworkManager-1.45.91-1.el9.x86_64. 14 MB/s | 2.3 MB 00:00 2024-02-14T14:50:11.848 INFO:teuthology.orchestra.run.smithi050.stdout:(140/173): python3-tempora-5.0.0-2.el9.noarch.r 251 kB/s | 36 kB 00:00 2024-02-14T14:50:11.873 INFO:teuthology.orchestra.run.smithi050.stdout:(141/173): python3-typing-extensions-4.4.0-2.el 359 kB/s | 51 kB 00:00 2024-02-14T14:50:11.892 INFO:teuthology.orchestra.run.smithi017.stdout:(145/173): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-02-14T14:50:11.906 INFO:teuthology.orchestra.run.smithi076.stdout:(151/173): NetworkManager-libnm-1.45.91-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-02-14T14:50:11.907 INFO:teuthology.orchestra.run.smithi050.stdout:(142/173): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-02-14T14:50:11.924 INFO:teuthology.orchestra.run.smithi076.stdout:(152/173): NetworkManager-team-1.45.91-1.el9.x8 566 kB/s | 41 kB 00:00 2024-02-14T14:50:11.967 INFO:teuthology.orchestra.run.smithi017.stdout:(146/173): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-02-14T14:50:11.974 INFO:teuthology.orchestra.run.smithi076.stdout:(153/173): NetworkManager-tui-1.45.91-1.el9.x86 3.6 MB/s | 246 kB 00:00 2024-02-14T14:50:11.982 INFO:teuthology.orchestra.run.smithi050.stdout:(143/173): python3-websocket-client-1.2.3-2.el9 670 kB/s | 90 kB 00:00 2024-02-14T14:50:11.999 INFO:teuthology.orchestra.run.smithi076.stdout:(154/173): acl-2.3.1-4.el9.x86_64.rpm 917 kB/s | 69 kB 00:00 2024-02-14T14:50:12.007 INFO:teuthology.orchestra.run.smithi050.stdout:(144/173): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-02-14T14:50:12.016 INFO:teuthology.orchestra.run.smithi076.stdout:(155/173): elfutils-debuginfod-client-0.190-2.e 848 kB/s | 35 kB 00:00 2024-02-14T14:50:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:(145/173): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-02-14T14:50:12.058 INFO:teuthology.orchestra.run.smithi076.stdout:(156/173): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 191 kB 00:00 2024-02-14T14:50:12.083 INFO:teuthology.orchestra.run.smithi076.stdout:(157/173): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 254 kB 00:00 2024-02-14T14:50:12.125 INFO:teuthology.orchestra.run.smithi050.stdout:(146/173): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-02-14T14:50:12.192 INFO:teuthology.orchestra.run.smithi076.stdout:(158/173): iproute-6.7.0-2.el9.x86_64.rpm 5.9 MB/s | 802 kB 00:00 2024-02-14T14:50:12.250 INFO:teuthology.orchestra.run.smithi076.stdout:(159/173): iproute-tc-6.7.0-2.el9.x86_64.rpm 2.5 MB/s | 433 kB 00:00 2024-02-14T14:50:12.267 INFO:teuthology.orchestra.run.smithi076.stdout:(160/173): libacl-2.3.1-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-02-14T14:50:12.309 INFO:teuthology.orchestra.run.smithi017.stdout:(147/173): librbd1-19.0.0-1280.ga6fe3fa3.el9.x8 9.2 MB/s | 3.1 MB 00:00 2024-02-14T14:50:12.343 INFO:teuthology.orchestra.run.smithi076.stdout:(161/173): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 107 kB 00:00 2024-02-14T14:50:12.368 INFO:teuthology.orchestra.run.smithi076.stdout:(162/173): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 154 kB 00:00 2024-02-14T14:50:12.376 INFO:teuthology.orchestra.run.smithi017.stdout:(148/173): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 68 kB 00:00 2024-02-14T14:50:12.402 INFO:teuthology.orchestra.run.smithi076.stdout:(163/173): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 135 kB 00:00 2024-02-14T14:50:12.408 INFO:teuthology.orchestra.run.smithi050.stdout:(147/173): thrift-0.15.0-2.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-02-14T14:50:12.419 INFO:teuthology.orchestra.run.smithi076.stdout:(164/173): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-14T14:50:12.469 INFO:teuthology.orchestra.run.smithi076.stdout:(165/173): libuuid-2.37.4-18.el9.x86_64.rpm 413 kB/s | 28 kB 00:00 2024-02-14T14:50:12.492 INFO:teuthology.orchestra.run.smithi050.stdout:(148/173): librbd1-19.0.0-1280.ga6fe3fa3.el9.x8 8.5 MB/s | 3.1 MB 00:00 2024-02-14T14:50:12.517 INFO:teuthology.orchestra.run.smithi050.stdout:(149/173): libacl-devel-2.3.1-4.el9.x86_64.rpm 623 kB/s | 68 kB 00:00 2024-02-14T14:50:12.528 INFO:teuthology.orchestra.run.smithi076.stdout:(166/173): openssl-3.0.7-25.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-02-14T14:50:12.560 INFO:teuthology.orchestra.run.smithi017.stdout:(149/173): NetworkManager-1.45.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-02-14T14:50:12.693 INFO:teuthology.orchestra.run.smithi050.stdout:(150/173): NetworkManager-1.45.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-02-14T14:50:12.695 INFO:teuthology.orchestra.run.smithi076.stdout:(167/173): openssl-libs-3.0.7-25.el9.x86_64.rpm 9.5 MB/s | 2.1 MB 00:00 2024-02-14T14:50:12.735 INFO:teuthology.orchestra.run.smithi050.stdout:(151/173): NetworkManager-libnm-1.45.91-1.el9.x 8.3 MB/s | 1.8 MB 00:00 2024-02-14T14:50:12.745 INFO:teuthology.orchestra.run.smithi076.stdout:(168/173): util-linux-2.37.4-18.el9.x86_64.rpm 10 MB/s | 2.2 MB 00:00 2024-02-14T14:50:12.751 INFO:teuthology.orchestra.run.smithi050.stdout:(152/173): NetworkManager-team-1.45.91-1.el9.x8 744 kB/s | 41 kB 00:00 2024-02-14T14:50:12.760 INFO:teuthology.orchestra.run.smithi017.stdout:(150/173): NetworkManager-libnm-1.45.91-1.el9.x 9.0 MB/s | 1.8 MB 00:00 2024-02-14T14:50:12.770 INFO:teuthology.orchestra.run.smithi076.stdout:(169/173): util-linux-core-2.37.4-18.el9.x86_64 5.9 MB/s | 438 kB 00:00 2024-02-14T14:50:12.794 INFO:teuthology.orchestra.run.smithi050.stdout:(153/173): NetworkManager-tui-1.45.91-1.el9.x86 4.1 MB/s | 246 kB 00:00 2024-02-14T14:50:12.804 INFO:teuthology.orchestra.run.smithi076.stdout:(170/173): util-linux-user-2.37.4-18.el9.x86_64 519 kB/s | 30 kB 00:00 2024-02-14T14:50:12.810 INFO:teuthology.orchestra.run.smithi017.stdout:(151/173): NetworkManager-team-1.45.91-1.el9.x8 832 kB/s | 41 kB 00:00 2024-02-14T14:50:12.819 INFO:teuthology.orchestra.run.smithi050.stdout:(154/173): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 69 kB 00:00 2024-02-14T14:50:12.835 INFO:teuthology.orchestra.run.smithi050.stdout:(155/173): elfutils-debuginfod-client-0.190-2.e 851 kB/s | 35 kB 00:00 2024-02-14T14:50:12.837 INFO:teuthology.orchestra.run.smithi076.stdout:(171/173): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-02-14T14:50:12.886 INFO:teuthology.orchestra.run.smithi017.stdout:(152/173): NetworkManager-tui-1.45.91-1.el9.x86 3.2 MB/s | 246 kB 00:00 2024-02-14T14:50:12.886 INFO:teuthology.orchestra.run.smithi050.stdout:(156/173): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 191 kB 00:00 2024-02-14T14:50:12.911 INFO:teuthology.orchestra.run.smithi050.stdout:(157/173): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 254 kB 00:00 2024-02-14T14:50:12.912 INFO:teuthology.orchestra.run.smithi076.stdout:(172/173): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.8 MB/s | 1.1 MB 00:00 2024-02-14T14:50:12.944 INFO:teuthology.orchestra.run.smithi017.stdout:(153/173): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2024-02-14T14:50:12.978 INFO:teuthology.orchestra.run.smithi050.stdout:(158/173): iproute-6.7.0-2.el9.x86_64.rpm 8.6 MB/s | 802 kB 00:00 2024-02-14T14:50:12.994 INFO:teuthology.orchestra.run.smithi017.stdout:(154/173): elfutils-debuginfod-client-0.190-2.e 707 kB/s | 35 kB 00:00 2024-02-14T14:50:13.003 INFO:teuthology.orchestra.run.smithi050.stdout:(159/173): iproute-tc-6.7.0-2.el9.x86_64.rpm 4.6 MB/s | 433 kB 00:00 2024-02-14T14:50:13.037 INFO:teuthology.orchestra.run.smithi050.stdout:(160/173): libacl-2.3.1-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-02-14T14:50:13.061 INFO:teuthology.orchestra.run.smithi017.stdout:(155/173): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 191 kB 00:00 2024-02-14T14:50:13.062 INFO:teuthology.orchestra.run.smithi050.stdout:(161/173): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 107 kB 00:00 2024-02-14T14:50:13.096 INFO:teuthology.orchestra.run.smithi050.stdout:(162/173): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 154 kB 00:00 2024-02-14T14:50:13.121 INFO:teuthology.orchestra.run.smithi050.stdout:(163/173): libmount-2.37.4-18.el9.x86_64.rpm 2.2 MB/s | 135 kB 00:00 2024-02-14T14:50:13.128 INFO:teuthology.orchestra.run.smithi017.stdout:(156/173): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 254 kB 00:00 2024-02-14T14:50:13.147 INFO:teuthology.orchestra.run.smithi050.stdout:(164/173): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-14T14:50:13.172 INFO:teuthology.orchestra.run.smithi050.stdout:(165/173): libuuid-2.37.4-18.el9.x86_64.rpm 551 kB/s | 28 kB 00:00 2024-02-14T14:50:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:(157/173): iproute-6.7.0-2.el9.x86_64.rpm 7.8 MB/s | 802 kB 00:00 2024-02-14T14:50:13.247 INFO:teuthology.orchestra.run.smithi050.stdout:(166/173): openssl-3.0.7-25.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-02-14T14:50:13.254 INFO:teuthology.orchestra.run.smithi076.stdout:(173/173): thrift-0.15.0-2.el9.x86_64.rpm 720 kB/s | 1.6 MB 00:02 2024-02-14T14:50:13.259 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-02-14T14:50:13.259 INFO:teuthology.orchestra.run.smithi076.stdout:Total 9.1 MB/s | 222 MB 00:24 2024-02-14T14:50:13.307 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-02-14T14:50:13.320 INFO:teuthology.orchestra.run.smithi017.stdout:(158/173): iproute-tc-6.7.0-2.el9.x86_64.rpm 4.6 MB/s | 433 kB 00:00 2024-02-14T14:50:13.347 INFO:teuthology.orchestra.run.smithi050.stdout:(167/173): openssl-libs-3.0.7-25.el9.x86_64.rpm 12 MB/s | 2.1 MB 00:00 2024-02-14T14:50:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:(159/173): libacl-2.3.1-4.el9.x86_64.rpm 331 kB/s | 22 kB 00:00 2024-02-14T14:50:13.439 INFO:teuthology.orchestra.run.smithi050.stdout:(168/173): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2024-02-14T14:50:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:(160/173): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 107 kB 00:00 2024-02-14T14:50:13.465 INFO:teuthology.orchestra.run.smithi050.stdout:(169/173): util-linux-core-2.37.4-18.el9.x86_64 3.8 MB/s | 438 kB 00:00 2024-02-14T14:50:13.490 INFO:teuthology.orchestra.run.smithi050.stdout:(170/173): util-linux-user-2.37.4-18.el9.x86_64 609 kB/s | 30 kB 00:00 2024-02-14T14:50:13.505 INFO:teuthology.orchestra.run.smithi017.stdout:(161/173): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 154 kB 00:00 2024-02-14T14:50:13.524 INFO:teuthology.orchestra.run.smithi050.stdout:(171/173): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-02-14T14:50:13.563 INFO:teuthology.orchestra.run.smithi017.stdout:(162/173): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 135 kB 00:00 2024-02-14T14:50:13.582 INFO:teuthology.orchestra.run.smithi050.stdout:(172/173): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-14T14:50:13.593 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-02-14T14:50:13.593 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-02-14T14:50:13.614 INFO:teuthology.orchestra.run.smithi017.stdout:(163/173): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-14T14:50:13.672 INFO:teuthology.orchestra.run.smithi017.stdout:(164/173): libuuid-2.37.4-18.el9.x86_64.rpm 474 kB/s | 28 kB 00:00 2024-02-14T14:50:13.798 INFO:teuthology.orchestra.run.smithi017.stdout:(165/173): openssl-3.0.7-25.el9.x86_64.rpm 9.2 MB/s | 1.2 MB 00:00 2024-02-14T14:50:13.998 INFO:teuthology.orchestra.run.smithi017.stdout:(166/173): openssl-libs-3.0.7-25.el9.x86_64.rpm 11 MB/s | 2.1 MB 00:00 2024-02-14T14:50:14.198 INFO:teuthology.orchestra.run.smithi017.stdout:(167/173): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-02-14T14:50:14.274 INFO:teuthology.orchestra.run.smithi017.stdout:(168/173): util-linux-core-2.37.4-18.el9.x86_64 5.7 MB/s | 438 kB 00:00 2024-02-14T14:50:14.382 INFO:teuthology.orchestra.run.smithi017.stdout:(169/173): librados2-19.0.0-1280.ga6fe3fa3.el9. 1.4 MB/s | 3.4 MB 00:02 2024-02-14T14:50:14.499 INFO:teuthology.orchestra.run.smithi017.stdout:(170/173): util-linux-user-2.37.4-18.el9.x86_64 133 kB/s | 30 kB 00:00 2024-02-14T14:50:14.541 INFO:teuthology.orchestra.run.smithi017.stdout:(171/173): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-02-14T14:50:14.633 INFO:teuthology.orchestra.run.smithi017.stdout:(172/173): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.9 MB/s | 1.1 MB 00:00 2024-02-14T14:50:15.132 INFO:teuthology.orchestra.run.smithi050.stdout:(173/173): librados2-19.0.0-1280.ga6fe3fa3.el9. 1.1 MB/s | 3.4 MB 00:03 2024-02-14T14:50:15.137 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-14T14:50:15.137 INFO:teuthology.orchestra.run.smithi050.stdout:Total 9.5 MB/s | 222 MB 00:23 2024-02-14T14:50:15.187 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-14T14:50:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-02-14T14:50:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-02-14T14:50:15.491 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-14T14:50:15.491 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-14T14:50:17.263 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-14T14:50:17.263 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-14T14:50:21.883 INFO:teuthology.orchestra.run.smithi017.stdout:(173/173): ceph-test-19.0.0-1280.ga6fe3fa3.el9. 1.9 MB/s | 47 MB 00:24 2024-02-14T14:50:21.900 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-02-14T14:50:21.900 INFO:teuthology.orchestra.run.smithi017.stdout:Total 7.3 MB/s | 222 MB 00:30 2024-02-14T14:50:22.010 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-02-14T14:50:22.332 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-02-14T14:50:22.332 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-02-14T14:50:22.407 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-02-14T14:50:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/199 2024-02-14T14:50:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/199 2024-02-14T14:50:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/199 2024-02-14T14:50:22.598 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/199 2024-02-14T14:50:22.643 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/199 2024-02-14T14:50:22.682 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/199 2024-02-14T14:50:22.738 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/199 2024-02-14T14:50:22.778 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/199 2024-02-14T14:50:22.805 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/199 2024-02-14T14:50:22.856 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/199 2024-02-14T14:50:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/199 2024-02-14T14:50:23.304 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:50:23.436 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:50:23.472 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:50:23.564 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:50:23.618 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 13/199 2024-02-14T14:50:23.774 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/199 2024-02-14T14:50:23.815 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/199 2024-02-14T14:50:23.843 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:50:23.946 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:50:23.969 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 17/199 2024-02-14T14:50:24.033 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 17/199 2024-02-14T14:50:24.073 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/199 2024-02-14T14:50:24.114 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/199 2024-02-14T14:50:24.204 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-14T14:50:24.212 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/199 2024-02-14T14:50:24.315 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/199 2024-02-14T14:50:24.324 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/199 2024-02-14T14:50:24.348 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/199 2024-02-14T14:50:24.350 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-02-14T14:50:24.351 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-02-14T14:50:24.368 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/199 2024-02-14T14:50:24.385 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/199 2024-02-14T14:50:24.417 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/199 2024-02-14T14:50:24.433 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:24.474 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/199 2024-02-14T14:50:24.494 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:24.525 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/199 2024-02-14T14:50:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/199 2024-02-14T14:50:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/199 2024-02-14T14:50:24.690 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:24.732 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/199 2024-02-14T14:50:24.802 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 23/199 2024-02-14T14:50:24.883 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/199 2024-02-14T14:50:24.894 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 24/199 2024-02-14T14:50:24.924 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 25/199 2024-02-14T14:50:24.993 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 26/199 2024-02-14T14:50:25.047 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/199 2024-02-14T14:50:25.077 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/199 2024-02-14T14:50:25.112 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/199 2024-02-14T14:50:25.153 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/199 2024-02-14T14:50:25.217 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 31/199 2024-02-14T14:50:25.218 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/199 2024-02-14T14:50:25.252 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 32/199 2024-02-14T14:50:25.257 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:50:25.295 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 33/199 2024-02-14T14:50:25.328 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 34/199 2024-02-14T14:50:25.356 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 35/199 2024-02-14T14:50:25.390 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:50:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:50:25.425 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 36/199 2024-02-14T14:50:25.461 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9. 37/199 2024-02-14T14:50:25.495 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el 38/199 2024-02-14T14:50:25.518 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:50:25.524 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 39/199 2024-02-14T14:50:25.554 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 13/199 2024-02-14T14:50:25.594 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 40/199 2024-02-14T14:50:25.639 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 41/199 2024-02-14T14:50:25.713 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/199 2024-02-14T14:50:25.713 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 42/199 2024-02-14T14:50:25.769 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/199 2024-02-14T14:50:25.796 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:50:25.847 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 43/199 2024-02-14T14:50:25.884 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:50:25.899 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 44/199 2024-02-14T14:50:25.913 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 17/199 2024-02-14T14:50:25.961 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 17/199 2024-02-14T14:50:26.010 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/199 2024-02-14T14:50:26.050 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/199 2024-02-14T14:50:26.157 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/199 2024-02-14T14:50:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/199 2024-02-14T14:50:26.296 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/199 2024-02-14T14:50:26.369 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:26.422 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 45/199 2024-02-14T14:50:26.511 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 46/199 2024-02-14T14:50:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 47/199 2024-02-14T14:50:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 48/199 2024-02-14T14:50:26.629 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 48/199 2024-02-14T14:50:26.674 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 49/199 2024-02-14T14:50:26.705 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/199 2024-02-14T14:50:26.739 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 23/199 2024-02-14T14:50:26.819 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 51/199 2024-02-14T14:50:26.865 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 52/199 2024-02-14T14:50:26.871 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 24/199 2024-02-14T14:50:27.099 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 53/199 2024-02-14T14:50:27.114 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 25/199 2024-02-14T14:50:27.193 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 26/199 2024-02-14T14:50:27.243 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/199 2024-02-14T14:50:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 54/199 2024-02-14T14:50:27.272 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/199 2024-02-14T14:50:27.308 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/199 2024-02-14T14:50:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 55/199 2024-02-14T14:50:27.348 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/199 2024-02-14T14:50:27.361 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 56/199 2024-02-14T14:50:27.424 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 31/199 2024-02-14T14:50:27.463 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 32/199 2024-02-14T14:50:27.510 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 57/199 2024-02-14T14:50:27.516 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 33/199 2024-02-14T14:50:27.548 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 34/199 2024-02-14T14:50:27.564 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 58/199 2024-02-14T14:50:27.576 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 35/199 2024-02-14T14:50:27.619 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/199 2024-02-14T14:50:27.648 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 36/199 2024-02-14T14:50:27.680 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9. 37/199 2024-02-14T14:50:27.692 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 60/199 2024-02-14T14:50:27.724 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el 38/199 2024-02-14T14:50:27.731 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 61/199 2024-02-14T14:50:27.753 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 39/199 2024-02-14T14:50:27.768 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 62/199 2024-02-14T14:50:27.815 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/199 2024-02-14T14:50:27.826 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 40/199 2024-02-14T14:50:27.851 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/199 2024-02-14T14:50:27.876 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-1.el9.noarch 41/199 2024-02-14T14:50:27.893 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 65/199 2024-02-14T14:50:27.936 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 66/199 2024-02-14T14:50:27.954 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 42/199 2024-02-14T14:50:28.071 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 67/199 2024-02-14T14:50:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 43/199 2024-02-14T14:50:28.133 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 68/199 2024-02-14T14:50:28.163 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 44/199 2024-02-14T14:50:28.163 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 69/199 2024-02-14T14:50:28.205 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 70/199 2024-02-14T14:50:28.299 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/199 2024-02-14T14:50:28.338 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 72/199 2024-02-14T14:50:28.378 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 73/199 2024-02-14T14:50:28.405 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 74/199 2024-02-14T14:50:28.429 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 75/199 2024-02-14T14:50:28.502 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 76/199 2024-02-14T14:50:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 77/199 2024-02-14T14:50:28.722 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 45/199 2024-02-14T14:50:28.731 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 78/199 2024-02-14T14:50:28.781 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 79/199 2024-02-14T14:50:28.801 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 46/199 2024-02-14T14:50:28.829 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 80/199 2024-02-14T14:50:28.885 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 81/199 2024-02-14T14:50:28.893 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 47/199 2024-02-14T14:50:28.935 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 48/199 2024-02-14T14:50:28.954 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 48/199 2024-02-14T14:50:28.995 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 49/199 2024-02-14T14:50:29.025 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/199 2024-02-14T14:50:29.070 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 82/199 2024-02-14T14:50:29.118 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 83/199 2024-02-14T14:50:29.129 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 51/199 2024-02-14T14:50:29.176 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 52/199 2024-02-14T14:50:29.249 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 84/199 2024-02-14T14:50:29.291 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 85/199 2024-02-14T14:50:29.324 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 86/199 2024-02-14T14:50:29.415 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 87/199 2024-02-14T14:50:29.415 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : jq-1.6-16.el9.x86_64 88/199 2024-02-14T14:50:29.417 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 53/199 2024-02-14T14:50:29.422 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/199 2024-02-14T14:50:29.450 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/199 2024-02-14T14:50:29.484 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/199 2024-02-14T14:50:29.524 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/199 2024-02-14T14:50:29.555 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/199 2024-02-14T14:50:29.581 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 54/199 2024-02-14T14:50:29.585 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/199 2024-02-14T14:50:29.645 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 55/199 2024-02-14T14:50:29.663 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/199 2024-02-14T14:50:29.690 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 56/199 2024-02-14T14:50:29.704 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/199 2024-02-14T14:50:29.737 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/199 2024-02-14T14:50:29.778 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/199 2024-02-14T14:50:29.813 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/199 2024-02-14T14:50:29.825 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 57/199 2024-02-14T14:50:29.885 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 58/199 2024-02-14T14:50:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/199 2024-02-14T14:50:29.950 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/199 2024-02-14T14:50:29.984 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/199 2024-02-14T14:50:30.025 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 60/199 2024-02-14T14:50:30.035 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/199 2024-02-14T14:50:30.067 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/199 2024-02-14T14:50:30.077 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 61/199 2024-02-14T14:50:30.113 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 62/199 2024-02-14T14:50:30.114 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/199 2024-02-14T14:50:30.162 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/199 2024-02-14T14:50:30.196 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/199 2024-02-14T14:50:30.212 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/199 2024-02-14T14:50:30.250 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 65/199 2024-02-14T14:50:30.260 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/199 2024-02-14T14:50:30.292 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 66/199 2024-02-14T14:50:30.436 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 67/199 2024-02-14T14:50:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 68/199 2024-02-14T14:50:30.518 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 69/199 2024-02-14T14:50:30.561 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 70/199 2024-02-14T14:50:30.659 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/199 2024-02-14T14:50:30.701 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 72/199 2024-02-14T14:50:30.742 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 73/199 2024-02-14T14:50:30.767 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 74/199 2024-02-14T14:50:30.792 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 75/199 2024-02-14T14:50:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 76/199 2024-02-14T14:50:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/199 2024-02-14T14:50:30.998 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 77/199 2024-02-14T14:50:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/199 2024-02-14T14:50:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/199 2024-02-14T14:50:31.114 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 78/199 2024-02-14T14:50:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/199 2024-02-14T14:50:31.159 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 79/199 2024-02-14T14:50:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/199 2024-02-14T14:50:31.217 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 80/199 2024-02-14T14:50:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/199 2024-02-14T14:50:31.281 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 81/199 2024-02-14T14:50:31.425 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-02-14T14:50:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/199 2024-02-14T14:50:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/199 2024-02-14T14:50:31.476 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 82/199 2024-02-14T14:50:31.530 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 83/199 2024-02-14T14:50:31.532 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/199 2024-02-14T14:50:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/199 2024-02-14T14:50:31.564 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/199 2024-02-14T14:50:31.602 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/199 2024-02-14T14:50:31.627 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/199 2024-02-14T14:50:31.676 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 84/199 2024-02-14T14:50:31.681 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/199 2024-02-14T14:50:31.719 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 85/199 2024-02-14T14:50:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/199 2024-02-14T14:50:31.752 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 86/199 2024-02-14T14:50:31.774 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/199 2024-02-14T14:50:31.783 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 87/199 2024-02-14T14:50:31.819 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : jq-1.6-16.el9.x86_64 88/199 2024-02-14T14:50:31.824 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/199 2024-02-14T14:50:31.848 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/199 2024-02-14T14:50:31.858 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/199 2024-02-14T14:50:31.894 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/199 2024-02-14T14:50:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/199 2024-02-14T14:50:31.937 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/199 2024-02-14T14:50:31.976 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/199 2024-02-14T14:50:32.008 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/199 2024-02-14T14:50:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/199 2024-02-14T14:50:32.117 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/199 2024-02-14T14:50:32.156 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/199 2024-02-14T14:50:32.189 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/199 2024-02-14T14:50:32.196 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/199 2024-02-14T14:50:32.231 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/199 2024-02-14T14:50:32.265 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/199 2024-02-14T14:50:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/199 2024-02-14T14:50:32.304 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/199 2024-02-14T14:50:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:50:32.360 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/199 2024-02-14T14:50:32.456 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:50:32.457 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/199 2024-02-14T14:50:32.483 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:50:32.506 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/199 2024-02-14T14:50:32.536 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/199 2024-02-14T14:50:32.559 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:50:32.584 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/199 2024-02-14T14:50:32.592 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 13/199 2024-02-14T14:50:32.697 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/199 2024-02-14T14:50:32.748 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/199 2024-02-14T14:50:32.762 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/199 2024-02-14T14:50:32.817 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/199 2024-02-14T14:50:32.845 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:50:32.934 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:50:32.963 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 17/199 2024-02-14T14:50:33.011 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 17/199 2024-02-14T14:50:33.051 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/199 2024-02-14T14:50:33.083 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/199 2024-02-14T14:50:33.247 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/199 2024-02-14T14:50:33.385 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/199 2024-02-14T14:50:33.445 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 21/199 2024-02-14T14:50:33.529 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/199 2024-02-14T14:50:33.535 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:33.536 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/199 2024-02-14T14:50:33.576 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/199 2024-02-14T14:50:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:33.602 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/199 2024-02-14T14:50:33.611 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/199 2024-02-14T14:50:33.629 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 120/199 2024-02-14T14:50:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/199 2024-02-14T14:50:33.752 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/199 2024-02-14T14:50:33.787 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/199 2024-02-14T14:50:33.815 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 22/199 2024-02-14T14:50:33.929 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : re2-1:20211101-3.el9.x86_64 23/199 2024-02-14T14:50:33.951 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 121/199 2024-02-14T14:50:33.978 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/199 2024-02-14T14:50:33.992 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 122/199 2024-02-14T14:50:34.008 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/199 2024-02-14T14:50:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 24/199 2024-02-14T14:50:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 25/199 2024-02-14T14:50:34.082 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 122/199 2024-02-14T14:50:34.088 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/199 2024-02-14T14:50:34.105 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 123/199 2024-02-14T14:50:34.138 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : liboath-2.6.7-2.el9.x86_64 26/199 2024-02-14T14:50:34.141 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 124/199 2024-02-14T14:50:34.199 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/199 2024-02-14T14:50:34.237 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/199 2024-02-14T14:50:34.272 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/199 2024-02-14T14:50:34.313 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/199 2024-02-14T14:50:34.388 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 31/199 2024-02-14T14:50:34.428 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 32/199 2024-02-14T14:50:34.472 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 33/199 2024-02-14T14:50:34.504 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 34/199 2024-02-14T14:50:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-5.4.4-4.el9.x86_64 35/199 2024-02-14T14:50:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 125/199 2024-02-14T14:50:34.603 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 36/199 2024-02-14T14:50:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 126/199 2024-02-14T14:50:34.645 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9. 37/199 2024-02-14T14:50:34.688 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el 38/199 2024-02-14T14:50:34.699 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 126/199 2024-02-14T14:50:34.716 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 39/199 2024-02-14T14:50:34.722 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 127/199 2024-02-14T14:50:34.787 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/199 2024-02-14T14:50:34.790 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 40/199 2024-02-14T14:50:34.839 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : luarocks-3.9.2-1.el9.noarch 41/199 2024-02-14T14:50:34.917 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 42/199 2024-02-14T14:50:34.920 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/199 2024-02-14T14:50:35.057 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 43/199 2024-02-14T14:50:35.113 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 44/199 2024-02-14T14:50:35.676 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rsa-4.9-2.el9.noarch 45/199 2024-02-14T14:50:35.758 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 46/199 2024-02-14T14:50:35.844 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 47/199 2024-02-14T14:50:35.884 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 48/199 2024-02-14T14:50:35.903 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 48/199 2024-02-14T14:50:35.935 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 49/199 2024-02-14T14:50:35.976 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/199 2024-02-14T14:50:36.085 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 51/199 2024-02-14T14:50:36.152 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 52/199 2024-02-14T14:50:36.235 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/199 2024-02-14T14:50:36.312 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/199 2024-02-14T14:50:36.339 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 120/199 2024-02-14T14:50:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 53/199 2024-02-14T14:50:36.592 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 54/199 2024-02-14T14:50:36.663 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 121/199 2024-02-14T14:50:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 55/199 2024-02-14T14:50:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 56/199 2024-02-14T14:50:36.715 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 122/199 2024-02-14T14:50:36.802 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 122/199 2024-02-14T14:50:36.832 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 123/199 2024-02-14T14:50:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:36.864 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 57/199 2024-02-14T14:50:36.869 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 124/199 2024-02-14T14:50:36.876 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:36.909 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 58/199 2024-02-14T14:50:36.966 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/199 2024-02-14T14:50:37.040 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 60/199 2024-02-14T14:50:37.076 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 61/199 2024-02-14T14:50:37.113 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 62/199 2024-02-14T14:50:37.160 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/199 2024-02-14T14:50:37.196 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/199 2024-02-14T14:50:37.239 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 65/199 2024-02-14T14:50:37.282 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 66/199 2024-02-14T14:50:37.350 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 125/199 2024-02-14T14:50:37.372 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 126/199 2024-02-14T14:50:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:37.424 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 67/199 2024-02-14T14:50:37.427 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 126/199 2024-02-14T14:50:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 129/199 2024-02-14T14:50:37.457 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 127/199 2024-02-14T14:50:37.470 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 68/199 2024-02-14T14:50:37.500 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 69/199 2024-02-14T14:50:37.520 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 129/199 2024-02-14T14:50:37.520 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-14T14:50:37.520 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-14T14:50:37.520 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:37.542 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 70/199 2024-02-14T14:50:37.565 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 130/199 2024-02-14T14:50:37.641 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/199 2024-02-14T14:50:37.675 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 72/199 2024-02-14T14:50:37.707 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 73/199 2024-02-14T14:50:37.749 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 74/199 2024-02-14T14:50:37.774 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 75/199 2024-02-14T14:50:37.855 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 76/199 2024-02-14T14:50:37.997 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 77/199 2024-02-14T14:50:38.103 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 78/199 2024-02-14T14:50:38.149 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 79/199 2024-02-14T14:50:38.208 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ply-3.11-14.el9.noarch 80/199 2024-02-14T14:50:38.272 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 81/199 2024-02-14T14:50:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 82/199 2024-02-14T14:50:38.520 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 83/199 2024-02-14T14:50:38.666 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 84/199 2024-02-14T14:50:38.710 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 85/199 2024-02-14T14:50:38.742 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 86/199 2024-02-14T14:50:38.774 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 87/199 2024-02-14T14:50:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : jq-1.6-16.el9.x86_64 88/199 2024-02-14T14:50:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/199 2024-02-14T14:50:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/199 2024-02-14T14:50:38.903 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/199 2024-02-14T14:50:38.942 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/199 2024-02-14T14:50:38.982 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/199 2024-02-14T14:50:39.012 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/199 2024-02-14T14:50:39.091 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/199 2024-02-14T14:50:39.130 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/199 2024-02-14T14:50:39.164 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/199 2024-02-14T14:50:39.205 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/199 2024-02-14T14:50:39.400 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/199 2024-02-14T14:50:39.575 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/199 2024-02-14T14:50:39.689 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/199 2024-02-14T14:50:39.746 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:39.754 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/199 2024-02-14T14:50:39.787 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/199 2024-02-14T14:50:39.858 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/199 2024-02-14T14:50:39.969 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/199 2024-02-14T14:50:40.021 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/199 2024-02-14T14:50:40.335 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:40.366 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 129/199 2024-02-14T14:50:40.431 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 129/199 2024-02-14T14:50:40.432 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-14T14:50:40.432 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-14T14:50:40.432 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:40.458 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 130/199 2024-02-14T14:50:40.788 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/199 2024-02-14T14:50:40.834 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/199 2024-02-14T14:50:40.868 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/199 2024-02-14T14:50:40.917 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/199 2024-02-14T14:50:41.007 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/199 2024-02-14T14:50:41.043 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/199 2024-02-14T14:50:41.226 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/199 2024-02-14T14:50:41.258 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/199 2024-02-14T14:50:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/199 2024-02-14T14:50:42.013 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/199 2024-02-14T14:50:42.136 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/199 2024-02-14T14:50:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/199 2024-02-14T14:50:43.514 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/199 2024-02-14T14:50:43.541 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 120/199 2024-02-14T14:50:43.865 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 121/199 2024-02-14T14:50:43.904 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 122/199 2024-02-14T14:50:43.995 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 122/199 2024-02-14T14:50:44.026 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 123/199 2024-02-14T14:50:44.063 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 124/199 2024-02-14T14:50:44.541 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 125/199 2024-02-14T14:50:44.565 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 126/199 2024-02-14T14:50:44.637 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 126/199 2024-02-14T14:50:44.659 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 127/199 2024-02-14T14:50:46.984 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:47.023 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 128/199 2024-02-14T14:50:47.610 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 129/199 2024-02-14T14:50:47.702 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 129/199 2024-02-14T14:50:47.702 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-14T14:50:47.702 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-14T14:50:47.702 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:50:47.737 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 130/199 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 130/199 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-02-14T14:50:49.215 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:49.389 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 131/199 2024-02-14T14:50:49.405 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 131/199 2024-02-14T14:50:49.405 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:49.405 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-14T14:50:49.405 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:50.843 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 132/199 2024-02-14T14:50:50.868 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 132/199 2024-02-14T14:50:50.868 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:50.868 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-14T14:50:50.868 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T14:50:50.868 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T14:50:50.868 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:51.076 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 133/199 2024-02-14T14:50:51.098 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 133/199 2024-02-14T14:50:51.099 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:51.099 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-14T14:50:51.099 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T14:50:51.099 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T14:50:51.099 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:51.535 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 134/199 2024-02-14T14:50:51.558 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 134/199 2024-02-14T14:50:51.558 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:51.558 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-14T14:50:51.558 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T14:50:51.558 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T14:50:51.558 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:51.662 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 135/199 2024-02-14T14:50:52.052 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 136/199 2024-02-14T14:50:52.094 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 136/199 2024-02-14T14:50:52.123 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.e 137/199 2024-02-14T14:50:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3. 138/199 2024-02-14T14:50:52.208 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 139/199 2024-02-14T14:50:52.439 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 139/199 2024-02-14T14:50:52.476 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 140/199 2024-02-14T14:50:52.622 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 130/199 2024-02-14T14:50:52.622 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-02-14T14:50:52.622 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-02-14T14:50:52.622 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-02-14T14:50:52.622 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-02-14T14:50:52.622 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-02-14T14:50:52.623 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-02-14T14:50:52.623 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-02-14T14:50:52.623 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:52.790 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 131/199 2024-02-14T14:50:52.807 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 131/199 2024-02-14T14:50:52.807 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:52.807 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-14T14:50:52.807 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:53.441 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 140/199 2024-02-14T14:50:53.468 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 141/199 2024-02-14T14:50:53.569 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 141/199 2024-02-14T14:50:53.739 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el 142/199 2024-02-14T14:50:53.770 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 143/199 2024-02-14T14:50:53.793 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 143/199 2024-02-14T14:50:53.793 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:53.793 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-14T14:50:53.793 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T14:50:53.793 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T14:50:53.794 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 144/199 2024-02-14T14:50:53.849 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 144/199 2024-02-14T14:50:53.893 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 145/199 2024-02-14T14:50:54.335 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 132/199 2024-02-14T14:50:54.363 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 132/199 2024-02-14T14:50:54.363 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:54.363 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-14T14:50:54.363 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T14:50:54.363 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T14:50:54.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:54.577 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 133/199 2024-02-14T14:50:54.601 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 133/199 2024-02-14T14:50:54.602 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:54.602 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-14T14:50:54.602 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T14:50:54.602 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T14:50:54.602 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 146/199 2024-02-14T14:50:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 147/199 2024-02-14T14:50:54.795 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 147/199 2024-02-14T14:50:54.795 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:54.796 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-14T14:50:54.796 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T14:50:54.796 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T14:50:54.796 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:54.840 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 148/199 2024-02-14T14:50:54.862 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 148/199 2024-02-14T14:50:54.862 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:54.862 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-14T14:50:54.862 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:54.907 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 149/199 2024-02-14T14:50:54.930 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 149/199 2024-02-14T14:50:54.930 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:54.930 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-14T14:50:54.930 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-14T14:50:54.930 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-14T14:50:54.930 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:55.070 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 134/199 2024-02-14T14:50:55.093 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 134/199 2024-02-14T14:50:55.093 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:55.094 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-14T14:50:55.094 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T14:50:55.094 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T14:50:55.094 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:55.188 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 135/199 2024-02-14T14:50:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:50:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:50:55.256 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:55.256 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-14T14:50:55.256 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T14:50:55.256 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T14:50:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:50:55.596 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 136/199 2024-02-14T14:50:55.628 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 136/199 2024-02-14T14:50:55.659 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.e 137/199 2024-02-14T14:50:55.723 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3. 138/199 2024-02-14T14:50:55.751 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 139/199 2024-02-14T14:50:55.993 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 139/199 2024-02-14T14:50:56.019 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 140/199 2024-02-14T14:50:57.035 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 140/199 2024-02-14T14:50:57.060 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 141/199 2024-02-14T14:50:57.167 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 141/199 2024-02-14T14:50:57.339 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el 142/199 2024-02-14T14:50:57.371 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 143/199 2024-02-14T14:50:57.395 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 143/199 2024-02-14T14:50:57.395 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:57.396 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-14T14:50:57.396 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T14:50:57.396 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T14:50:57.396 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:57.447 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 144/199 2024-02-14T14:50:57.458 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 144/199 2024-02-14T14:50:57.513 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 145/199 2024-02-14T14:50:58.413 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-45.el9.x86_64 146/199 2024-02-14T14:50:58.440 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 147/199 2024-02-14T14:50:58.465 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 147/199 2024-02-14T14:50:58.466 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:58.466 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-14T14:50:58.466 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T14:50:58.466 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T14:50:58.466 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:58.508 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 148/199 2024-02-14T14:50:58.531 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 148/199 2024-02-14T14:50:58.532 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:58.532 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-14T14:50:58.532 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:58.575 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 149/199 2024-02-14T14:50:58.599 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 149/199 2024-02-14T14:50:58.599 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:58.599 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-14T14:50:58.599 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-14T14:50:58.600 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-14T14:50:58.600 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:58.917 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:50:58.942 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:50:58.942 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:50:58.942 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-14T14:50:58.942 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T14:50:58.942 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T14:50:58.942 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:50:59.671 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 151/199 2024-02-14T14:50:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 152/199 2024-02-14T14:50:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 153/199 2024-02-14T14:50:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 154/199 2024-02-14T14:50:59.944 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 155/199 2024-02-14T14:50:59.963 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 130/199 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /sys 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /proc 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /mnt 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /var/tmp 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /home 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /root 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /tmp 2024-02-14T14:50:59.964 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:00.052 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 156/199 2024-02-14T14:51:00.101 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 157/199 2024-02-14T14:51:00.133 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 131/199 2024-02-14T14:51:00.140 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 158/199 2024-02-14T14:51:00.148 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 131/199 2024-02-14T14:51:00.148 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:00.149 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-14T14:51:00.149 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:00.193 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 159/199 2024-02-14T14:51:00.257 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 160/199 2024-02-14T14:51:00.303 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 161/199 2024-02-14T14:51:00.347 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/199 2024-02-14T14:51:00.369 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_ 163/199 2024-02-14T14:51:00.399 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 164/199 2024-02-14T14:51:00.471 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 165/199 2024-02-14T14:51:00.521 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 166/199 2024-02-14T14:51:00.564 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/199 2024-02-14T14:51:01.198 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:01.237 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:01.257 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:01.679 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 132/199 2024-02-14T14:51:01.704 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 132/199 2024-02-14T14:51:01.704 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:01.704 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-14T14:51:01.704 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T14:51:01.705 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T14:51:01.705 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:01.921 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 133/199 2024-02-14T14:51:01.945 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 133/199 2024-02-14T14:51:01.945 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:01.945 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-14T14:51:01.945 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T14:51:01.945 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T14:51:01.945 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:02.095 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 169/199 2024-02-14T14:51:02.166 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 170/199 2024-02-14T14:51:02.207 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 171/199 2024-02-14T14:51:02.305 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/199 2024-02-14T14:51:02.337 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/199 2024-02-14T14:51:02.370 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 174/199 2024-02-14T14:51:02.395 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 175/199 2024-02-14T14:51:02.405 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 134/199 2024-02-14T14:51:02.422 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 176/199 2024-02-14T14:51:02.422 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 177/199 2024-02-14T14:51:02.427 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 134/199 2024-02-14T14:51:02.428 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:02.428 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-14T14:51:02.428 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T14:51:02.428 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T14:51:02.428 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:02.506 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 177/199 2024-02-14T14:51:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : automake-1.16.2-8.el9.noarch 135/199 2024-02-14T14:51:02.526 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 178/199 2024-02-14T14:51:02.556 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/199 2024-02-14T14:51:02.556 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/199 2024-02-14T14:51:02.641 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/199 2024-02-14T14:51:02.667 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/199 2024-02-14T14:51:02.701 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/199 2024-02-14T14:51:02.726 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 183/199 2024-02-14T14:51:02.753 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 184/199 2024-02-14T14:51:02.785 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 185/199 2024-02-14T14:51:02.836 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 186/199 2024-02-14T14:51:02.875 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 187/199 2024-02-14T14:51:02.907 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 188/199 2024-02-14T14:51:02.923 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 136/199 2024-02-14T14:51:02.932 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 189/199 2024-02-14T14:51:02.955 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 136/199 2024-02-14T14:51:02.958 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 190/199 2024-02-14T14:51:02.983 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 191/199 2024-02-14T14:51:02.985 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.e 137/199 2024-02-14T14:51:03.008 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 192/199 2024-02-14T14:51:03.015 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:03.015 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3. 138/199 2024-02-14T14:51:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 139/199 2024-02-14T14:51:03.203 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:03.203 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 194/199 2024-02-14T14:51:03.286 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 194/199 2024-02-14T14:51:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 195/199 2024-02-14T14:51:03.321 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 139/199 2024-02-14T14:51:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 196/199 2024-02-14T14:51:03.354 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 140/199 2024-02-14T14:51:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 197/199 2024-02-14T14:51:03.393 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 198/199 2024-02-14T14:51:03.459 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 199/199 2024-02-14T14:51:03.623 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 151/199 2024-02-14T14:51:03.664 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 152/199 2024-02-14T14:51:03.694 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 153/199 2024-02-14T14:51:03.721 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 154/199 2024-02-14T14:51:03.915 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 155/199 2024-02-14T14:51:04.033 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 156/199 2024-02-14T14:51:04.079 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 157/199 2024-02-14T14:51:04.117 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 158/199 2024-02-14T14:51:04.186 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 159/199 2024-02-14T14:51:04.269 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 160/199 2024-02-14T14:51:04.313 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 161/199 2024-02-14T14:51:04.348 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/199 2024-02-14T14:51:04.366 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 140/199 2024-02-14T14:51:04.378 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_ 163/199 2024-02-14T14:51:04.396 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 141/199 2024-02-14T14:51:04.408 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 164/199 2024-02-14T14:51:04.471 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 165/199 2024-02-14T14:51:04.504 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 141/199 2024-02-14T14:51:04.523 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 166/199 2024-02-14T14:51:04.548 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/199 2024-02-14T14:51:04.666 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el 142/199 2024-02-14T14:51:04.698 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 143/199 2024-02-14T14:51:04.722 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 143/199 2024-02-14T14:51:04.722 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:04.722 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-14T14:51:04.722 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T14:51:04.722 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T14:51:04.722 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:04.782 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 144/199 2024-02-14T14:51:04.793 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 144/199 2024-02-14T14:51:04.846 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 145/199 2024-02-14T14:51:05.145 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:05.188 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:05.209 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libtool-2.4.6-45.el9.x86_64 146/199 2024-02-14T14:51:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 147/199 2024-02-14T14:51:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 147/199 2024-02-14T14:51:05.807 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:05.807 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-14T14:51:05.807 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T14:51:05.807 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T14:51:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 148/199 2024-02-14T14:51:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 148/199 2024-02-14T14:51:05.873 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:05.873 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-14T14:51:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 149/199 2024-02-14T14:51:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 149/199 2024-02-14T14:51:05.941 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:05.941 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-14T14:51:05.941 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-14T14:51:05.942 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-14T14:51:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:06.116 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 169/199 2024-02-14T14:51:06.175 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 170/199 2024-02-14T14:51:06.209 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 171/199 2024-02-14T14:51:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:51:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:51:06.292 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T14:51:06.292 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-14T14:51:06.292 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T14:51:06.292 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T14:51:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:06.300 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/199 2024-02-14T14:51:06.337 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/199 2024-02-14T14:51:06.362 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 174/199 2024-02-14T14:51:06.387 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 175/199 2024-02-14T14:51:06.414 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 176/199 2024-02-14T14:51:06.414 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 177/199 2024-02-14T14:51:06.498 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 177/199 2024-02-14T14:51:06.526 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 178/199 2024-02-14T14:51:06.557 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/199 2024-02-14T14:51:06.557 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/199 2024-02-14T14:51:06.667 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/199 2024-02-14T14:51:06.692 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/199 2024-02-14T14:51:06.726 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/199 2024-02-14T14:51:06.751 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 183/199 2024-02-14T14:51:06.779 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 184/199 2024-02-14T14:51:06.811 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 185/199 2024-02-14T14:51:06.845 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 186/199 2024-02-14T14:51:06.892 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 187/199 2024-02-14T14:51:06.924 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 188/199 2024-02-14T14:51:06.949 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 189/199 2024-02-14T14:51:06.975 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 190/199 2024-02-14T14:51:07.000 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 191/199 2024-02-14T14:51:07.025 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 192/199 2024-02-14T14:51:07.033 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:07.033 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:07.228 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:07.228 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 194/199 2024-02-14T14:51:07.320 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 194/199 2024-02-14T14:51:07.344 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 195/199 2024-02-14T14:51:07.369 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 196/199 2024-02-14T14:51:07.395 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 197/199 2024-02-14T14:51:07.429 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 198/199 2024-02-14T14:51:07.504 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 199/199 2024-02-14T14:51:08.433 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 199/199 2024-02-14T14:51:08.433 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 1/199 2024-02-14T14:51:08.433 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2/199 2024-02-14T14:51:08.433 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 3/199 2024-02-14T14:51:08.433 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 4/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 5/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 6/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 7/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 8/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 9/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 10/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 13/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 14/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_ 15/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 17/199 2024-02-14T14:51:08.434 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 18/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 19/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 20/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el 21/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9. 22/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 23/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 24/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 25/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 26/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 27/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 28/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 29/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3. 30/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 31/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 32/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 33/199 2024-02-14T14:51:08.435 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el 34/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 35/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.e 36/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 37/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 38/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 39/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/199 2024-02-14T14:51:08.436 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/199 2024-02-14T14:51:08.437 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/199 2024-02-14T14:51:08.438 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 87/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 88/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 89/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 90/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : jq-1.6-16.el9.x86_64 91/199 2024-02-14T14:51:08.439 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 92/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 93/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 94/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 95/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 96/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 97/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 98/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 99/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 100/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 101/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 102/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 103/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 104/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 105/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 106/199 2024-02-14T14:51:08.440 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 107/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 108/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 109/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 110/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 111/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 112/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 113/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 114/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 115/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 116/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 117/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 118/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 119/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 120/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 121/199 2024-02-14T14:51:08.441 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 122/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 123/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 124/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 125/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 126/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 127/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 128/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 129/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 130/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 131/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 132/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 133/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 134/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 135/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 136/199 2024-02-14T14:51:08.442 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 137/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 138/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 139/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 140/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 141/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 142/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 143/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 144/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 145/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 146/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 147/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 148/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 149/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 151/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 152/199 2024-02-14T14:51:08.443 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 153/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 154/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 155/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 156/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 157/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 158/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 160/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 161/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-4.el9.x86_64 162/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-3.el9.x86_64 163/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 165/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 166/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 167/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 168/199 2024-02-14T14:51:08.444 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 169/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 170/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 171/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 172/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 173/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 174/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 175/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 176/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 177/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 178/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 179/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 180/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 181/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 182/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 183/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 184/199 2024-02-14T14:51:08.445 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 185/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 186/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 187/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 188/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 189/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 190/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 191/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 192/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 193/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 194/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 195/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 196/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 197/199 2024-02-14T14:51:08.446 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 198/199 2024-02-14T14:51:10.991 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 151/199 2024-02-14T14:51:11.032 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 152/199 2024-02-14T14:51:11.062 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 153/199 2024-02-14T14:51:11.090 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 154/199 2024-02-14T14:51:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 155/199 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 199/199 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: acl-2.3.1-4.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-14T14:51:11.285 INFO:teuthology.orchestra.run.smithi076.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.286 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.287 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.288 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: jq-1.6-16.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.289 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-14T14:51:11.290 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.11-1.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-14T14:51:11.291 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-14T14:51:11.292 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-14T14:51:11.293 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-14T14:51:11.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-14T14:51:11.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-14T14:51:11.296 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-14T14:51:11.297 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-14T14:51:11.297 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:51:11.297 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-02-14T14:51:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 156/199 2024-02-14T14:51:11.438 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 157/199 2024-02-14T14:51:11.495 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 158/199 2024-02-14T14:51:11.546 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 159/199 2024-02-14T14:51:11.619 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 160/199 2024-02-14T14:51:11.665 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 161/199 2024-02-14T14:51:11.700 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/199 2024-02-14T14:51:11.739 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_ 163/199 2024-02-14T14:51:11.769 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 164/199 2024-02-14T14:51:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 165/199 2024-02-14T14:51:11.889 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 166/199 2024-02-14T14:51:11.928 DEBUG:teuthology.parallel:result is None 2024-02-14T14:51:11.934 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/199 2024-02-14T14:51:12.476 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 199/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 1/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 3/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 4/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 5/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 6/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 7/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 8/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 9/199 2024-02-14T14:51:12.477 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 10/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 13/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 14/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_ 15/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 17/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 18/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 19/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 20/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el 21/199 2024-02-14T14:51:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9. 22/199 2024-02-14T14:51:12.479 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 23/199 2024-02-14T14:51:12.479 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 24/199 2024-02-14T14:51:12.479 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 25/199 2024-02-14T14:51:12.479 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 26/199 2024-02-14T14:51:12.479 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 27/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 28/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 29/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3. 30/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 31/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 32/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 33/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el 34/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 35/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.e 36/199 2024-02-14T14:51:12.498 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 37/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 38/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 39/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/199 2024-02-14T14:51:12.499 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/199 2024-02-14T14:51:12.500 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/199 2024-02-14T14:51:12.501 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/199 2024-02-14T14:51:12.502 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 87/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 88/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 89/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 90/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : jq-1.6-16.el9.x86_64 91/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 92/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 93/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 94/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 95/199 2024-02-14T14:51:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 96/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 97/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 98/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 99/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 100/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 101/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 102/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 103/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 104/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 105/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 106/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 107/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 108/199 2024-02-14T14:51:12.504 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 109/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 110/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 111/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 112/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 113/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 114/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 115/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 116/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 117/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 118/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 119/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 120/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 121/199 2024-02-14T14:51:12.505 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 122/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 123/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 124/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 125/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 126/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 127/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 128/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 129/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 130/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 131/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 132/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 133/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 134/199 2024-02-14T14:51:12.506 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 135/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 136/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 137/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 138/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 139/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 140/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 141/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 142/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 143/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 144/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 145/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 146/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 147/199 2024-02-14T14:51:12.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 148/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 149/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 151/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 152/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 153/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 154/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 155/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 156/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 157/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 158/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/199 2024-02-14T14:51:12.508 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 160/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 161/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-4.el9.x86_64 162/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-3.el9.x86_64 163/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 165/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 166/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 167/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 168/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 169/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 170/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 171/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 172/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 173/199 2024-02-14T14:51:12.509 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 174/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 175/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 176/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 177/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 178/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 179/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 180/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 181/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 182/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 183/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 184/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 185/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 186/199 2024-02-14T14:51:12.510 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 187/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 188/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 189/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 190/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 191/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 192/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 193/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 194/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 195/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 196/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 197/199 2024-02-14T14:51:12.511 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 198/199 2024-02-14T14:51:12.529 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:12.565 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:12.586 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 168/199 2024-02-14T14:51:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 169/199 2024-02-14T14:51:13.529 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 170/199 2024-02-14T14:51:13.571 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 171/199 2024-02-14T14:51:13.665 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/199 2024-02-14T14:51:13.692 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/199 2024-02-14T14:51:13.717 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 174/199 2024-02-14T14:51:13.742 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 175/199 2024-02-14T14:51:13.769 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 176/199 2024-02-14T14:51:13.769 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 177/199 2024-02-14T14:51:13.845 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 177/199 2024-02-14T14:51:13.873 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 178/199 2024-02-14T14:51:13.903 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/199 2024-02-14T14:51:13.903 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/199 2024-02-14T14:51:13.988 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/199 2024-02-14T14:51:14.022 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/199 2024-02-14T14:51:14.056 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/199 2024-02-14T14:51:14.081 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 183/199 2024-02-14T14:51:14.116 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 184/199 2024-02-14T14:51:14.149 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 185/199 2024-02-14T14:51:14.184 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 186/199 2024-02-14T14:51:14.222 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 187/199 2024-02-14T14:51:14.262 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 188/199 2024-02-14T14:51:14.287 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 189/199 2024-02-14T14:51:14.312 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 190/199 2024-02-14T14:51:14.338 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 191/199 2024-02-14T14:51:14.363 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 192/199 2024-02-14T14:51:14.381 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:14.382 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:14.566 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 193/199 2024-02-14T14:51:14.566 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 194/199 2024-02-14T14:51:14.649 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 194/199 2024-02-14T14:51:14.674 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 195/199 2024-02-14T14:51:14.699 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 196/199 2024-02-14T14:51:14.716 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 197/199 2024-02-14T14:51:14.749 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 198/199 2024-02-14T14:51:14.827 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 199/199 2024-02-14T14:51:15.210 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 199/199 2024-02-14T14:51:15.210 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:51:15.210 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-02-14T14:51:15.210 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:15.210 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: acl-2.3.1-4.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-14T14:51:15.211 INFO:teuthology.orchestra.run.smithi050.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.212 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.214 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-14T14:51:15.215 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: jq-1.6-16.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.216 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-14T14:51:15.217 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.11-1.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-14T14:51:15.218 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-14T14:51:15.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-14T14:51:15.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-14T14:51:15.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-14T14:51:15.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-14T14:51:15.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:15.224 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-14T14:51:15.225 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-14T14:51:15.801 DEBUG:teuthology.parallel:result is None 2024-02-14T14:51:19.503 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 199/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 1/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 3/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 4/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3 5/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 6/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 7/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 8/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 9/199 2024-02-14T14:51:19.504 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 10/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 11/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 12/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 13/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 14/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_ 15/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 16/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 17/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 18/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86 19/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 20/199 2024-02-14T14:51:19.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el 21/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9. 22/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_6 23/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 24/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 25/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 26/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 27/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 28/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 29/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3. 30/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noa 31/199 2024-02-14T14:51:19.506 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.n 32/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6f 33/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el 34/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 35/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.e 36/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 37/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 38/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 39/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/199 2024-02-14T14:51:19.507 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/199 2024-02-14T14:51:19.508 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/199 2024-02-14T14:51:19.509 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/199 2024-02-14T14:51:19.510 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 87/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 88/199 2024-02-14T14:51:19.511 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 89/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 90/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : jq-1.6-16.el9.x86_64 91/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 92/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 93/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 94/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 95/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 96/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ply-3.11-14.el9.noarch 97/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 98/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 99/199 2024-02-14T14:51:19.512 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 100/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 101/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 102/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 103/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 104/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 105/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 106/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 107/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 108/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 109/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 110/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 111/199 2024-02-14T14:51:19.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 112/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 113/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 114/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 115/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 116/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 117/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 118/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 119/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 120/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 121/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 122/199 2024-02-14T14:51:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 123/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 124/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 125/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 126/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 127/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 128/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 129/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 130/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 131/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 132/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 133/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 134/199 2024-02-14T14:51:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 135/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 136/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 137/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 138/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 139/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 140/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 141/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 142/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 143/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 144/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 145/199 2024-02-14T14:51:19.516 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : re2-1:20211101-3.el9.x86_64 146/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 147/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 148/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 149/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 150/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 151/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 152/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 153/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 154/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 155/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 156/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 157/199 2024-02-14T14:51:19.517 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 158/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 160/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 161/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : acl-2.3.1-4.el9.x86_64 162/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : acl-2.3.1-3.el9.x86_64 163/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 165/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 166/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 167/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 168/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 169/199 2024-02-14T14:51:19.518 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 170/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 171/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 172/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 173/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 174/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 175/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 176/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 177/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 178/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 179/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 180/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 181/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 182/199 2024-02-14T14:51:19.519 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 183/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 184/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 185/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 186/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 187/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 188/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 189/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 190/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 191/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 192/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 193/199 2024-02-14T14:51:19.520 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 194/199 2024-02-14T14:51:19.521 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 195/199 2024-02-14T14:51:19.521 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 196/199 2024-02-14T14:51:19.521 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 197/199 2024-02-14T14:51:19.521 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 198/199 2024-02-14T14:51:22.289 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 199/199 2024-02-14T14:51:22.289 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:22.289 INFO:teuthology.orchestra.run.smithi017.stdout:Upgraded: 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: acl-2.3.1-4.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-14T14:51:22.290 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: librados2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.291 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-14T14:51:22.292 INFO:teuthology.orchestra.run.smithi017.stdout: automake-1.16.2-8.el9.noarch 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: bison-3.7.4-5.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.293 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.294 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-shell-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top-2:19.0.0-1280.ga6fe3fa3.el9.noarch 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: flex-2.6.4-9.el9.x86_64 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-14T14:51:22.295 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: jq-1.6-16.el9.x86_64 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-14T14:51:22.296 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.297 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: lua-5.4.4-4.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-14T14:51:22.298 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-13.11-1.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-14T14:51:22.299 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-14T14:51:22.300 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-14T14:51:22.301 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-14T14:51:22.302 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply-3.11-14.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-14T14:51:22.303 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.304 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-14T14:51:22.305 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd-2:19.0.0-1280.ga6fe3fa3.el9.x86_64 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: re2-1:20211101-3.el9.x86_64 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-14T14:51:22.306 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-14T14:51:22.307 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-14T14:51:22.307 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-14T14:51:22.307 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:51:22.307 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-02-14T14:51:22.917 DEBUG:teuthology.parallel:result is None 2024-02-14T14:51:22.918 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:51:23.231 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-14T14:51:23.255 INFO:teuthology.orchestra.run.smithi017.stdout:19.0.0-1280.ga6fe3fa3.el9 2024-02-14T14:51:23.255 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1280.ga6fe3fa3.el9 2024-02-14T14:51:23.256 INFO:teuthology.task.install:The correct ceph version 19.0.0-1280.ga6fe3fa3 is installed. 2024-02-14T14:51:23.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:51:23.538 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-14T14:51:23.565 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-1280.ga6fe3fa3.el9 2024-02-14T14:51:23.565 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1280.ga6fe3fa3.el9 2024-02-14T14:51:23.565 INFO:teuthology.task.install:The correct ceph version 19.0.0-1280.ga6fe3fa3 is installed. 2024-02-14T14:51:23.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:51:23.839 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-14T14:51:23.864 INFO:teuthology.orchestra.run.smithi076.stdout:19.0.0-1280.ga6fe3fa3.el9 2024-02-14T14:51:23.864 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1280.ga6fe3fa3.el9 2024-02-14T14:51:23.864 INFO:teuthology.task.install:The correct ceph version 19.0.0-1280.ga6fe3fa3 is installed. 2024-02-14T14:51:23.866 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-14T14:51:23.866 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:51:23.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-14T14:51:23.900 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:51:23.900 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-14T14:51:23.939 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:51:23.939 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-14T14:51:23.972 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-14T14:51:23.972 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:51:23.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2024-02-14T14:51:24.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-14T14:51:24.072 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:51:24.072 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-02-14T14:51:24.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-14T14:51:24.175 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:51:24.176 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-02-14T14:51:24.207 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-14T14:51:24.276 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-14T14:51:24.277 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:51:24.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-14T14:51:24.306 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-14T14:51:24.375 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:51:24.375 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-14T14:51:24.406 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-14T14:51:24.477 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:51:24.477 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-14T14:51:24.505 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-14T14:51:24.574 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-14T14:51:24.574 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:51:24.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2024-02-14T14:51:24.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-14T14:51:24.675 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:51:24.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-02-14T14:51:24.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-14T14:51:24.776 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:51:24.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-02-14T14:51:24.804 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-14T14:51:24.871 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-14T14:51:24.967 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-02-14T14:51:24.967 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:51:24.968 INFO:tasks.cephadm:Cluster fsid is 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:51:24.968 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-14T14:51:24.968 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.17', 'mon.b': '172.21.15.50', 'mon.c': '172.21.15.76'} 2024-02-14T14:51:24.968 INFO:tasks.cephadm:First mon is mon.a on smithi017 2024-02-14T14:51:24.968 INFO:tasks.cephadm:First mgr is x 2024-02-14T14:51:24.968 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-14T14:51:24.969 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname $(hostname -s) 2024-02-14T14:51:25.000 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-02-14T14:51:25.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo hostname $(hostname -s) 2024-02-14T14:51:25.058 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-14T14:51:25.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:51:25.201 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76546/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1280-ga6fe3fa3', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1280.ga6fe3fa3'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-06 07:00:06.982899', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240206.051942', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-14T14:51:25.340 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-vshankar-testing-20240206.051942, sha1 a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:51:25.342 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/x86_64/flavors/default/cephadm 2024-02-14T14:51:25.342 DEBUG:teuthology.orchestra.run.smithi017:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:26.377 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r--. 1 ubuntu ubuntu 766939 Feb 14 14:51 /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:26.378 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:27.208 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 766939 Feb 14 14:51 /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:27.209 DEBUG:teuthology.orchestra.run.smithi076:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240206.051942/a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:28.103 INFO:teuthology.orchestra.run.smithi076.stdout:-rw-r--r--. 1 ubuntu ubuntu 766939 Feb 14 14:51 /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:28.103 DEBUG:teuthology.orchestra.run.smithi017:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:28.122 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:28.141 DEBUG:teuthology.orchestra.run.smithi076:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T14:51:28.173 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 on all hosts... 2024-02-14T14:51:28.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 pull 2024-02-14T14:51:28.176 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 pull 2024-02-14T14:51:28.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 pull 2024-02-14T14:51:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0... 2024-02-14T14:51:29.080 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0... 2024-02-14T14:51:29.108 INFO:teuthology.orchestra.run.smithi076.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0... 2024-02-14T14:52:15.152 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-02-14T14:52:15.152 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev)", 2024-02-14T14:52:15.153 INFO:teuthology.orchestra.run.smithi076.stdout: "image_id": "87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90", 2024-02-14T14:52:15.153 INFO:teuthology.orchestra.run.smithi076.stdout: "repo_digests": [ 2024-02-14T14:52:15.153 INFO:teuthology.orchestra.run.smithi076.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836" 2024-02-14T14:52:15.153 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-02-14T14:52:15.153 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-02-14T14:52:31.760 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T14:52:31.760 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev)", 2024-02-14T14:52:31.760 INFO:teuthology.orchestra.run.smithi017.stdout: "image_id": "87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90", 2024-02-14T14:52:31.760 INFO:teuthology.orchestra.run.smithi017.stdout: "repo_digests": [ 2024-02-14T14:52:31.760 INFO:teuthology.orchestra.run.smithi017.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836" 2024-02-14T14:52:31.761 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T14:52:31.761 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T14:52:36.953 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-02-14T14:52:36.953 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev)", 2024-02-14T14:52:36.953 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90", 2024-02-14T14:52:36.953 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-02-14T14:52:36.953 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836" 2024-02-14T14:52:36.953 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-02-14T14:52:36.953 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-02-14T14:52:36.978 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph 2024-02-14T14:52:37.013 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-02-14T14:52:37.048 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph 2024-02-14T14:52:37.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /etc/ceph 2024-02-14T14:52:37.111 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-02-14T14:52:37.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /etc/ceph 2024-02-14T14:52:37.171 INFO:tasks.cephadm:Writing seed config... 2024-02-14T14:52:37.172 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-14T14:52:37.173 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-14T14:52:37.174 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-14T14:52:37.175 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-02-14T14:52:37.176 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:52:37.176 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-14T14:52:37.192 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 867ff6ea-cb48-11ee-95b9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-14T14:52:37.193 DEBUG:teuthology.orchestra.run.smithi017:mon.a> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a.service 2024-02-14T14:52:37.235 DEBUG:teuthology.orchestra.run.smithi017:mgr.x> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.x.service 2024-02-14T14:52:37.277 INFO:tasks.cephadm:Bootstrapping... 2024-02-14T14:52:37.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 -v bootstrap --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.17 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-14T14:52:37.476 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-02-14T14:52:37.476 INFO:teuthology.orchestra.run.smithi017.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0', '-v', 'bootstrap', '--fsid', '867ff6ea-cb48-11ee-95b9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.17', '--skip-admin-label'] 2024-02-14T14:52:37.500 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T14:52:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-14T14:52:37.500 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying podman|docker is present... 2024-02-14T14:52:37.525 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T14:52:37.525 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying lvm2 is present... 2024-02-14T14:52:37.525 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying time synchronization is in place... 2024-02-14T14:52:37.532 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T14:52:37.532 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T14:52:37.538 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T14:52:37.538 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-02-14T14:52:37.544 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout enabled 2024-02-14T14:52:37.550 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout active 2024-02-14T14:52:37.550 INFO:teuthology.orchestra.run.smithi017.stdout:Unit chronyd.service is enabled and running 2024-02-14T14:52:37.550 INFO:teuthology.orchestra.run.smithi017.stdout:Repeating the final host check... 2024-02-14T14:52:37.572 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T14:52:37.572 INFO:teuthology.orchestra.run.smithi017.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-14T14:52:37.573 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl is present 2024-02-14T14:52:37.573 INFO:teuthology.orchestra.run.smithi017.stdout:lvcreate is present 2024-02-14T14:52:37.579 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T14:52:37.579 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T14:52:37.584 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T14:52:37.585 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-02-14T14:52:37.591 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout enabled 2024-02-14T14:52:37.597 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout active 2024-02-14T14:52:37.597 INFO:teuthology.orchestra.run.smithi017.stdout:Unit chronyd.service is enabled and running 2024-02-14T14:52:37.597 INFO:teuthology.orchestra.run.smithi017.stdout:Host looks OK 2024-02-14T14:52:37.597 INFO:teuthology.orchestra.run.smithi017.stdout:Cluster fsid: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:52:37.597 INFO:teuthology.orchestra.run.smithi017.stdout:Acquiring lock 140206500704208 on /run/cephadm/867ff6ea-cb48-11ee-95b9-87774f69a715.lock 2024-02-14T14:52:37.597 INFO:teuthology.orchestra.run.smithi017.stdout:Lock 140206500704208 acquired on /run/cephadm/867ff6ea-cb48-11ee-95b9-87774f69a715.lock 2024-02-14T14:52:37.598 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying IP 172.21.15.17 port 3300 ... 2024-02-14T14:52:37.598 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying IP 172.21.15.17 port 6789 ... 2024-02-14T14:52:37.598 INFO:teuthology.orchestra.run.smithi017.stdout:Base mon IP(s) is [172.21.15.17:3300, 172.21.15.17:6789], mon addrv is [v2:172.21.15.17:3300,v1:172.21.15.17:6789] 2024-02-14T14:52:37.601 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.17 metric 100 2024-02-14T14:52:37.601 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.17 metric 100 2024-02-14T14:52:37.604 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-14T14:52:37.604 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-14T14:52:37.604 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-14T14:52:37.606 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-14T14:52:37.606 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-14T14:52:37.607 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T14:52:37.607 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-14T14:52:37.607 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-02-14T14:52:37.607 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T14:52:37.608 INFO:teuthology.orchestra.run.smithi017.stdout:Mon IP `172.21.15.17` is in CIDR network `172.21.0.0/20` 2024-02-14T14:52:37.608 INFO:teuthology.orchestra.run.smithi017.stdout:Mon IP `172.21.15.17` is in CIDR network `172.21.0.0/20` 2024-02-14T14:52:37.608 INFO:teuthology.orchestra.run.smithi017.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-14T14:52:37.608 INFO:teuthology.orchestra.run.smithi017.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-14T14:52:37.608 INFO:teuthology.orchestra.run.smithi017.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0... 2024-02-14T14:52:38.170 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 2024-02-14T14:52:38.170 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0... 2024-02-14T14:52:38.170 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Getting image source signatures 2024-02-14T14:52:38.170 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-14T14:52:38.170 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying blob sha256:9c40cf8047e524d6cb701163cfe8fa35604e82efa085066a178ca58e787e46ad 2024-02-14T14:52:38.170 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying config sha256:87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 2024-02-14T14:52:38.170 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-14T14:52:39.150 INFO:teuthology.orchestra.run.smithi017.stdout:ceph: stdout ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T14:52:39.150 INFO:teuthology.orchestra.run.smithi017.stdout:Ceph version: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T14:52:39.150 INFO:teuthology.orchestra.run.smithi017.stdout:Extracting ceph user uid/gid from container image... 2024-02-14T14:52:39.987 INFO:teuthology.orchestra.run.smithi017.stdout:stat: stdout 167 167 2024-02-14T14:52:39.987 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial keys... 2024-02-14T14:52:40.926 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQC408xlRoG1ExAATG6knn7jrvlXKCa0+niN1A== 2024-02-14T14:52:41.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQC508xlBcceDhAADnT11HrcwUZ1bWLCGPlzlw== 2024-02-14T14:52:42.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQC608xl8B/rChAANJpVtBko8c6eigROAopAkA== 2024-02-14T14:52:42.764 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial monmap... 2024-02-14T14:52:43.687 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T14:52:43.687 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool for a [v2:172.21.15.17:3300,v1:172.21.15.17:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = quincy 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: set fsid to 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:52:43.688 INFO:teuthology.orchestra.run.smithi017.stdout:Creating mon... 2024-02-14T14:52:44.591 INFO:teuthology.orchestra.run.smithi017.stdout:create mon.a on 2024-02-14T14:52:44.887 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-14T14:52:45.026 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-14T14:52:45.206 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715.target. 2024-02-14T14:52:45.206 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715.target. 2024-02-14T14:52:45.441 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a 2024-02-14T14:52:45.441 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to reset failed state of unit ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a.service: Unit ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a.service not loaded. 2024-02-14T14:52:45.600 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715.target.wants/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@.service. 2024-02-14T14:52:46.024 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T14:52:46.024 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-02-14T14:52:46.031 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T14:52:46.031 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-02-14T14:52:46.032 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-02-14T14:52:46.032 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T14:52:46.032 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mon to start... 2024-02-14T14:52:46.032 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mon... 2024-02-14T14:52:47.099 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout cluster: 2024-02-14T14:52:47.099 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout id: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:52:47.099 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-14T14:52:47.099 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout services: 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.496913s) 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout data: 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout pgs: 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.445+0000 7fe7ed63a700 1 Processor -- start 2024-02-14T14:52:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.445+0000 7fe7ed63a700 1 -- start start 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.445+0000 7fe7ed63a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e8106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.445+0000 7fe7ed63a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e807ae50 con 0x7fe7e8105f50 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e8106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e8106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37578/0 (socket says 172.21.15.17:37578) 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e6ffd700 1 -- 172.21.15.17:0/428182409 learned_addr learned my addr 172.21.15.17:0/428182409 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e6ffd700 1 -- 172.21.15.17:0/428182409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e81068b0 con 0x7fe7e8105f50 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e6ffd700 1 --2- 172.21.15.17:0/428182409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e8106370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80098a0 tx=0x7fe7d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f942d3c93fbbd17 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e5ffb700 1 -- 172.21.15.17:0/428182409 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7d8004160 con 0x7fe7e8105f50 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e5ffb700 1 -- 172.21.15.17:0/428182409 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d80042c0 con 0x7fe7e8105f50 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.446+0000 7fe7e5ffb700 1 -- 172.21.15.17:0/428182409 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7d80045a0 con 0x7fe7e8105f50 2024-02-14T14:52:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.447+0000 7fe7ed63a700 1 -- 172.21.15.17:0/428182409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 msgr2=0x7fe7e8106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.447+0000 7fe7ed63a700 1 --2- 172.21.15.17:0/428182409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e8106370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80098a0 tx=0x7fe7d8009bb0 comp rx=0 tx=0).stop 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.447+0000 7fe7ed63a700 1 -- 172.21.15.17:0/428182409 shutdown_connections 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.447+0000 7fe7ed63a700 1 --2- 172.21.15.17:0/428182409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e8106370 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.447+0000 7fe7ed63a700 1 -- 172.21.15.17:0/428182409 >> 172.21.15.17:0/428182409 conn(0x7fe7e8101c10 msgr2=0x7fe7e8104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.447+0000 7fe7ed63a700 1 -- 172.21.15.17:0/428182409 shutdown_connections 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.447+0000 7fe7ed63a700 1 -- 172.21.15.17:0/428182409 wait complete. 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7ed63a700 1 Processor -- start 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7ed63a700 1 -- start start 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7ed63a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7ed63a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e819d370 con 0x7fe7e8105f50 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37594/0 (socket says 172.21.15.17:37594) 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7e6ffd700 1 -- 172.21.15.17:0/765604564 learned_addr learned my addr 172.21.15.17:0/765604564 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:47.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.448+0000 7fe7e6ffd700 1 -- 172.21.15.17:0/765604564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8009550 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7e6ffd700 1 --2- 172.21.15.17:0/765604564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e819ce30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe7e807a290 tx=0x7fe7d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7dffff700 1 -- 172.21.15.17:0/765604564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7d8011630 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7dffff700 1 -- 172.21.15.17:0/765604564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d8011790 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e819d570 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7dffff700 1 -- 172.21.15.17:0/765604564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7d8011a30 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e819da10 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7dffff700 1 -- 172.21.15.17:0/765604564 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe7d8021070 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.449+0000 7fe7dffff700 1 -- 172.21.15.17:0/765604564 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe7d801b7d0 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.450+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e8106c20 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.452+0000 7fe7dffff700 1 -- 172.21.15.17:0/765604564 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe7d801ba30 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.524+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe7e81a04c0 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.524+0000 7fe7dffff700 1 -- 172.21.15.17:0/765604564 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fe7d801bbd0 con 0x7fe7e8105f50 2024-02-14T14:52:47.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.525+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 msgr2=0x7fe7e819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.525+0000 7fe7ed63a700 1 --2- 172.21.15.17:0/765604564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e819ce30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe7e807a290 tx=0x7fe7d8004750 comp rx=0 tx=0).stop 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.525+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 shutdown_connections 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.525+0000 7fe7ed63a700 1 --2- 172.21.15.17:0/765604564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e8105f50 0x7fe7e819ce30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.525+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 >> 172.21.15.17:0/765604564 conn(0x7fe7e8101c10 msgr2=0x7fe7e81935d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.525+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 shutdown_connections 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:46.525+0000 7fe7ed63a700 1 -- 172.21.15.17:0/765604564 wait complete. 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:mon is available 2024-02-14T14:52:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:Assimilating anything we can from ceph.conf... 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout fsid = 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.17:3300,v1:172.21.15.17:6789] 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T14:52:48.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a913fe700 1 Processor -- start 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a913fe700 1 -- start start 2024-02-14T14:52:48.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a913fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a913fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c108620 con 0x7f8a8c107c30 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a8affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a8affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37610/0 (socket says 172.21.15.17:37610) 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a8affd700 1 -- 172.21.15.17:0/317172450 learned_addr learned my addr 172.21.15.17:0/317172450 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a8affd700 1 -- 172.21.15.17:0/317172450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c106800 con 0x7f8a8c107c30 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.492+0000 7f8a8affd700 1 --2- 172.21.15.17:0/317172450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c108050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009e80 tx=0x7f8a7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82efad5cff0105bc server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a89ffb700 1 -- 172.21.15.17:0/317172450 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c00e070 con 0x7f8a8c107c30 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a89ffb700 1 -- 172.21.15.17:0/317172450 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c003a20 con 0x7f8a8c107c30 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a89ffb700 1 -- 172.21.15.17:0/317172450 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c003ce0 con 0x7f8a8c107c30 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a913fe700 1 -- 172.21.15.17:0/317172450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 msgr2=0x7f8a8c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a913fe700 1 --2- 172.21.15.17:0/317172450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c108050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009e80 tx=0x7f8a7c00b330 comp rx=0 tx=0).stop 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a913fe700 1 -- 172.21.15.17:0/317172450 shutdown_connections 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a913fe700 1 --2- 172.21.15.17:0/317172450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c108050 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:48.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a913fe700 1 -- 172.21.15.17:0/317172450 >> 172.21.15.17:0/317172450 conn(0x7f8a8c1013b0 msgr2=0x7f8a8c1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a913fe700 1 -- 172.21.15.17:0/317172450 shutdown_connections 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.493+0000 7f8a913fe700 1 -- 172.21.15.17:0/317172450 wait complete. 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.494+0000 7f8a913fe700 1 Processor -- start 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.494+0000 7f8a913fe700 1 -- start start 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.494+0000 7f8a913fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c106190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.494+0000 7f8a913fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c19d8d0 con 0x7f8a8c107c30 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.494+0000 7f8a8affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.494+0000 7f8a8affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c106190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37620/0 (socket says 172.21.15.17:37620) 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.494+0000 7f8a8affd700 1 -- 172.21.15.17:0/4261468298 learned_addr learned my addr 172.21.15.17:0/4261468298 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a8affd700 1 -- 172.21.15.17:0/4261468298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c009b30 con 0x7f8a8c107c30 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a8affd700 1 --2- 172.21.15.17:0/4261468298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c106190 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0747e0 tx=0x7f8a7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c004370 con 0x7f8a8c107c30 2024-02-14T14:52:48.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c004510 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c0047f0 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c19da10 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c19ddf0 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8a7c0127c0 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.495+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a7c01dbf0 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.496+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c1937d0 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.498+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8a7c02b060 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.568+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8a8c068e40 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.571+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f8a7c021030 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.572+0000 7f8a83fff700 1 -- 172.21.15.17:0/4261468298 <== mon.0 v2:172.21.15.17:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a7c004950 con 0x7f8a8c107c30 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.573+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 msgr2=0x7f8a8c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.573+0000 7f8a913fe700 1 --2- 172.21.15.17:0/4261468298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c106190 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0747e0 tx=0x7f8a7c004ad0 comp rx=0 tx=0).stop 2024-02-14T14:52:48.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.573+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 shutdown_connections 2024-02-14T14:52:48.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.573+0000 7f8a913fe700 1 --2- 172.21.15.17:0/4261468298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c107c30 0x7f8a8c106190 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:48.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.573+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 >> 172.21.15.17:0/4261468298 conn(0x7f8a8c1013b0 msgr2=0x7f8a8c190910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:48.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.573+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 shutdown_connections 2024-02-14T14:52:48.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:47.573+0000 7f8a913fe700 1 -- 172.21.15.17:0/4261468298 wait complete. 2024-02-14T14:52:48.093 INFO:teuthology.orchestra.run.smithi017.stdout:Generating new minimal ceph.conf... 2024-02-14T14:52:49.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.504+0000 7f4ad1598700 1 Processor -- start 2024-02-14T14:52:49.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.504+0000 7f4ad1598700 1 -- start start 2024-02-14T14:52:49.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.505+0000 7f4ad1598700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc105d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:49.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.505+0000 7f4ad1598700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc106360 con 0x7f4acc105970 2024-02-14T14:52:49.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.505+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc105d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:49.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.505+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc105d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37636/0 (socket says 172.21.15.17:37636) 2024-02-14T14:52:49.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.505+0000 7f4acaffd700 1 -- 172.21.15.17:0/42326806 learned_addr learned my addr 172.21.15.17:0/42326806 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.505+0000 7f4acaffd700 1 -- 172.21.15.17:0/42326806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc1064a0 con 0x7f4acc105970 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.505+0000 7f4acaffd700 1 --2- 172.21.15.17:0/42326806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc105d90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00098a0 tx=0x7f4ac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99086a150ab9d196 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ac9ffb700 1 -- 172.21.15.17:0/42326806 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ac0004160 con 0x7f4acc105970 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ac9ffb700 1 -- 172.21.15.17:0/42326806 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac00042c0 con 0x7f4acc105970 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ac9ffb700 1 -- 172.21.15.17:0/42326806 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ac0011ac0 con 0x7f4acc105970 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ad1598700 1 -- 172.21.15.17:0/42326806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 msgr2=0x7f4acc105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ad1598700 1 --2- 172.21.15.17:0/42326806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc105d90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00098a0 tx=0x7f4ac0009bb0 comp rx=0 tx=0).stop 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ad1598700 1 -- 172.21.15.17:0/42326806 shutdown_connections 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ad1598700 1 --2- 172.21.15.17:0/42326806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc105d90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ad1598700 1 -- 172.21.15.17:0/42326806 >> 172.21.15.17:0/42326806 conn(0x7f4acc1013b0 msgr2=0x7f4acc1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ad1598700 1 -- 172.21.15.17:0/42326806 shutdown_connections 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.506+0000 7f4ad1598700 1 -- 172.21.15.17:0/42326806 wait complete. 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.507+0000 7f4ad1598700 1 Processor -- start 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.507+0000 7f4ad1598700 1 -- start start 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.507+0000 7f4ad1598700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.507+0000 7f4ad1598700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19ba00 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.507+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.507+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37638/0 (socket says 172.21.15.17:37638) 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.507+0000 7f4acaffd700 1 -- 172.21.15.17:0/133675449 learned_addr learned my addr 172.21.15.17:0/133675449 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4acaffd700 1 -- 172.21.15.17:0/133675449 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac0009550 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4acaffd700 1 --2- 172.21.15.17:0/133675449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc19b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0011780 tx=0x7f4ac0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4ab3fff700 1 -- 172.21.15.17:0/133675449 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ac0003af0 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc19bc00 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4ab3fff700 1 -- 172.21.15.17:0/133675449 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac0003c50 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4ab3fff700 1 -- 172.21.15.17:0/133675449 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ac00197e0 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc19c0a0 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4ab3fff700 1 -- 172.21.15.17:0/133675449 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4ac0025460 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.508+0000 7f4ab3fff700 1 -- 172.21.15.17:0/133675449 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4ac000bb20 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.509+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc068df0 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.511+0000 7f4ab3fff700 1 -- 172.21.15.17:0/133675449 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4ac002a060 con 0x7f4acc105970 2024-02-14T14:52:49.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.582+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4acc19ec30 con 0x7f4acc105970 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.582+0000 7f4ab3fff700 1 -- 172.21.15.17:0/133675449 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f4ac0019d80 con 0x7f4acc105970 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.583+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 msgr2=0x7f4acc19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.583+0000 7f4ad1598700 1 --2- 172.21.15.17:0/133675449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc19b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0011780 tx=0x7f4ac0011b40 comp rx=0 tx=0).stop 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.583+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 shutdown_connections 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.583+0000 7f4ad1598700 1 --2- 172.21.15.17:0/133675449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4acc105970 0x7f4acc19b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.583+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 >> 172.21.15.17:0/133675449 conn(0x7f4acc1013b0 msgr2=0x7f4acc101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.583+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 shutdown_connections 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:48.583+0000 7f4ad1598700 1 -- 172.21.15.17:0/133675449 wait complete. 2024-02-14T14:52:49.061 INFO:teuthology.orchestra.run.smithi017.stdout:Restarting the monitor... 2024-02-14T14:52:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 systemd[1]: Stopping Ceph mon.a for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T14:52:49.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32171]: 2024-02-14T14:52:49.312+0000 7ff3976ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T14:52:49.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32171]: 2024-02-14T14:52:49.312+0000 7ff3976ad700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-14T14:52:49.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 podman[32421]: 2024-02-14 14:52:49.333914102 +0000 UTC m=+0.116590704 container died 1a88f691508c0f20854d24a614dc13be2cbbcf290fea433d995af602a5c381c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T14:52:49.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 podman[32421]: 2024-02-14 14:52:49.620597956 +0000 UTC m=+0.403274600 container cleanup 1a88f691508c0f20854d24a614dc13be2cbbcf290fea433d995af602a5c381c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-14T14:52:49.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 bash[32421]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a 2024-02-14T14:52:49.970 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 podman[32435]: 2024-02-14 14:52:49.846302753 +0000 UTC m=+0.511710914 container remove 1a88f691508c0f20854d24a614dc13be2cbbcf290fea433d995af602a5c381c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-14T14:52:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-14T14:52:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 systemd[1]: Stopped Ceph mon.a for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T14:52:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:49 smithi017 systemd[1]: Starting Ceph mon.a for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T14:52:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 podman[32531]: 2024-02-14 14:52:50.188693925 +0000 UTC m=+0.103793402 container create 5ea5e790fba548fc9618650355e2bc949b3b985166bcdb775684e5501e41cf2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-14T14:52:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 podman[32531]: 2024-02-14 14:52:50.101598726 +0000 UTC m=+0.016698174 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T14:52:50.478 INFO:teuthology.orchestra.run.smithi017.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-14T14:52:50.605 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 podman[32531]: 2024-02-14 14:52:50.39923781 +0000 UTC m=+0.314337257 container init 5ea5e790fba548fc9618650355e2bc949b3b985166bcdb775684e5501e41cf2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-14T14:52:50.605 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 podman[32531]: 2024-02-14 14:52:50.401965711 +0000 UTC m=+0.317065149 container start 5ea5e790fba548fc9618650355e2bc949b3b985166bcdb775684e5501e41cf2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-14T14:52:50.605 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T14:52:50.606 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev), process ceph-mon, pid 2 2024-02-14T14:52:50.606 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: pidfile_write: ignore empty --pid-file 2024-02-14T14:52:50.606 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: load: jerasure load: lrc 2024-02-14T14:52:50.606 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: RocksDB version: 7.9.2 2024-02-14T14:52:50.606 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Git sha 0 2024-02-14T14:52:50.606 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Compile date 2024-02-06 05:38:06 2024-02-14T14:52:50.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: DB SUMMARY 2024-02-14T14:52:50.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: DB Session ID: NP2Y2W57LA84K3VWKNQJ 2024-02-14T14:52:50.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: CURRENT file: CURRENT 2024-02-14T14:52:50.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T14:52:50.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-14T14:52:50.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-14T14:52:50.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100288 ; 2024-02-14T14:52:50.608 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.error_if_exists: 0 2024-02-14T14:52:50.608 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.create_if_missing: 0 2024-02-14T14:52:50.608 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.paranoid_checks: 1 2024-02-14T14:52:50.608 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T14:52:50.608 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T14:52:50.608 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T14:52:50.608 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.env: 0x55f57a5edae0 2024-02-14T14:52:50.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T14:52:50.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.info_log: 0x55f57c71f8c0 2024-02-14T14:52:50.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T14:52:50.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.statistics: (nil) 2024-02-14T14:52:50.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.use_fsync: 0 2024-02-14T14:52:50.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_log_file_size: 0 2024-02-14T14:52:50.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.allow_fallocate: 1 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.use_direct_reads: 0 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.db_log_dir: 2024-02-14T14:52:50.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.wal_dir: 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.write_buffer_manager: 0x55f57c6f3400 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T14:52:50.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.unordered_write: 0 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.row_cache: None 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.wal_filter: None 2024-02-14T14:52:50.612 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.two_write_queues: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.wal_compression: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.atomic_flush: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.log_readahead_size: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T14:52:50.613 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_background_jobs: 2 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_background_compactions: -1 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_subcompactions: 1 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T14:52:50.614 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_open_files: -1 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_background_flushes: -1 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Compression algorithms supported: 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kZSTD supported: 0 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kXpressCompression supported: 0 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kZlibCompression supported: 1 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kSnappyCompression supported: 1 2024-02-14T14:52:50.615 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kLZ4Compression supported: 1 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: kBZip2Compression supported: 0 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.merge_operator: 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_filter: None 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T14:52:50.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f57c71f9c0) 2024-02-14T14:52:50.617 INFO:journalctl@ceph.mon.a.smithi017.stdout: cache_index_and_filter_blocks: 1 2024-02-14T14:52:50.617 INFO:journalctl@ceph.mon.a.smithi017.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T14:52:50.617 INFO:journalctl@ceph.mon.a.smithi017.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T14:52:50.617 INFO:journalctl@ceph.mon.a.smithi017.stdout: pin_top_level_index_and_filter: 1 2024-02-14T14:52:50.617 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_type: 0 2024-02-14T14:52:50.618 INFO:journalctl@ceph.mon.a.smithi017.stdout: data_block_index_type: 0 2024-02-14T14:52:50.618 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_shortening: 1 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: checksum: 4 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: no_block_cache: 0 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache: 0x55f57c723090 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_name: BinnedLRUCache 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_options: 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: capacity : 536870912 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: num_shard_bits : 4 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: strict_capacity_limit : 0 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: high_pri_pool_ratio: 0.000 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_compressed: (nil) 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: persistent_cache: (nil) 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_size: 4096 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_size_deviation: 10 2024-02-14T14:52:50.619 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_restart_interval: 16 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_block_restart_interval: 1 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: metadata_block_size: 4096 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: partition_filters: 0 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: use_delta_encoding: 1 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: filter_policy: bloomfilter 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: whole_key_filtering: 1 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: verify_compression: 0 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: read_amp_bytes_per_bit: 0 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: format_version: 5 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: enable_index_compression: 1 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_align: 0 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: max_auto_readahead_size: 262144 2024-02-14T14:52:50.620 INFO:journalctl@ceph.mon.a.smithi017.stdout: prepopulate_block_cache: 0 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout: initial_auto_readahead_size: 8192 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression: NoCompression 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.num_levels: 7 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T14:52:50.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T14:52:50.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T14:52:50.623 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T14:52:50.624 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T14:52:50.624 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T14:52:50.624 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T14:52:50.624 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T14:52:50.624 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T14:52:50.624 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T14:52:50.625 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.table_properties_collectors: 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.inplace_update_support: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.bloom_locality: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.max_successive_merges: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T14:52:50.626 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.ttl: 2592000 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.enable_blob_files: false 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.min_blob_size: 0 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T14:52:50.627 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 660ba651-5c72-449d-9da7-b5fb9355987e 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707922370430804, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707922370431719, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93412, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13914, "raw_average_key_size": 50, "raw_value_size": 85845, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707922370, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "660ba651-5c72-449d-9da7-b5fb9355987e", "db_session_id": "NP2Y2W57LA84K3VWKNQJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707922370431807, "job": 1, "event": "recovery_finished"} 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f57c814000 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: DB pointer 0x55f57c7fe000 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T14:52:50.628 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** DB Stats ** 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** Compaction Stats [default] ** 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: L0 2/0 95.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Sum 2/0 95.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T14:52:50.629 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** Compaction Stats [default] ** 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T14:52:50.630 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative compaction: 0.00 GB write, 31.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval compaction: 0.00 GB write, 31.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: Block cache BinnedLRUCache@0x55f57c723090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: starting mon.a rank 0 at public addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] at bind addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???) e1 preinit fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???).mds e1 new map 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???).mds e1 print_map 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: e1 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: legacy client fscid: -1 2024-02-14T14:52:50.631 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout: No filesystems configured 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 bash[32531]: 5ea5e790fba548fc9618650355e2bc949b3b985166bcdb775684e5501e41cf2a 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 systemd[1]: Started Ceph mon.a for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: monmap e1: 1 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: fsmap 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: osdmap e1: 0 total, 0 up, 0 in 2024-02-14T14:52:50.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:50 smithi017 ceph-mon[32549]: mgrmap e1: no daemons active 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.904+0000 7f553e7c7700 1 Processor -- start 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f553e7c7700 1 -- start start 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f553e7c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f553810a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f553e7c7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553807ae50 con 0x7f5538108180 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f5537fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f553810a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f5537fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f553810a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52120/0 (socket says 172.21.15.17:52120) 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f5537fff700 1 -- 172.21.15.17:0/2378905955 learned_addr learned my addr 172.21.15.17:0/2378905955 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:51.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f5537fff700 1 -- 172.21.15.17:0/2378905955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553810aab0 con 0x7f5538108180 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.905+0000 7f5537fff700 1 --2- 172.21.15.17:0/2378905955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f553810a570 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55280098a0 tx=0x7f5528009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c70ee8e82f5898 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f5536ffd700 1 -- 172.21.15.17:0/2378905955 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5528004160 con 0x7f5538108180 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f5536ffd700 1 -- 172.21.15.17:0/2378905955 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55280042c0 con 0x7f5538108180 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f5536ffd700 1 -- 172.21.15.17:0/2378905955 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5528011ac0 con 0x7f5538108180 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f553e7c7700 1 -- 172.21.15.17:0/2378905955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 msgr2=0x7f553810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f553e7c7700 1 --2- 172.21.15.17:0/2378905955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f553810a570 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55280098a0 tx=0x7f5528009bb0 comp rx=0 tx=0).stop 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f553e7c7700 1 -- 172.21.15.17:0/2378905955 shutdown_connections 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f553e7c7700 1 --2- 172.21.15.17:0/2378905955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f553810a570 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f553e7c7700 1 -- 172.21.15.17:0/2378905955 >> 172.21.15.17:0/2378905955 conn(0x7f5538101bb0 msgr2=0x7f5538103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f553e7c7700 1 -- 172.21.15.17:0/2378905955 shutdown_connections 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.906+0000 7f553e7c7700 1 -- 172.21.15.17:0/2378905955 wait complete. 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.907+0000 7f553e7c7700 1 Processor -- start 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.907+0000 7f553e7c7700 1 -- start start 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.907+0000 7f553e7c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f5538107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:51.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.907+0000 7f553e7c7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5538107f40 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.907+0000 7f5537fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f5538107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.907+0000 7f5537fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f5538107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52134/0 (socket says 172.21.15.17:52134) 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.907+0000 7f5537fff700 1 -- 172.21.15.17:0/189510218 learned_addr learned my addr 172.21.15.17:0/189510218 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f5537fff700 1 -- 172.21.15.17:0/189510218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528009550 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f5537fff700 1 --2- 172.21.15.17:0/189510218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f5538107a00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5528009130 tx=0x7f5528011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f55357fa700 1 -- 172.21.15.17:0/189510218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5528003aa0 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f55357fa700 1 -- 172.21.15.17:0/189510218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5528003c00 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5538106130 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f55357fa700 1 -- 172.21.15.17:0/189510218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5528024800 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55381065d0 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f55357fa700 1 -- 172.21.15.17:0/189510218 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f55280193c0 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.908+0000 7f55357fa700 1 -- 172.21.15.17:0/189510218 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f552800baf0 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.909+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553810b8b0 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.911+0000 7f55357fa700 1 -- 172.21.15.17:0/189510218 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f552800b3e0 con 0x7f5538108180 2024-02-14T14:52:51.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.981+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f5538106810 con 0x7f5538108180 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f55357fa700 1 -- 172.21.15.17:0/189510218 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f552801abd0 con 0x7f5538108180 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 msgr2=0x7f5538107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f553e7c7700 1 --2- 172.21.15.17:0/189510218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f5538107a00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5528009130 tx=0x7f5528011b40 comp rx=0 tx=0).stop 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 shutdown_connections 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f553e7c7700 1 --2- 172.21.15.17:0/189510218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5538108180 0x7f5538107a00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 >> 172.21.15.17:0/189510218 conn(0x7f5538101bb0 msgr2=0x7f5538102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 shutdown_connections 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:50.983+0000 7f553e7c7700 1 -- 172.21.15.17:0/189510218 wait complete. 2024-02-14T14:52:51.557 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-14T14:52:51.580 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-14T14:52:51.580 INFO:teuthology.orchestra.run.smithi017.stdout:Creating mgr... 2024-02-14T14:52:51.580 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-14T14:52:51.581 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-14T14:52:51.815 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.x 2024-02-14T14:52:51.815 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to reset failed state of unit ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.x.service: Unit ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.x.service not loaded. 2024-02-14T14:52:51.954 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715.target.wants/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@.service. 2024-02-14T14:52:52.157 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:51 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/189510218' entity='client.admin' 2024-02-14T14:52:52.426 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:52 smithi017 podman[32761]: 2024-02-14 14:52:52.361880726 +0000 UTC m=+0.291354207 container start 75150f7c5ceea6e130a14c8057c39357b00491a297bad7d2ab497769f017ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T14:52:52.426 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:52 smithi017 bash[32761]: 75150f7c5ceea6e130a14c8057c39357b00491a297bad7d2ab497769f017ddb9 2024-02-14T14:52:52.439 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T14:52:52.439 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-02-14T14:52:52.446 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T14:52:52.446 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-02-14T14:52:52.446 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-02-14T14:52:52.446 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T14:52:52.454 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T14:52:52.454 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-02-14T14:52:52.461 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T14:52:52.461 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-02-14T14:52:52.461 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-02-14T14:52:52.462 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-14T14:52:52.462 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr to start... 2024-02-14T14:52:52.462 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr... 2024-02-14T14:52:52.731 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:52 smithi017 systemd[1]: Started Ceph mgr.x for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T14:52:53.001 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:52.928+0000 7f677ebea200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T14:52:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:53 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2868442098' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T14:52:53.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:53.030+0000 7f677ebea200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T14:52:53.534 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2024-02-14T14:52:53.537 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T14:52:53.538 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T14:52:53.546 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T14:52:53.547 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T14:52:46.029667+0000", 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.920+0000 7f396523f700 1 Processor -- start 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.920+0000 7f396523f700 1 -- start start 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.920+0000 7f396523f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f396010a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.920+0000 7f396523f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396007ae70 con 0x7f3960108120 2024-02-14T14:52:53.548 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f396010a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f396010a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52166/0 (socket says 172.21.15.17:52166) 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395ed9d700 1 -- 172.21.15.17:0/3231026165 learned_addr learned my addr 172.21.15.17:0/3231026165 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395ed9d700 1 -- 172.21.15.17:0/3231026165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396010aa50 con 0x7f3960108120 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395ed9d700 1 --2- 172.21.15.17:0/3231026165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f396010a510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39500098a0 tx=0x7f3950009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42910ecaf881728f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395dd9b700 1 -- 172.21.15.17:0/3231026165 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3950004160 con 0x7f3960108120 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395dd9b700 1 -- 172.21.15.17:0/3231026165 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39500042c0 con 0x7f3960108120 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.921+0000 7f395dd9b700 1 -- 172.21.15.17:0/3231026165 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3950011ac0 con 0x7f3960108120 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.922+0000 7f396523f700 1 -- 172.21.15.17:0/3231026165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 msgr2=0x7f396010a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.922+0000 7f396523f700 1 --2- 172.21.15.17:0/3231026165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f396010a510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39500098a0 tx=0x7f3950009bb0 comp rx=0 tx=0).stop 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.922+0000 7f396523f700 1 -- 172.21.15.17:0/3231026165 shutdown_connections 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.922+0000 7f396523f700 1 --2- 172.21.15.17:0/3231026165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f396010a510 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.922+0000 7f396523f700 1 -- 172.21.15.17:0/3231026165 >> 172.21.15.17:0/3231026165 conn(0x7f3960101b90 msgr2=0x7f3960103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:53.549 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.922+0000 7f396523f700 1 -- 172.21.15.17:0/3231026165 shutdown_connections 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.922+0000 7f396523f700 1 -- 172.21.15.17:0/3231026165 wait complete. 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.923+0000 7f396523f700 1 Processor -- start 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.923+0000 7f396523f700 1 -- start start 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.923+0000 7f396523f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f39601079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.923+0000 7f396523f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3960107ee0 con 0x7f3960108120 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.923+0000 7f395ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f39601079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.923+0000 7f395ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f39601079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52174/0 (socket says 172.21.15.17:52174) 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.923+0000 7f395ed9d700 1 -- 172.21.15.17:0/2868442098 learned_addr learned my addr 172.21.15.17:0/2868442098 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f395ed9d700 1 -- 172.21.15.17:0/2868442098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3950009550 con 0x7f3960108120 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f395ed9d700 1 --2- 172.21.15.17:0/2868442098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f39601079a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3950009fb0 tx=0x7f3950011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f3957fff700 1 -- 172.21.15.17:0/2868442098 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39500038b0 con 0x7f3960108120 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39601060d0 con 0x7f3960108120 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f3957fff700 1 -- 172.21.15.17:0/2868442098 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3950003a10 con 0x7f3960108120 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f3957fff700 1 -- 172.21.15.17:0/2868442098 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39500195e0 con 0x7f3960108120 2024-02-14T14:52:53.550 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3960106570 con 0x7f3960108120 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f3957fff700 1 -- 172.21.15.17:0/2868442098 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3950019c60 con 0x7f3960108120 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.924+0000 7f3957fff700 1 -- 172.21.15.17:0/2868442098 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3950024990 con 0x7f3960108120 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.925+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396010b850 con 0x7f3960108120 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:52.927+0000 7f3957fff700 1 -- 172.21.15.17:0/2868442098 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3950003b80 con 0x7f3960108120 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.001+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f396010ba40 con 0x7f3960108120 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.001+0000 7f3957fff700 1 -- 172.21.15.17:0/2868442098 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f395001a9f0 con 0x7f3960108120 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.002+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 msgr2=0x7f39601079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.002+0000 7f396523f700 1 --2- 172.21.15.17:0/2868442098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f39601079a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3950009fb0 tx=0x7f3950011b20 comp rx=0 tx=0).stop 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.002+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 shutdown_connections 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.002+0000 7f396523f700 1 --2- 172.21.15.17:0/2868442098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3960108120 0x7f39601079a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.002+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 >> 172.21.15.17:0/2868442098 conn(0x7f3960101b90 msgr2=0x7f3960102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.002+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 shutdown_connections 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:53.002+0000 7f396523f700 1 -- 172.21.15.17:0/2868442098 wait complete. 2024-02-14T14:52:53.551 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (1/15)... 2024-02-14T14:52:54.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:53.697+0000 7f677ebea200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T14:52:54.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:53.791+0000 7f677ebea200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T14:52:54.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:53.934+0000 7f677ebea200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T14:52:54.381 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:54.031+0000 7f677ebea200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T14:52:54.381 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:54.115+0000 7f677ebea200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T14:52:54.381 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:54.380+0000 7f677ebea200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T14:52:54.676 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:54.492+0000 7f677ebea200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T14:52:54.676 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:54.579+0000 7f677ebea200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T14:52:54.676 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:54.675+0000 7f677ebea200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T14:52:55.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:54.852+0000 7f677ebea200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T14:52:55.911 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:55 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:55.861+0000 7f677ebea200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T14:52:56.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:56.024+0000 7f677ebea200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T14:52:56.661 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:56 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2215248176' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T14:52:56.661 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:56.364+0000 7f677ebea200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T14:52:56.661 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:56.456+0000 7f677ebea200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T14:52:56.661 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:56.659+0000 7f677ebea200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:52:57.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:56.847+0000 7f677ebea200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T14:52:57.403 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:57 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:57.136+0000 7f677ebea200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:52:57.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:57 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:57.401+0000 7f677ebea200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T14:52:57.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:57 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:57.642+0000 7f677ebea200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T14:52:58.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:58 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:58.474+0000 7f677ebea200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T14:52:59.170 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:59.066+0000 7f677ebea200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T14:52:59.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:59.167+0000 7f677ebea200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T14:52:59.456 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:52:59.390+0000 7f677ebea200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: Activating manager daemon x 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: mgrmap e2: x(active, starting, since 0.00283203s) 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T14:52:59.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: Manager daemon x is now available 2024-02-14T14:52:59.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-14T14:52:59.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-14T14:52:59.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' 2024-02-14T14:52:59.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' 2024-02-14T14:52:59.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:52:59 smithi017 ceph-mon[32549]: from='mgr.14100 172.21.15.17:0/4019197442' entity='mgr.x' 2024-02-14T14:53:01.542 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:53:01.542 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-02-14T14:53:01.542 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T14:53:01.542 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T14:53:01.542 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T14:53:01.542 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T14:53:01.542 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T14:53:01.543 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:01.544 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T14:53:01.544 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T14:53:01.544 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T14:53:01.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T14:53:01.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T14:53:01.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T14:53:01.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:01.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T14:53:01.551 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T14:53:01.552 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T14:52:46.029667+0000", 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:01.553 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.909+0000 7f62c9ab8700 1 Processor -- start 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.909+0000 7f62c9ab8700 1 -- start start 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.909+0000 7f62c9ab8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c410a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c9ab8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c407ae50 con 0x7f62c4108160 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c410a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c410a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52186/0 (socket says 172.21.15.17:52186) 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c37fe700 1 -- 172.21.15.17:0/287970875 learned_addr learned my addr 172.21.15.17:0/287970875 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c37fe700 1 -- 172.21.15.17:0/287970875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c410aa90 con 0x7f62c4108160 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c37fe700 1 --2- 172.21.15.17:0/287970875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c410a550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0098a0 tx=0x7f62ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862d5968fddee887 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c27fc700 1 -- 172.21.15.17:0/287970875 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62ac004160 con 0x7f62c4108160 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c27fc700 1 -- 172.21.15.17:0/287970875 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62ac0042c0 con 0x7f62c4108160 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.910+0000 7f62c27fc700 1 -- 172.21.15.17:0/287970875 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62ac011ac0 con 0x7f62c4108160 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.911+0000 7f62c9ab8700 1 -- 172.21.15.17:0/287970875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 msgr2=0x7f62c410a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.911+0000 7f62c9ab8700 1 --2- 172.21.15.17:0/287970875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c410a550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0098a0 tx=0x7f62ac009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:01.554 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.911+0000 7f62c9ab8700 1 -- 172.21.15.17:0/287970875 shutdown_connections 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.911+0000 7f62c9ab8700 1 --2- 172.21.15.17:0/287970875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c410a550 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.911+0000 7f62c9ab8700 1 -- 172.21.15.17:0/287970875 >> 172.21.15.17:0/287970875 conn(0x7f62c4101b90 msgr2=0x7f62c4103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.911+0000 7f62c9ab8700 1 -- 172.21.15.17:0/287970875 shutdown_connections 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.911+0000 7f62c9ab8700 1 -- 172.21.15.17:0/287970875 wait complete. 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.912+0000 7f62c9ab8700 1 Processor -- start 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.912+0000 7f62c9ab8700 1 -- start start 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.912+0000 7f62c9ab8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.912+0000 7f62c9ab8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c419b980 con 0x7f62c4108160 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.912+0000 7f62c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.912+0000 7f62c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52198/0 (socket says 172.21.15.17:52198) 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.912+0000 7f62c37fe700 1 -- 172.21.15.17:0/2215248176 learned_addr learned my addr 172.21.15.17:0/2215248176 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c37fe700 1 -- 172.21.15.17:0/2215248176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac009550 con 0x7f62c4108160 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c37fe700 1 --2- 172.21.15.17:0/2215248176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c419b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009130 tx=0x7f62ac011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c0ff9700 1 -- 172.21.15.17:0/2215248176 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62ac003aa0 con 0x7f62c4108160 2024-02-14T14:53:01.555 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c419bb80 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c419c020 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c0ff9700 1 -- 172.21.15.17:0/2215248176 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62ac003c00 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c0ff9700 1 -- 172.21.15.17:0/2215248176 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62ac024800 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c0ff9700 1 -- 172.21.15.17:0/2215248176 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f62ac02d3f0 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.913+0000 7f62c0ff9700 1 -- 172.21.15.17:0/2215248176 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f62ac0197d0 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.914+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a4005360 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.917+0000 7f62c0ff9700 1 -- 172.21.15.17:0/2215248176 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f62ac01abd0 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.992+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f62a40058e0 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.992+0000 7f62c0ff9700 1 -- 172.21.15.17:0/2215248176 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f62ac01ad70 con 0x7f62c4108160 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.993+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 msgr2=0x7f62c419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.993+0000 7f62c9ab8700 1 --2- 172.21.15.17:0/2215248176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c419b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009130 tx=0x7f62ac011b40 comp rx=0 tx=0).stop 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.994+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 shutdown_connections 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.994+0000 7f62c9ab8700 1 --2- 172.21.15.17:0/2215248176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c4108160 0x7f62c419b440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.994+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 >> 172.21.15.17:0/2215248176 conn(0x7f62c4101b90 msgr2=0x7f62c4102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.994+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 shutdown_connections 2024-02-14T14:53:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:52:55.994+0000 7f62c9ab8700 1 -- 172.21.15.17:0/2215248176 wait complete. 2024-02-14T14:53:01.557 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (2/15)... 2024-02-14T14:53:01.690 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:01 smithi017 ceph-mon[32549]: mgrmap e3: x(active, since 1.00704s) 2024-02-14T14:53:03.676 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:03 smithi017 ceph-mon[32549]: mgrmap e4: x(active, since 3s) 2024-02-14T14:53:06.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:53:06.160 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-02-14T14:53:06.161 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T14:53:06.162 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T14:53:06.170 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T14:53:06.170 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T14:53:06.170 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T14:53:06.170 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:06.170 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T14:53:06.170 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T14:53:06.170 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T14:53:06.171 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T14:52:46.029667+0000", 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.311+0000 7fed40196700 1 Processor -- start 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.311+0000 7fed40196700 1 -- start start 2024-02-14T14:53:06.172 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.312+0000 7fed40196700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed381069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.312+0000 7fed40196700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38106f80 con 0x7fed38106590 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.312+0000 7fed3df32700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed381069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.312+0000 7fed3df32700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed381069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34260/0 (socket says 172.21.15.17:34260) 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.312+0000 7fed3df32700 1 -- 172.21.15.17:0/3557191078 learned_addr learned my addr 172.21.15.17:0/3557191078 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.312+0000 7fed3df32700 1 -- 172.21.15.17:0/3557191078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38107790 con 0x7fed38106590 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.312+0000 7fed3df32700 1 --2- 172.21.15.17:0/3557191078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed381069b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed340098a0 tx=0x7fed34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6fb56998200c70 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed3cf30700 1 -- 172.21.15.17:0/3557191078 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed34004160 con 0x7fed38106590 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed3cf30700 1 -- 172.21.15.17:0/3557191078 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed340042c0 con 0x7fed38106590 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed3cf30700 1 -- 172.21.15.17:0/3557191078 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed34011ac0 con 0x7fed38106590 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed40196700 1 -- 172.21.15.17:0/3557191078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 msgr2=0x7fed381069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed40196700 1 --2- 172.21.15.17:0/3557191078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed381069b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed340098a0 tx=0x7fed34009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed40196700 1 -- 172.21.15.17:0/3557191078 shutdown_connections 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed40196700 1 --2- 172.21.15.17:0/3557191078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed381069b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed40196700 1 -- 172.21.15.17:0/3557191078 >> 172.21.15.17:0/3557191078 conn(0x7fed38101b90 msgr2=0x7fed38103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:06.173 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed40196700 1 -- 172.21.15.17:0/3557191078 shutdown_connections 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.313+0000 7fed40196700 1 -- 172.21.15.17:0/3557191078 wait complete. 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.314+0000 7fed40196700 1 Processor -- start 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.314+0000 7fed40196700 1 -- start start 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.314+0000 7fed40196700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed3819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.314+0000 7fed40196700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3819dbe0 con 0x7fed38106590 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.314+0000 7fed3df32700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed3819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.314+0000 7fed3df32700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed3819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34262/0 (socket says 172.21.15.17:34262) 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.314+0000 7fed3df32700 1 -- 172.21.15.17:0/2769886746 learned_addr learned my addr 172.21.15.17:0/2769886746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.315+0000 7fed3df32700 1 -- 172.21.15.17:0/2769886746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009550 con 0x7fed38106590 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.315+0000 7fed3df32700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed3819d6a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fed34003f70 tx=0x7fed34011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.315+0000 7fed2effd700 1 -- 172.21.15.17:0/2769886746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed340039f0 con 0x7fed38106590 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.315+0000 7fed2effd700 1 -- 172.21.15.17:0/2769886746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed34003b50 con 0x7fed38106590 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.315+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3819dde0 con 0x7fed38106590 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.315+0000 7fed2effd700 1 -- 172.21.15.17:0/2769886746 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed3401a690 con 0x7fed38106590 2024-02-14T14:53:06.174 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.315+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3819e280 con 0x7fed38106590 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.316+0000 7fed2effd700 1 -- 172.21.15.17:0/2769886746 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7fed34024430 con 0x7fed38106590 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.316+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed38197490 con 0x7fed38106590 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.316+0000 7fed2effd700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fed24038e40 0x7fed2403b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.316+0000 7fed2effd700 1 -- 172.21.15.17:0/2769886746 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fed3404d0e0 con 0x7fed38106590 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.316+0000 7fed3d731700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fed24038e40 0x7fed2403b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.317+0000 7fed3d731700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fed24038e40 0x7fed2403b300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed28006fd0 tx=0x7fed28006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.321+0000 7fed2effd700 1 -- 172.21.15.17:0/2769886746 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed34018c20 con 0x7fed38106590 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.607+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fed3802d970 con 0x7fed38106590 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.607+0000 7fed2effd700 1 -- 172.21.15.17:0/2769886746 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fed34018350 con 0x7fed38106590 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fed24038e40 msgr2=0x7fed2403b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fed24038e40 0x7fed2403b300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed28006fd0 tx=0x7fed28006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 msgr2=0x7fed3819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed3819d6a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fed34003f70 tx=0x7fed34011b40 comp rx=0 tx=0).stop 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 shutdown_connections 2024-02-14T14:53:06.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fed24038e40 0x7fed2403b300 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:06.176 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 --2- 172.21.15.17:0/2769886746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed38106590 0x7fed3819d6a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:06.176 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 >> 172.21.15.17:0/2769886746 conn(0x7fed38101b90 msgr2=0x7fed38103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:06.176 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 shutdown_connections 2024-02-14T14:53:06.176 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:05.609+0000 7fed40196700 1 -- 172.21.15.17:0/2769886746 wait complete. 2024-02-14T14:53:06.176 INFO:teuthology.orchestra.run.smithi017.stdout:mgr is available 2024-02-14T14:53:06.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:06 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2769886746' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout fsid = 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T14:53:07.346 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.571+0000 7f9a95848700 1 Processor -- start 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.571+0000 7f9a95848700 1 -- start start 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.571+0000 7f9a95848700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a901063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.571+0000 7f9a95848700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9007ae50 con 0x7f9a90105f80 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.571+0000 7f9a8effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a901063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.571+0000 7f9a8effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a901063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34274/0 (socket says 172.21.15.17:34274) 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.571+0000 7f9a8effd700 1 -- 172.21.15.17:0/3497358457 learned_addr learned my addr 172.21.15.17:0/3497358457 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:07.347 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.572+0000 7f9a8effd700 1 -- 172.21.15.17:0/3497358457 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a901068e0 con 0x7f9a90105f80 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.572+0000 7f9a8effd700 1 --2- 172.21.15.17:0/3497358457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a901063a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9a800098a0 tx=0x7f9a80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94d5393c222021c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.572+0000 7f9a8dffb700 1 -- 172.21.15.17:0/3497358457 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a80004160 con 0x7f9a90105f80 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.572+0000 7f9a8dffb700 1 -- 172.21.15.17:0/3497358457 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a800042c0 con 0x7f9a90105f80 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.572+0000 7f9a8dffb700 1 -- 172.21.15.17:0/3497358457 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a80011ac0 con 0x7f9a90105f80 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 -- 172.21.15.17:0/3497358457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 msgr2=0x7f9a901063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 --2- 172.21.15.17:0/3497358457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a901063a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9a800098a0 tx=0x7f9a80009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 -- 172.21.15.17:0/3497358457 shutdown_connections 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 --2- 172.21.15.17:0/3497358457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a901063a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 -- 172.21.15.17:0/3497358457 >> 172.21.15.17:0/3497358457 conn(0x7f9a90101c60 msgr2=0x7f9a90104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 -- 172.21.15.17:0/3497358457 shutdown_connections 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 -- 172.21.15.17:0/3497358457 wait complete. 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.573+0000 7f9a95848700 1 Processor -- start 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a95848700 1 -- start start 2024-02-14T14:53:07.348 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a95848700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a9019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a95848700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9019dbc0 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a8effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a9019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a8effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a9019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34280/0 (socket says 172.21.15.17:34280) 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a8effd700 1 -- 172.21.15.17:0/3625310347 learned_addr learned my addr 172.21.15.17:0/3625310347 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a8effd700 1 -- 172.21.15.17:0/3625310347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80009550 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a8effd700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a9019d680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009fb0 tx=0x7f9a80011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a94846700 1 -- 172.21.15.17:0/3625310347 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a800038b0 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.574+0000 7f9a94846700 1 -- 172.21.15.17:0/3625310347 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a80003a10 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.575+0000 7f9a94846700 1 -- 172.21.15.17:0/3625310347 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a800195e0 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.575+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9019ddc0 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.575+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9019e260 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.575+0000 7f9a94846700 1 -- 172.21.15.17:0/3625310347 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f9a80019850 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.576+0000 7f9a94846700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f9a7c038e90 0x7f9a7c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.575+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a901972a0 con 0x7f9a90105f80 2024-02-14T14:53:07.349 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.576+0000 7f9a94846700 1 -- 172.21.15.17:0/3625310347 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9a8004d230 con 0x7f9a90105f80 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.576+0000 7f9a8e7fc700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f9a7c038e90 0x7f9a7c03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.576+0000 7f9a8e7fc700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f9a7c038e90 0x7f9a7c03b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a78006fd0 tx=0x7f9a78006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.581+0000 7f9a94846700 1 -- 172.21.15.17:0/3625310347 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a80019b00 con 0x7f9a90105f80 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.781+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9a9002d820 con 0x7f9a90105f80 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.782+0000 7f9a94846700 1 -- 172.21.15.17:0/3625310347 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9a80018350 con 0x7f9a90105f80 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f9a7c038e90 msgr2=0x7f9a7c03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f9a7c038e90 0x7f9a7c03b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a78006fd0 tx=0x7f9a78006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 msgr2=0x7f9a9019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a9019d680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009fb0 tx=0x7f9a80011b20 comp rx=0 tx=0).stop 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 shutdown_connections 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f9a7c038e90 0x7f9a7c03b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 --2- 172.21.15.17:0/3625310347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a90105f80 0x7f9a9019d680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 >> 172.21.15.17:0/3625310347 conn(0x7f9a90101c60 msgr2=0x7f9a901943e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 shutdown_connections 2024-02-14T14:53:07.350 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:06.784+0000 7f9a95848700 1 -- 172.21.15.17:0/3625310347 wait complete. 2024-02-14T14:53:07.351 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling cephadm module... 2024-02-14T14:53:07.639 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:07 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3625310347' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-14T14:53:08.751 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:08 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4181905242' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-14T14:53:08.751 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:08 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: ignoring --setuser ceph since I am not root 2024-02-14T14:53:08.751 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:08 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: ignoring --setgroup ceph since I am not root 2024-02-14T14:53:09.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.743+0000 7f1bea52a700 1 Processor -- start 2024-02-14T14:53:09.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.743+0000 7f1bea52a700 1 -- start start 2024-02-14T14:53:09.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.743+0000 7f1bea52a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:09.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.743+0000 7f1bea52a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be407ae30 con 0x7f1be41081a0 2024-02-14T14:53:09.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.743+0000 7f1be3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:09.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.743+0000 7f1be3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34288/0 (socket says 172.21.15.17:34288) 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.743+0000 7f1be3fff700 1 -- 172.21.15.17:0/1927092209 learned_addr learned my addr 172.21.15.17:0/1927092209 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.744+0000 7f1be3fff700 1 -- 172.21.15.17:0/1927092209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be410aad0 con 0x7f1be41081a0 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.744+0000 7f1be3fff700 1 --2- 172.21.15.17:0/1927092209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be410a590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0098a0 tx=0x7f1bcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882892811eb4b37f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.744+0000 7f1be2ffd700 1 -- 172.21.15.17:0/1927092209 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc004160 con 0x7f1be41081a0 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.744+0000 7f1be2ffd700 1 -- 172.21.15.17:0/1927092209 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bcc0042c0 con 0x7f1be41081a0 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.744+0000 7f1be2ffd700 1 -- 172.21.15.17:0/1927092209 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc011ac0 con 0x7f1be41081a0 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 -- 172.21.15.17:0/1927092209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 msgr2=0x7f1be410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 --2- 172.21.15.17:0/1927092209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be410a590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0098a0 tx=0x7f1bcc009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 -- 172.21.15.17:0/1927092209 shutdown_connections 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 --2- 172.21.15.17:0/1927092209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be410a590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 -- 172.21.15.17:0/1927092209 >> 172.21.15.17:0/1927092209 conn(0x7f1be4101bb0 msgr2=0x7f1be4103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 -- 172.21.15.17:0/1927092209 shutdown_connections 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 -- 172.21.15.17:0/1927092209 wait complete. 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 Processor -- start 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.745+0000 7f1bea52a700 1 -- start start 2024-02-14T14:53:09.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1bea52a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1bea52a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4107f60 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34298/0 (socket says 172.21.15.17:34298) 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be3fff700 1 -- 172.21.15.17:0/4181905242 learned_addr learned my addr 172.21.15.17:0/4181905242 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be3fff700 1 -- 172.21.15.17:0/4181905242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc009550 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be3fff700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be4107a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009130 tx=0x7f1bcc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc003aa0 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bcc003c00 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be4106150 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc024800 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.746+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be41065f0 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.747+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f1bcc024a90 con 0x7f1be41081a0 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.747+0000 7f1be17fa700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f1bd0038ee0 0x7f1bd003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:09.060 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.747+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1bcc04cca0 con 0x7f1be41081a0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.747+0000 7f1be37fe700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f1bd0038ee0 0x7f1bd003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.747+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be4063350 con 0x7f1be41081a0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.748+0000 7f1be37fe700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f1bd0038ee0 0x7f1bd003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4006fd0 tx=0x7f1bd4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:07.752+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1bcc024d40 con 0x7f1be41081a0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.014+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1be4106a40 con 0x7f1be41081a0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.441+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f1bcc02c350 con 0x7f1be41081a0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.443+0000 7f1be17fa700 1 -- 172.21.15.17:0/4181905242 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f1bcc018350 con 0x7f1be41081a0 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.444+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f1bd0038ee0 msgr2=0x7f1bd003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.444+0000 7f1bea52a700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f1bd0038ee0 0x7f1bd003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4006fd0 tx=0x7f1bd4006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.444+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 msgr2=0x7f1be4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.444+0000 7f1bea52a700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be4107a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009130 tx=0x7f1bcc011b40 comp rx=0 tx=0).stop 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.445+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 shutdown_connections 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.445+0000 7f1bea52a700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7f1bd0038ee0 0x7f1bd003b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.445+0000 7f1bea52a700 1 --2- 172.21.15.17:0/4181905242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1be41081a0 0x7f1be4107a20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:09.061 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.445+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 >> 172.21.15.17:0/4181905242 conn(0x7f1be4101bb0 msgr2=0x7f1be4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.445+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 shutdown_connections 2024-02-14T14:53:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:08.445+0000 7f1bea52a700 1 -- 172.21.15.17:0/4181905242 wait complete. 2024-02-14T14:53:09.268 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:09.015+0000 7f9353c38200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T14:53:09.268 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:09.113+0000 7f9353c38200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T14:53:09.659 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:09 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4181905242' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-14T14:53:09.659 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:09 smithi017 ceph-mon[32549]: mgrmap e5: x(active, since 9s) 2024-02-14T14:53:10.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:09.783+0000 7f9353c38200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T14:53:10.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:09.878+0000 7f9353c38200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T14:53:10.444 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.023+0000 7f9353c38200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T14:53:10.444 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.121+0000 7f9353c38200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T14:53:10.444 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.207+0000 7f9353c38200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T14:53:10.616 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-02-14T14:53:10.616 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-14T14:53:10.616 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T14:53:10.616 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-14T14:53:10.616 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.452+0000 7facd71c9700 1 Processor -- start 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.452+0000 7facd71c9700 1 -- start start 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.452+0000 7facd71c9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd010a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.452+0000 7facd71c9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd007ae70 con 0x7facd0108120 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7facd4f65700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd010a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7facd4f65700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd010a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34316/0 (socket says 172.21.15.17:34316) 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7facd4f65700 1 -- 172.21.15.17:0/1985014234 learned_addr learned my addr 172.21.15.17:0/1985014234 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7facd4f65700 1 -- 172.21.15.17:0/1985014234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd010aa50 con 0x7facd0108120 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7facd4f65700 1 --2- 172.21.15.17:0/1985014234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd010a510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17c4fc24ac206e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7faccf7fe700 1 -- 172.21.15.17:0/1985014234 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc4004160 con 0x7facd0108120 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7faccf7fe700 1 -- 172.21.15.17:0/1985014234 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facc40042c0 con 0x7facd0108120 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.453+0000 7faccf7fe700 1 -- 172.21.15.17:0/1985014234 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc4011ac0 con 0x7facd0108120 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.454+0000 7facd71c9700 1 -- 172.21.15.17:0/1985014234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 msgr2=0x7facd010a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:10.617 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.454+0000 7facd71c9700 1 --2- 172.21.15.17:0/1985014234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd010a510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc4009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.454+0000 7facd71c9700 1 -- 172.21.15.17:0/1985014234 shutdown_connections 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.454+0000 7facd71c9700 1 --2- 172.21.15.17:0/1985014234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd010a510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.454+0000 7facd71c9700 1 -- 172.21.15.17:0/1985014234 >> 172.21.15.17:0/1985014234 conn(0x7facd0101b90 msgr2=0x7facd0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.454+0000 7facd71c9700 1 -- 172.21.15.17:0/1985014234 shutdown_connections 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.454+0000 7facd71c9700 1 -- 172.21.15.17:0/1985014234 wait complete. 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd71c9700 1 Processor -- start 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd71c9700 1 -- start start 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd71c9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd71c9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd019b9f0 con 0x7facd0108120 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd4f65700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd4f65700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34324/0 (socket says 172.21.15.17:34324) 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd4f65700 1 -- 172.21.15.17:0/3241019365 learned_addr learned my addr 172.21.15.17:0/3241019365 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd4f65700 1 -- 172.21.15.17:0/3241019365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009550 con 0x7facd0108120 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.455+0000 7facd4f65700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd019b4b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7facc4009fb0 tx=0x7facc4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:10.618 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.456+0000 7faccdffb700 1 -- 172.21.15.17:0/3241019365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc40038b0 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.456+0000 7faccdffb700 1 -- 172.21.15.17:0/3241019365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facc4003a10 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.456+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd019bbf0 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.456+0000 7faccdffb700 1 -- 172.21.15.17:0/3241019365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc40195e0 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.456+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd019c090 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.457+0000 7faccdffb700 1 -- 172.21.15.17:0/3241019365 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7facc401a9f0 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.457+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd010b850 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.457+0000 7faccdffb700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 0x7facc003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.457+0000 7faccdffb700 1 -- 172.21.15.17:0/3241019365 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7facc404d020 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.457+0000 7faccffff700 1 -- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 msgr2=0x7facc003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2869822133 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.457+0000 7faccffff700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 0x7facc003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.462+0000 7faccdffb700 1 -- 172.21.15.17:0/3241019365 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7facc4080d20 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.658+0000 7faccffff700 1 -- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 msgr2=0x7facc003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2869822133 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.658+0000 7faccffff700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 0x7facc003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.725+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7facd019e9d0 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.726+0000 7faccdffb700 1 -- 172.21.15.17:0/3241019365 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7facc4018350 con 0x7facd0108120 2024-02-14T14:53:10.619 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 msgr2=0x7facc003b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 0x7facc003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 msgr2=0x7facd019b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd019b4b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7facc4009fb0 tx=0x7facc4011b20 comp rx=0 tx=0).stop 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 shutdown_connections 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7facc0038ee0 0x7facc003b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 --2- 172.21.15.17:0/3241019365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd0108120 0x7facd019b4b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 >> 172.21.15.17:0/3241019365 conn(0x7facd0101b90 msgr2=0x7facd0102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 shutdown_connections 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:09.728+0000 7facd71c9700 1 -- 172.21.15.17:0/3241019365 wait complete. 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for the mgr to restart... 2024-02-14T14:53:10.620 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr epoch 5... 2024-02-14T14:53:10.740 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3241019365' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T14:53:10.740 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.476+0000 7f9353c38200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T14:53:10.740 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.588+0000 7f9353c38200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T14:53:10.740 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.686+0000 7f9353c38200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T14:53:11.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.784+0000 7f9353c38200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T14:53:11.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:10.965+0000 7f9353c38200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T14:53:12.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:11 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:11.953+0000 7f9353c38200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T14:53:12.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:12.087+0000 7f9353c38200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T14:53:12.719 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:12.425+0000 7f9353c38200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T14:53:12.719 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:12.517+0000 7f9353c38200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T14:53:12.719 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:12.716+0000 7f9353c38200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:53:13.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:12.904+0000 7f9353c38200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T14:53:13.457 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:13.193+0000 7f9353c38200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:53:13.458 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:13.455+0000 7f9353c38200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T14:53:13.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:13.697+0000 7f9353c38200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T14:53:15.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:14 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:14.544+0000 7f9353c38200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T14:53:15.452 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:15.127+0000 7f9353c38200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T14:53:15.452 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:15.229+0000 7f9353c38200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T14:53:15.452 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:15.451+0000 7f9353c38200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T14:53:16.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: Active manager daemon x restarted 2024-02-14T14:53:16.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: Activating manager daemon x 2024-02-14T14:53:16.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: osdmap e2: 0 total, 0 up, 0 in 2024-02-14T14:53:16.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: mgrmap e6: x(active, starting, since 0.00332504s) 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: Manager daemon x is now available 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-14T14:53:16.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-14T14:53:16.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:16.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:16 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:16.963 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-02-14T14:53:16.963 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-14T14:53:16.963 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.017+0000 7fbc17348700 1 Processor -- start 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.017+0000 7fbc17348700 1 -- start start 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.017+0000 7fbc17348700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc10105d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc17348700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10106340 con 0x7fbc10105950 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc150e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc10105d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc150e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc10105d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57672/0 (socket says 172.21.15.17:57672) 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc150e4700 1 -- 172.21.15.17:0/195023233 learned_addr learned my addr 172.21.15.17:0/195023233 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc150e4700 1 -- 172.21.15.17:0/195023233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc101088e0 con 0x7fbc10105950 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc150e4700 1 --2- 172.21.15.17:0/195023233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc10105d70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc040098a0 tx=0x7fbc04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873f11e05e632105 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc03fff700 1 -- 172.21.15.17:0/195023233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc04004160 con 0x7fbc10105950 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc03fff700 1 -- 172.21.15.17:0/195023233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc040042c0 con 0x7fbc10105950 2024-02-14T14:53:16.964 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.018+0000 7fbc03fff700 1 -- 172.21.15.17:0/195023233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc04011ac0 con 0x7fbc10105950 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.019+0000 7fbc17348700 1 -- 172.21.15.17:0/195023233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 msgr2=0x7fbc10105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.019+0000 7fbc17348700 1 --2- 172.21.15.17:0/195023233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc10105d70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc040098a0 tx=0x7fbc04009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.019+0000 7fbc17348700 1 -- 172.21.15.17:0/195023233 shutdown_connections 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.019+0000 7fbc17348700 1 --2- 172.21.15.17:0/195023233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc10105d70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.019+0000 7fbc17348700 1 -- 172.21.15.17:0/195023233 >> 172.21.15.17:0/195023233 conn(0x7fbc10101320 msgr2=0x7fbc10103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.019+0000 7fbc17348700 1 -- 172.21.15.17:0/195023233 shutdown_connections 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.019+0000 7fbc17348700 1 -- 172.21.15.17:0/195023233 wait complete. 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc17348700 1 Processor -- start 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc17348700 1 -- start start 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc17348700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc1019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc17348700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1019b9d0 con 0x7fbc10105950 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc150e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc1019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc150e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc1019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57678/0 (socket says 172.21.15.17:57678) 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc150e4700 1 -- 172.21.15.17:0/489642446 learned_addr learned my addr 172.21.15.17:0/489642446 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:16.965 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.020+0000 7fbc150e4700 1 -- 172.21.15.17:0/489642446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04009550 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.021+0000 7fbc150e4700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc1019b490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc04011780 tx=0x7fbc04011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.021+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc04003af0 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.021+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc04003c50 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.021+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1019bbd0 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.021+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc040197e0 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.021+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1019c070 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.022+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fbc04019a20 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.022+0000 7fbc027fc700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.022+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 --> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbfc03bab0 con 0x7fbbfc038ee0 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.022+0000 7fbc148e3700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 msgr2=0x7fbbfc03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2869822133 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.022+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbc0404d420 con 0x7fbc10105950 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.022+0000 7fbc148e3700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.223+0000 7fbc148e3700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 msgr2=0x7fbbfc03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2869822133 2024-02-14T14:53:16.966 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.223+0000 7fbc148e3700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.624+0000 7fbc148e3700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 msgr2=0x7fbbfc03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2869822133 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:11.624+0000 7fbc148e3700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:12.425+0000 7fbc148e3700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 msgr2=0x7fbbfc03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2869822133 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:12.425+0000 7fbc148e3700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:14.026+0000 7fbc148e3700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 msgr2=0x7fbbfc03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2869822133 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:14.026+0000 7fbc148e3700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:15.465+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7fbc04019d20 con 0x7fbc10105950 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:15.466+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 msgr2=0x7fbbfc03b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:15.466+0000 7fbc027fc700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.467+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fbc0404edc0 con 0x7fbc10105950 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.467+0000 7fbc027fc700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fbbfc03c7c0 0x7fbbfc03ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.467+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbfc03bab0 con 0x7fbbfc03c7c0 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.467+0000 7fbc148e3700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fbbfc03c7c0 0x7fbbfc03ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.468+0000 7fbc148e3700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fbbfc03c7c0 0x7fbbfc03ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c003a10 tx=0x7fbc0c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:16.967 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.469+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fbbfc03bab0 con 0x7fbbfc03c7c0 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.475+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbc1010baf0 con 0x7fbbfc03c7c0 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.475+0000 7fbc027fc700 1 -- 172.21.15.17:0/489642446 <== mgr.14118 v2:172.21.15.17:6800/1044675107 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbc1010baf0 con 0x7fbbfc03c7c0 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fbbfc03c7c0 msgr2=0x7fbbfc03ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fbbfc03c7c0 0x7fbbfc03ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c003a10 tx=0x7fbc0c009290 comp rx=0 tx=0).stop 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 msgr2=0x7fbc1019b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc1019b490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc04011780 tx=0x7fbc04011b40 comp rx=0 tx=0).stop 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 shutdown_connections 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fbbfc03c7c0 0x7fbbfc03ebb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10105950 0x7fbc1019b490 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 --2- 172.21.15.17:0/489642446 >> [v2:172.21.15.17:6800/2869822133,v1:172.21.15.17:6801/2869822133] conn(0x7fbbfc038ee0 0x7fbbfc03b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 >> 172.21.15.17:0/489642446 conn(0x7fbc10101320 msgr2=0x7fbc10101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 shutdown_connections 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:16.476+0000 7fbc17348700 1 -- 172.21.15.17:0/489642446 wait complete. 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:mgr epoch 5 is available 2024-02-14T14:53:16.968 INFO:teuthology.orchestra.run.smithi017.stdout:Setting orchestrator backend to cephadm... 2024-02-14T14:53:17.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:17 smithi017 ceph-mon[32549]: Found migration_current of "None". Setting to last migration. 2024-02-14T14:53:17.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:17 smithi017 ceph-mon[32549]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-14T14:53:17.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:17 smithi017 ceph-mon[32549]: mgrmap e7: x(active, since 1.00533s) 2024-02-14T14:53:17.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:17 smithi017 ceph-mon[32549]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae810d700 1 Processor -- start 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae810d700 1 -- start start 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae810d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae010a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae810d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae007ae70 con 0x7f4ae0108140 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae5ea9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae010a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae5ea9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae010a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57764/0 (socket says 172.21.15.17:57764) 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae5ea9700 1 -- 172.21.15.17:0/2984002544 learned_addr learned my addr 172.21.15.17:0/2984002544 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.324+0000 7f4ae5ea9700 1 -- 172.21.15.17:0/2984002544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae010aa70 con 0x7f4ae0108140 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.325+0000 7f4ae5ea9700 1 --2- 172.21.15.17:0/2984002544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae010a530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0098a0 tx=0x7f4adc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf1a70d065963e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:18.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.325+0000 7f4ae4ea7700 1 -- 172.21.15.17:0/2984002544 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4adc004160 con 0x7f4ae0108140 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.325+0000 7f4ae4ea7700 1 -- 172.21.15.17:0/2984002544 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4adc0042c0 con 0x7f4ae0108140 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.325+0000 7f4ae4ea7700 1 -- 172.21.15.17:0/2984002544 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4adc011ac0 con 0x7f4ae0108140 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 -- 172.21.15.17:0/2984002544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 msgr2=0x7f4ae010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 --2- 172.21.15.17:0/2984002544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae010a530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0098a0 tx=0x7f4adc009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 -- 172.21.15.17:0/2984002544 shutdown_connections 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 --2- 172.21.15.17:0/2984002544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae010a530 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 -- 172.21.15.17:0/2984002544 >> 172.21.15.17:0/2984002544 conn(0x7f4ae0101b90 msgr2=0x7f4ae0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 -- 172.21.15.17:0/2984002544 shutdown_connections 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 -- 172.21.15.17:0/2984002544 wait complete. 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 Processor -- start 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 -- start start 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:18.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.326+0000 7f4ae810d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae019dc20 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ae5ea9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ae5ea9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57780/0 (socket says 172.21.15.17:57780) 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ae5ea9700 1 -- 172.21.15.17:0/335623527 learned_addr learned my addr 172.21.15.17:0/335623527 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ae5ea9700 1 -- 172.21.15.17:0/335623527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc009550 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ae5ea9700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae019d6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011a70 tx=0x7f4adc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ad6ffd700 1 -- 172.21.15.17:0/335623527 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4adc0039c0 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ad6ffd700 1 -- 172.21.15.17:0/335623527 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4adc003b20 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae019de20 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae019e2c0 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.327+0000 7f4ad6ffd700 1 -- 172.21.15.17:0/335623527 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4adc024610 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.328+0000 7f4ad6ffd700 1 -- 172.21.15.17:0/335623527 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f4adc02d3f0 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.328+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac4005360 con 0x7f4ae0108140 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.328+0000 7f4ad6ffd700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4acc038e10 0x7f4acc03b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:18.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.328+0000 7f4ad6ffd700 1 -- 172.21.15.17:0/335623527 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4adc04cef0 con 0x7f4ae0108140 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.328+0000 7f4ae56a8700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4acc038e10 0x7f4acc03b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.329+0000 7f4ae56a8700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4acc038e10 0x7f4acc03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0006fd0 tx=0x7f4ad0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.333+0000 7f4ad6ffd700 1 -- 172.21.15.17:0/335623527 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4adc010740 con 0x7f4ae0108140 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.556+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ac4000bf0 con 0x7f4acc038e10 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.560+0000 7f4ad6ffd700 1 -- 172.21.15.17:0/335623527 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4ac4000bf0 con 0x7f4acc038e10 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4acc038e10 msgr2=0x7f4acc03b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4acc038e10 0x7f4acc03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0006fd0 tx=0x7f4ad0006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 msgr2=0x7f4ae019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae019d6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011a70 tx=0x7f4adc011b50 comp rx=0 tx=0).stop 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 shutdown_connections 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4acc038e10 0x7f4acc03b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 --2- 172.21.15.17:0/335623527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ae0108140 0x7f4ae019d6e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 >> 172.21.15.17:0/335623527 conn(0x7f4ae0101b90 msgr2=0x7f4ae01026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:18.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 shutdown_connections 2024-02-14T14:53:18.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:17.562+0000 7f4ae810d700 1 -- 172.21.15.17:0/335623527 wait complete. 2024-02-14T14:53:18.722 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:18 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:17] ENGINE Bus STARTING 2024-02-14T14:53:18.723 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:18 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:17] ENGINE Serving on https://172.21.15.17:7150 2024-02-14T14:53:18.723 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:18 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:18.723 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:18 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.508+0000 7f178998a700 1 Processor -- start 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.508+0000 7f178998a700 1 -- start start 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.508+0000 7f178998a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f17841069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.508+0000 7f178998a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1784106fb0 con 0x7f17841065c0 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.508+0000 7f1782ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f17841069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.508+0000 7f1782ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f17841069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57782/0 (socket says 172.21.15.17:57782) 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.508+0000 7f1782ffd700 1 -- 172.21.15.17:0/2351204837 learned_addr learned my addr 172.21.15.17:0/2351204837 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.509+0000 7f1782ffd700 1 -- 172.21.15.17:0/2351204837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17841077c0 con 0x7f17841065c0 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.509+0000 7f1782ffd700 1 --2- 172.21.15.17:0/2351204837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f17841069e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f177c009e80 tx=0x7f177c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5b41021aa56ff0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.509+0000 7f1781ffb700 1 -- 172.21.15.17:0/2351204837 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f177c00e070 con 0x7f17841065c0 2024-02-14T14:53:19.305 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.509+0000 7f1781ffb700 1 -- 172.21.15.17:0/2351204837 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f177c00b8c0 con 0x7f17841065c0 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.509+0000 7f1781ffb700 1 -- 172.21.15.17:0/2351204837 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f177c004550 con 0x7f17841065c0 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 -- 172.21.15.17:0/2351204837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 msgr2=0x7f17841069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 --2- 172.21.15.17:0/2351204837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f17841069e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f177c009e80 tx=0x7f177c00b330 comp rx=0 tx=0).stop 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 -- 172.21.15.17:0/2351204837 shutdown_connections 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 --2- 172.21.15.17:0/2351204837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f17841069e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 -- 172.21.15.17:0/2351204837 >> 172.21.15.17:0/2351204837 conn(0x7f1784101c60 msgr2=0x7f1784104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 -- 172.21.15.17:0/2351204837 shutdown_connections 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 -- 172.21.15.17:0/2351204837 wait complete. 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.510+0000 7f178998a700 1 Processor -- start 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f178998a700 1 -- start start 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f178998a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f178407aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f178998a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178407af90 con 0x7f17841065c0 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f1782ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f178407aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f1782ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f178407aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57792/0 (socket says 172.21.15.17:57792) 2024-02-14T14:53:19.306 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f1782ffd700 1 -- 172.21.15.17:0/2402321947 learned_addr learned my addr 172.21.15.17:0/2402321947 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f1782ffd700 1 -- 172.21.15.17:0/2402321947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c009b30 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f1782ffd700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f178407aa50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fd0 tx=0x7f177c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.511+0000 7f1788988700 1 -- 172.21.15.17:0/2402321947 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f177c00e040 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.512+0000 7f1788988700 1 -- 172.21.15.17:0/2402321947 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f177c0128f0 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.512+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178419e130 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.512+0000 7f1788988700 1 -- 172.21.15.17:0/2402321947 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f177c01c770 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.512+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178419e5b0 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.512+0000 7f1788988700 1 -- 172.21.15.17:0/2402321947 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f177c01c9e0 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.513+0000 7f1788988700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1770038f30 0x7f177003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.513+0000 7f1788988700 1 -- 172.21.15.17:0/2402321947 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f177c04cf00 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.513+0000 7f17827fc700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1770038f30 0x7f177003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.513+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1784068df0 con 0x7f17841065c0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.513+0000 7f17827fc700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1770038f30 0x7f177003b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f176c006fd0 tx=0x7f176c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:19.307 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.518+0000 7f1788988700 1 -- 172.21.15.17:0/2402321947 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f177c017020 con 0x7f17841065c0 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.722+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1784102dd0 con 0x7f1770038f30 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.722+0000 7f1788988700 1 -- 172.21.15.17:0/2402321947 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f1784102dd0 con 0x7f1770038f30 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1770038f30 msgr2=0x7f177003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1770038f30 0x7f177003b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f176c006fd0 tx=0x7f176c006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 msgr2=0x7f178407aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f178407aa50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fd0 tx=0x7f177c003d00 comp rx=0 tx=0).stop 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 shutdown_connections 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1770038f30 0x7f177003b3f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 --2- 172.21.15.17:0/2402321947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17841065c0 0x7f178407aa50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 >> 172.21.15.17:0/2402321947 conn(0x7f1784101c60 msgr2=0x7f17841026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 shutdown_connections 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:18.724+0000 7f178998a700 1 -- 172.21.15.17:0/2402321947 wait complete. 2024-02-14T14:53:19.308 INFO:teuthology.orchestra.run.smithi017.stdout:Generating ssh key... 2024-02-14T14:53:19.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:19 smithi017 ceph-mon[32549]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:19.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:19 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:17] ENGINE Serving on http://172.21.15.17:8765 2024-02-14T14:53:19.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:19 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:17] ENGINE Bus STARTED 2024-02-14T14:53:19.611 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:19 smithi017 ceph-mon[32549]: mgrmap e8: x(active, since 3s) 2024-02-14T14:53:20.671 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-mon[32549]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:20.672 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:20.672 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:20.672 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: Generating public/private rsa key pair. 2024-02-14T14:53:20.672 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: Your identification has been saved in /tmp/tmpi9yxa420/key. 2024-02-14T14:53:20.672 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: Your public key has been saved in /tmp/tmpi9yxa420/key.pub. 2024-02-14T14:53:20.672 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: The key fingerprint is: 2024-02-14T14:53:20.672 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: SHA256:Xxpn51MMdHapbPhkxh0YgFsedU3O/VJkTI9CB/OYQKI ceph-867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: The key's randomart image is: 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: +---[RSA 3072]----+ 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | ..+.*+==@| 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | . o = O.@*| 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | E + O * B| 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | . o X =.| 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | S . X o +| 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | . * + o | 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | o o | 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | . | 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: | | 2024-02-14T14:53:20.673 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: +----[SHA256]-----+ 2024-02-14T14:53:20.949 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.716+0000 7f333f7e7700 1 Processor -- start 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.716+0000 7f333f7e7700 1 -- start start 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.716+0000 7f333f7e7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f33381063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.716+0000 7f333f7e7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333807ae10 con 0x7f3338105f90 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f333d583700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f33381063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f333d583700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f33381063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57796/0 (socket says 172.21.15.17:57796) 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f333d583700 1 -- 172.21.15.17:0/2774310916 learned_addr learned my addr 172.21.15.17:0/2774310916 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f333d583700 1 -- 172.21.15.17:0/2774310916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333807af50 con 0x7f3338105f90 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f333d583700 1 --2- 172.21.15.17:0/2774310916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f33381063b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3330009e80 tx=0x7f333000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b30a5e4210bc66 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:20.950 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f3337fff700 1 -- 172.21.15.17:0/2774310916 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f333000e070 con 0x7f3338105f90 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f3337fff700 1 -- 172.21.15.17:0/2774310916 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f333000b8c0 con 0x7f3338105f90 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.717+0000 7f3337fff700 1 -- 172.21.15.17:0/2774310916 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3330004550 con 0x7f3338105f90 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.718+0000 7f333f7e7700 1 -- 172.21.15.17:0/2774310916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 msgr2=0x7f33381063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.718+0000 7f333f7e7700 1 --2- 172.21.15.17:0/2774310916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f33381063b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3330009e80 tx=0x7f333000b330 comp rx=0 tx=0).stop 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.718+0000 7f333f7e7700 1 -- 172.21.15.17:0/2774310916 shutdown_connections 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.718+0000 7f333f7e7700 1 --2- 172.21.15.17:0/2774310916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f33381063b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.718+0000 7f333f7e7700 1 -- 172.21.15.17:0/2774310916 >> 172.21.15.17:0/2774310916 conn(0x7f3338101c10 msgr2=0x7f3338104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.718+0000 7f333f7e7700 1 -- 172.21.15.17:0/2774310916 shutdown_connections 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.718+0000 7f333f7e7700 1 -- 172.21.15.17:0/2774310916 wait complete. 2024-02-14T14:53:20.951 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333f7e7700 1 Processor -- start 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333f7e7700 1 -- start start 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333f7e7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f333819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333f7e7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333819dae0 con 0x7f3338105f90 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333d583700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f333819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333d583700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f333819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57804/0 (socket says 172.21.15.17:57804) 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333d583700 1 -- 172.21.15.17:0/2528891263 learned_addr learned my addr 172.21.15.17:0/2528891263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333d583700 1 -- 172.21.15.17:0/2528891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330009b30 con 0x7f3338105f90 2024-02-14T14:53:20.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f333d583700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f333819d5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3330009fd0 tx=0x7f3330003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f33367fc700 1 -- 172.21.15.17:0/2528891263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f333000e040 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f33367fc700 1 -- 172.21.15.17:0/2528891263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33300128f0 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.719+0000 7f33367fc700 1 -- 172.21.15.17:0/2528891263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f333001c770 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.720+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333819dce0 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.720+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333819e160 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.720+0000 7f33367fc700 1 -- 172.21.15.17:0/2528891263 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f3330012a60 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.720+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3338197270 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.721+0000 7f33367fc700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3320038f30 0x7f332003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.721+0000 7f33367fc700 1 -- 172.21.15.17:0/2528891263 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f333004cf60 con 0x7f3338105f90 2024-02-14T14:53:20.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.721+0000 7f333cd82700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3320038f30 0x7f332003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.721+0000 7f333cd82700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3320038f30 0x7f332003b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f332c006fd0 tx=0x7f332c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.726+0000 7f33367fc700 1 -- 172.21.15.17:0/2528891263 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3330024da0 con 0x7f3338105f90 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:19.928+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f333802d9f0 con 0x7f3320038f30 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.323+0000 7f33367fc700 1 -- 172.21.15.17:0/2528891263 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f333802d9f0 con 0x7f3320038f30 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3320038f30 msgr2=0x7f332003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3320038f30 0x7f332003b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f332c006fd0 tx=0x7f332c006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 msgr2=0x7f333819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f333819d5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3330009fd0 tx=0x7f3330003d00 comp rx=0 tx=0).stop 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 shutdown_connections 2024-02-14T14:53:20.954 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3320038f30 0x7f332003b3f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:20.955 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 --2- 172.21.15.17:0/2528891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3338105f90 0x7f333819d5a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:20.955 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.325+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 >> 172.21.15.17:0/2528891263 conn(0x7f3338101c10 msgr2=0x7f33381943b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:20.955 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.326+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 shutdown_connections 2024-02-14T14:53:20.955 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:20.326+0000 7f333f7e7700 1 -- 172.21.15.17:0/2528891263 wait complete. 2024-02-14T14:53:21.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:21 smithi017 ceph-mon[32549]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:21.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:21 smithi017 ceph-mon[32549]: Generating ssh key... 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoIoh4vAnlcLiBiiCWlGRa9E9wGk51HmfAoHvFQjZjg8XNwd6X+J9mllrxoNwI5Raomo6ya6hK1aWJ1dsJM321QiKrEFU0cQg8u2k1d3QKlWRteb6kzX2ASm7Ew9rGygno1lO9mfjzqYfWChlYHt6PjKhMAK8mALhaQQdJ0S6lkmnVpkABKIu3imMz50gzugCPtMaBDAEzgz5GFawJC2W8aGKMbNMGOx7CINVMmDfh3ONqg6Ry+nRNnWkt77wtSfMljNyMvP7uZCH71mOFROMIm9ZX/0HPmxtO1UdeXkoHeoljXqqwAPztdgeJnETZzyp5ewUGrHU+udtPTOT3VgteHzHWDFeO74eaBQW1u+2tArqctbsFwD8LhhedTJhL3F30mgwS1sCo1n+JMGjvX0Dhs7GMyq4N3VwuuX/VsaDHsQLn2NvXCz125YKvmidtYNhE0FLQPW5aMEuHW8YAFT4UiBDEVx2hlhrWyBM/lBWjjNh2DtSAwOwmbARjNyU+z78= ceph-867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.392+0000 7fd88ec36700 1 Processor -- start 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.392+0000 7fd88ec36700 1 -- start start 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.392+0000 7fd88ec36700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88807b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.392+0000 7fd88ec36700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88807b8d0 con 0x7fd88807ce50 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.392+0000 7fd88c9d2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88807b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.392+0000 7fd88c9d2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88807b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60162/0 (socket says 172.21.15.17:60162) 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.392+0000 7fd88c9d2700 1 -- 172.21.15.17:0/4029383730 learned_addr learned my addr 172.21.15.17:0/4029383730 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.393+0000 7fd88c9d2700 1 -- 172.21.15.17:0/4029383730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88807ba10 con 0x7fd88807ce50 2024-02-14T14:53:22.154 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.393+0000 7fd88c9d2700 1 --2- 172.21.15.17:0/4029383730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88807b300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098a0 tx=0x7fd87c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e1475ee8a807ae server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.393+0000 7fd8877fe700 1 -- 172.21.15.17:0/4029383730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd87c004160 con 0x7fd88807ce50 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.393+0000 7fd8877fe700 1 -- 172.21.15.17:0/4029383730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd87c0042c0 con 0x7fd88807ce50 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.393+0000 7fd8877fe700 1 -- 172.21.15.17:0/4029383730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd87c011ac0 con 0x7fd88807ce50 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 -- 172.21.15.17:0/4029383730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 msgr2=0x7fd88807b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 --2- 172.21.15.17:0/4029383730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88807b300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098a0 tx=0x7fd87c009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 -- 172.21.15.17:0/4029383730 shutdown_connections 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 --2- 172.21.15.17:0/4029383730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88807b300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 -- 172.21.15.17:0/4029383730 >> 172.21.15.17:0/4029383730 conn(0x7fd888102d50 msgr2=0x7fd888105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 -- 172.21.15.17:0/4029383730 shutdown_connections 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 -- 172.21.15.17:0/4029383730 wait complete. 2024-02-14T14:53:22.155 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 Processor -- start 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 -- start start 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.394+0000 7fd88ec36700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88819dbb0 con 0x7fd88807ce50 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd88c9d2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd88c9d2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60164/0 (socket says 172.21.15.17:60164) 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd88c9d2700 1 -- 172.21.15.17:0/2619919190 learned_addr learned my addr 172.21.15.17:0/2619919190 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd88c9d2700 1 -- 172.21.15.17:0/2619919190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c009550 con 0x7fd88807ce50 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd88c9d2700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88819d670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd87c011a70 tx=0x7fd87c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd885ffb700 1 -- 172.21.15.17:0/2619919190 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd87c0039c0 con 0x7fd88807ce50 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd885ffb700 1 -- 172.21.15.17:0/2619919190 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd87c003b20 con 0x7fd88807ce50 2024-02-14T14:53:22.156 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88819ddb0 con 0x7fd88807ce50 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd885ffb700 1 -- 172.21.15.17:0/2619919190 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd87c024610 con 0x7fd88807ce50 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.395+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88819e250 con 0x7fd88807ce50 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.396+0000 7fd885ffb700 1 -- 172.21.15.17:0/2619919190 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fd87c02d3f0 con 0x7fd88807ce50 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.396+0000 7fd885ffb700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fd878038ee0 0x7fd87803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.396+0000 7fd885ffb700 1 -- 172.21.15.17:0/2619919190 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd87c04cf40 con 0x7fd88807ce50 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.396+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88810dac0 con 0x7fd88807ce50 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.397+0000 7fd887fff700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fd878038ee0 0x7fd87803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.397+0000 7fd887fff700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fd878038ee0 0x7fd87803b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.402+0000 7fd885ffb700 1 -- 172.21.15.17:0/2619919190 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd87c010970 con 0x7fd88807ce50 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.608+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd888103fb0 con 0x7fd878038ee0 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.608+0000 7fd885ffb700 1 -- 172.21.15.17:0/2619919190 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd888103fb0 con 0x7fd878038ee0 2024-02-14T14:53:22.157 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fd878038ee0 msgr2=0x7fd87803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fd878038ee0 0x7fd87803b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 msgr2=0x7fd88819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88819d670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd87c011a70 tx=0x7fd87c011b50 comp rx=0 tx=0).stop 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 shutdown_connections 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fd878038ee0 0x7fd87803b3a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 --2- 172.21.15.17:0/2619919190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd88807ce50 0x7fd88819d670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 >> 172.21.15.17:0/2619919190 conn(0x7fd888102d50 msgr2=0x7fd8881038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 shutdown_connections 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:21.611+0000 7fd88ec36700 1 -- 172.21.15.17:0/2619919190 wait complete. 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:Adding key to root@localhost authorized_keys... 2024-02-14T14:53:22.158 INFO:teuthology.orchestra.run.smithi017.stdout:Adding host smithi017... 2024-02-14T14:53:22.810 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:22 smithi017 ceph-mon[32549]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:23.588 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:23 smithi017 ceph-mon[32549]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi017", "addr": "172.21.15.17", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:23.588 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:23 smithi017 ceph-mon[32549]: Deploying cephadm binary to smithi017 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Added host 'smithi017' with addr '172.21.15.17' 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.517+0000 7f3c5d9c5700 1 Processor -- start 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c5d9c5700 1 -- start start 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c5d9c5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c58105d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c5d9c5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58106340 con 0x7f3c58105950 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c56ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c58105d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c56ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c58105d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60170/0 (socket says 172.21.15.17:60170) 2024-02-14T14:53:27.677 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c56ffd700 1 -- 172.21.15.17:0/1719225939 learned_addr learned my addr 172.21.15.17:0/1719225939 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c56ffd700 1 -- 172.21.15.17:0/1719225939 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c581088e0 con 0x7f3c58105950 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c56ffd700 1 --2- 172.21.15.17:0/1719225939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c58105d70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c480098a0 tx=0x7f3c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630df568469ed5e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c55ffb700 1 -- 172.21.15.17:0/1719225939 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c48004160 con 0x7f3c58105950 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c55ffb700 1 -- 172.21.15.17:0/1719225939 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c480042c0 con 0x7f3c58105950 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.518+0000 7f3c55ffb700 1 -- 172.21.15.17:0/1719225939 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c48011ac0 con 0x7f3c58105950 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.519+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1719225939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 msgr2=0x7f3c58105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.519+0000 7f3c5d9c5700 1 --2- 172.21.15.17:0/1719225939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c58105d70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c480098a0 tx=0x7f3c48009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.519+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1719225939 shutdown_connections 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.519+0000 7f3c5d9c5700 1 --2- 172.21.15.17:0/1719225939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c58105d70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.519+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1719225939 >> 172.21.15.17:0/1719225939 conn(0x7f3c58101320 msgr2=0x7f3c58103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.519+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1719225939 shutdown_connections 2024-02-14T14:53:27.678 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.519+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1719225939 wait complete. 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c5d9c5700 1 Processor -- start 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c5d9c5700 1 -- start start 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c5d9c5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c5819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c5d9c5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5819b9a0 con 0x7f3c58105950 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c56ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c5819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c56ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c5819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60184/0 (socket says 172.21.15.17:60184) 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c56ffd700 1 -- 172.21.15.17:0/1743260951 learned_addr learned my addr 172.21.15.17:0/1743260951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.520+0000 7f3c56ffd700 1 -- 172.21.15.17:0/1743260951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48009550 con 0x7f3c58105950 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.521+0000 7f3c56ffd700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c5819b460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c48000c00 tx=0x7f3c48011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.521+0000 7f3c5c9c3700 1 -- 172.21.15.17:0/1743260951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c48003aa0 con 0x7f3c58105950 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.521+0000 7f3c5c9c3700 1 -- 172.21.15.17:0/1743260951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c48003c00 con 0x7f3c58105950 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.521+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5819bba0 con 0x7f3c58105950 2024-02-14T14:53:27.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.521+0000 7f3c5c9c3700 1 -- 172.21.15.17:0/1743260951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c480248a0 con 0x7f3c58105950 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.521+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5819c040 con 0x7f3c58105950 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.522+0000 7f3c5c9c3700 1 -- 172.21.15.17:0/1743260951 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f3c4802d3f0 con 0x7f3c58105950 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.522+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5810bb10 con 0x7f3c58105950 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.522+0000 7f3c5c9c3700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3c40038ee0 0x7f3c4003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.522+0000 7f3c5c9c3700 1 -- 172.21.15.17:0/1743260951 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3c4804d1d0 con 0x7f3c58105950 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.522+0000 7f3c567fc700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3c40038ee0 0x7f3c4003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.523+0000 7f3c567fc700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3c40038ee0 0x7f3c4003b3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006fd0 tx=0x7f3c4c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.527+0000 7f3c5c9c3700 1 -- 172.21.15.17:0/1743260951 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c48018ba0 con 0x7f3c58105950 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:22.733+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi017", "addr": "172.21.15.17", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c581025a0 con 0x7f3c40038ee0 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.081+0000 7f3c5c9c3700 1 -- 172.21.15.17:0/1743260951 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f3c581025a0 con 0x7f3c40038ee0 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3c40038ee0 msgr2=0x7f3c4003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3c40038ee0 0x7f3c4003b3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006fd0 tx=0x7f3c4c006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 msgr2=0x7f3c5819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c5819b460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c48000c00 tx=0x7f3c48011b40 comp rx=0 tx=0).stop 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 shutdown_connections 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f3c40038ee0 0x7f3c4003b3a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 --2- 172.21.15.17:0/1743260951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58105950 0x7f3c5819b460 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 >> 172.21.15.17:0/1743260951 conn(0x7f3c58101320 msgr2=0x7f3c58101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 shutdown_connections 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:27.084+0000 7f3c5d9c5700 1 -- 172.21.15.17:0/1743260951 wait complete. 2024-02-14T14:53:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:Deploying unmanaged mon service... 2024-02-14T14:53:28.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:28 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:28.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:28 smithi017 ceph-mon[32549]: Added host smithi017 2024-02-14T14:53:28.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:28 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:29.046 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-14T14:53:29.046 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6609a4c700 1 Processor -- start 2024-02-14T14:53:29.046 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6609a4c700 1 -- start start 2024-02-14T14:53:29.046 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6609a4c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f6604106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6609a4c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041074c0 con 0x7f6604106ad0 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6602ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f6604106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6602ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f6604106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60200/0 (socket says 172.21.15.17:60200) 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6602ffd700 1 -- 172.21.15.17:0/1271367677 learned_addr learned my addr 172.21.15.17:0/1271367677 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6602ffd700 1 -- 172.21.15.17:0/1271367677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66041077f0 con 0x7f6604106ad0 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.049+0000 7f6602ffd700 1 --2- 172.21.15.17:0/1271367677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f6604106ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65f40098a0 tx=0x7f65f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498073897a42143d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6601ffb700 1 -- 172.21.15.17:0/1271367677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f4004160 con 0x7f6604106ad0 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6601ffb700 1 -- 172.21.15.17:0/1271367677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65f40042c0 con 0x7f6604106ad0 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6601ffb700 1 -- 172.21.15.17:0/1271367677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f4011ac0 con 0x7f6604106ad0 2024-02-14T14:53:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6609a4c700 1 -- 172.21.15.17:0/1271367677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 msgr2=0x7f6604106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6609a4c700 1 --2- 172.21.15.17:0/1271367677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f6604106ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65f40098a0 tx=0x7f65f4009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6609a4c700 1 -- 172.21.15.17:0/1271367677 shutdown_connections 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6609a4c700 1 --2- 172.21.15.17:0/1271367677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f6604106ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6609a4c700 1 -- 172.21.15.17:0/1271367677 >> 172.21.15.17:0/1271367677 conn(0x7f6604101370 msgr2=0x7f6604103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6609a4c700 1 -- 172.21.15.17:0/1271367677 shutdown_connections 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.050+0000 7f6609a4c700 1 -- 172.21.15.17:0/1271367677 wait complete. 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.051+0000 7f6609a4c700 1 Processor -- start 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.051+0000 7f6609a4c700 1 -- start start 2024-02-14T14:53:29.048 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.051+0000 7f6609a4c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f66041060b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.051+0000 7f6609a4c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041065f0 con 0x7f6604106ad0 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.051+0000 7f6602ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f66041060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.051+0000 7f6602ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f66041060b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60216/0 (socket says 172.21.15.17:60216) 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.051+0000 7f6602ffd700 1 -- 172.21.15.17:0/22940278 learned_addr learned my addr 172.21.15.17:0/22940278 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.052+0000 7f6602ffd700 1 -- 172.21.15.17:0/22940278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f4009550 con 0x7f6604106ad0 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.052+0000 7f6602ffd700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f66041060b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65f4011a70 tx=0x7f65f4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.052+0000 7f6608a4a700 1 -- 172.21.15.17:0/22940278 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f40039c0 con 0x7f6604106ad0 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.052+0000 7f6608a4a700 1 -- 172.21.15.17:0/22940278 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65f4003b20 con 0x7f6604106ad0 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.052+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660419c030 con 0x7f6604106ad0 2024-02-14T14:53:29.049 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.052+0000 7f6608a4a700 1 -- 172.21.15.17:0/22940278 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f4024610 con 0x7f6604106ad0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.052+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66041a08f0 con 0x7f6604106ad0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.053+0000 7f6608a4a700 1 -- 172.21.15.17:0/22940278 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f65f402d3f0 con 0x7f6604106ad0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.053+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660410aa50 con 0x7f6604106ad0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.053+0000 7f6608a4a700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f65ec038f30 0x7f65ec03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.053+0000 7f6608a4a700 1 -- 172.21.15.17:0/22940278 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f65f404d090 con 0x7f6604106ad0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.053+0000 7f66027fc700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f65ec038f30 0x7f65ec03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.054+0000 7f66027fc700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f65ec038f30 0x7f65ec03b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f65f8006fd0 tx=0x7f65f8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.058+0000 7f6608a4a700 1 -- 172.21.15.17:0/22940278 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65f4010740 con 0x7f6604106ad0 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.290+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6604102650 con 0x7f65ec038f30 2024-02-14T14:53:29.050 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.294+0000 7f6608a4a700 1 -- 172.21.15.17:0/22940278 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6604102650 con 0x7f65ec038f30 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f65ec038f30 msgr2=0x7f65ec03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f65ec038f30 0x7f65ec03b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f65f8006fd0 tx=0x7f65f8006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 msgr2=0x7f66041060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f66041060b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65f4011a70 tx=0x7f65f4011b50 comp rx=0 tx=0).stop 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 shutdown_connections 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f65ec038f30 0x7f65ec03b3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 --2- 172.21.15.17:0/22940278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6604106ad0 0x7f66041060b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 >> 172.21.15.17:0/22940278 conn(0x7f6604101370 msgr2=0x7f6604101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 shutdown_connections 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:28.297+0000 7f6609a4c700 1 -- 172.21.15.17:0/22940278 wait complete. 2024-02-14T14:53:29.051 INFO:teuthology.orchestra.run.smithi017.stdout:Deploying unmanaged mgr service... 2024-02-14T14:53:29.582 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:29 smithi017 ceph-mon[32549]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:29.582 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:29 smithi017 ceph-mon[32549]: Saving service mon spec with placement count:5 2024-02-14T14:53:29.582 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:29 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:30.841 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-14T14:53:30.841 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.438+0000 7fe0aed9d700 1 Processor -- start 2024-02-14T14:53:30.841 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.438+0000 7fe0aed9d700 1 -- start start 2024-02-14T14:53:30.841 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.438+0000 7fe0aed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a81069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:30.841 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.438+0000 7fe0aed9d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8106f80 con 0x7fe0a8106590 2024-02-14T14:53:30.841 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.438+0000 7fe0acb39700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a81069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.438+0000 7fe0acb39700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a81069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60220/0 (socket says 172.21.15.17:60220) 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.438+0000 7fe0acb39700 1 -- 172.21.15.17:0/4113838096 learned_addr learned my addr 172.21.15.17:0/4113838096 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.439+0000 7fe0acb39700 1 -- 172.21.15.17:0/4113838096 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a8107790 con 0x7fe0a8106590 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.439+0000 7fe0acb39700 1 --2- 172.21.15.17:0/4113838096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a81069b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0098a0 tx=0x7fe09c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cdcc90272ee291 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.439+0000 7fe0a77fe700 1 -- 172.21.15.17:0/4113838096 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe09c004160 con 0x7fe0a8106590 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.439+0000 7fe0a77fe700 1 -- 172.21.15.17:0/4113838096 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe09c0042c0 con 0x7fe0a8106590 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.439+0000 7fe0a77fe700 1 -- 172.21.15.17:0/4113838096 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe09c011ac0 con 0x7fe0a8106590 2024-02-14T14:53:30.842 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 -- 172.21.15.17:0/4113838096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 msgr2=0x7fe0a81069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 --2- 172.21.15.17:0/4113838096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a81069b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0098a0 tx=0x7fe09c009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 -- 172.21.15.17:0/4113838096 shutdown_connections 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 --2- 172.21.15.17:0/4113838096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a81069b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 -- 172.21.15.17:0/4113838096 >> 172.21.15.17:0/4113838096 conn(0x7fe0a8101b90 msgr2=0x7fe0a8103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 -- 172.21.15.17:0/4113838096 shutdown_connections 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 -- 172.21.15.17:0/4113838096 wait complete. 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 Processor -- start 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 -- start start 2024-02-14T14:53:30.843 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.440+0000 7fe0aed9d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a819a960 con 0x7fe0a8106590 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0acb39700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0acb39700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60228/0 (socket says 172.21.15.17:60228) 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0acb39700 1 -- 172.21.15.17:0/1545041625 learned_addr learned my addr 172.21.15.17:0/1545041625 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0acb39700 1 -- 172.21.15.17:0/1545041625 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c009550 con 0x7fe0a8106590 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0acb39700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a819a420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009fb0 tx=0x7fe09c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0a5ffb700 1 -- 172.21.15.17:0/1545041625 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe09c0038b0 con 0x7fe0a8106590 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0a5ffb700 1 -- 172.21.15.17:0/1545041625 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe09c003a10 con 0x7fe0a8106590 2024-02-14T14:53:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0a5ffb700 1 -- 172.21.15.17:0/1545041625 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe09c0195e0 con 0x7fe0a8106590 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a819ab60 con 0x7fe0a8106590 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.441+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a819b000 con 0x7fe0a8106590 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.442+0000 7fe0a5ffb700 1 -- 172.21.15.17:0/1545041625 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fe09c01aa10 con 0x7fe0a8106590 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.442+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a810a9c0 con 0x7fe0a8106590 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.442+0000 7fe0a5ffb700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe098038ee0 0x7fe09803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.443+0000 7fe0a5ffb700 1 -- 172.21.15.17:0/1545041625 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe09c04d090 con 0x7fe0a8106590 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.443+0000 7fe0a7fff700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe098038ee0 0x7fe09803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:30.845 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.443+0000 7fe0a7fff700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe098038ee0 0x7fe09803b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe094006fd0 tx=0x7fe094006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.447+0000 7fe0a5ffb700 1 -- 172.21.15.17:0/1545041625 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe09c003b80 con 0x7fe0a8106590 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.666+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe0a8102e30 con 0x7fe098038ee0 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.671+0000 7fe0a5ffb700 1 -- 172.21.15.17:0/1545041625 <== mgr.14118 v2:172.21.15.17:6800/1044675107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe0a8102e30 con 0x7fe098038ee0 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe098038ee0 msgr2=0x7fe09803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe098038ee0 0x7fe09803b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe094006fd0 tx=0x7fe094006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 msgr2=0x7fe0a819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a819a420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009fb0 tx=0x7fe09c011b20 comp rx=0 tx=0).stop 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 shutdown_connections 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe098038ee0 0x7fe09803b3a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:30.846 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 --2- 172.21.15.17:0/1545041625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0a8106590 0x7fe0a819a420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:30.847 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 >> 172.21.15.17:0/1545041625 conn(0x7fe0a8101b90 msgr2=0x7fe0a8102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:30.847 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 shutdown_connections 2024-02-14T14:53:30.847 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:29.674+0000 7fe0aed9d700 1 -- 172.21.15.17:0/1545041625 wait complete. 2024-02-14T14:53:30.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:30 smithi017 ceph-mon[32549]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:30.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:30 smithi017 ceph-mon[32549]: Saving service mgr spec with placement count:2 2024-02-14T14:53:30.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:30 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:30.957 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:30 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:32.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:31 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:32.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:31 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2204467855' entity='client.admin' 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.270+0000 7f499ab31700 1 Processor -- start 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.270+0000 7f499ab31700 1 -- start start 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.270+0000 7f499ab31700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f499410a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.270+0000 7f499ab31700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499407ae70 con 0x7f4994108120 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.270+0000 7f49988cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f499410a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.271+0000 7f49988cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f499410a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49030/0 (socket says 172.21.15.17:49030) 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.271+0000 7f49988cd700 1 -- 172.21.15.17:0/2192619310 learned_addr learned my addr 172.21.15.17:0/2192619310 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.271+0000 7f49988cd700 1 -- 172.21.15.17:0/2192619310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499410aa50 con 0x7f4994108120 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.271+0000 7f49988cd700 1 --2- 172.21.15.17:0/2192619310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f499410a510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49880098a0 tx=0x7f4988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b863ef2f704b3c8f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:32.781 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.271+0000 7f49937fe700 1 -- 172.21.15.17:0/2192619310 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4988004160 con 0x7f4994108120 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.271+0000 7f49937fe700 1 -- 172.21.15.17:0/2192619310 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49880042c0 con 0x7f4994108120 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.271+0000 7f49937fe700 1 -- 172.21.15.17:0/2192619310 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4988011a80 con 0x7f4994108120 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 -- 172.21.15.17:0/2192619310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 msgr2=0x7f499410a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 --2- 172.21.15.17:0/2192619310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f499410a510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49880098a0 tx=0x7f4988009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 -- 172.21.15.17:0/2192619310 shutdown_connections 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 --2- 172.21.15.17:0/2192619310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f499410a510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 -- 172.21.15.17:0/2192619310 >> 172.21.15.17:0/2192619310 conn(0x7f4994101b90 msgr2=0x7f4994103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 -- 172.21.15.17:0/2192619310 shutdown_connections 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 -- 172.21.15.17:0/2192619310 wait complete. 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 Processor -- start 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.272+0000 7f499ab31700 1 -- start start 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f499ab31700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f4994107950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:32.782 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f499ab31700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4994107e90 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f49988cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f4994107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f49988cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f4994107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49036/0 (socket says 172.21.15.17:49036) 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f49988cd700 1 -- 172.21.15.17:0/2204467855 learned_addr learned my addr 172.21.15.17:0/2204467855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f49988cd700 1 -- 172.21.15.17:0/2204467855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988009550 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f49988cd700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f4994107950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4988009130 tx=0x7f4988011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f4991ffb700 1 -- 172.21.15.17:0/2204467855 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4988003aa0 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f4991ffb700 1 -- 172.21.15.17:0/2204467855 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4988003c00 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49941060d0 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.273+0000 7f4991ffb700 1 -- 172.21.15.17:0/2204467855 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49880248a0 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.274+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4994106570 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.274+0000 7f4991ffb700 1 -- 172.21.15.17:0/2204467855 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f4988024a40 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.274+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499410b890 con 0x7f4994108120 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.274+0000 7f4991ffb700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4984038ee0 0x7f498403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:32.783 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.275+0000 7f4991ffb700 1 -- 172.21.15.17:0/2204467855 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f498804d1f0 con 0x7f4994108120 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.275+0000 7f4993fff700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4984038ee0 0x7f498403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.275+0000 7f4993fff700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4984038ee0 0x7f498403b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4980006fd0 tx=0x7f4980006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.281+0000 7f4991ffb700 1 -- 172.21.15.17:0/2204467855 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4988024cf0 con 0x7f4994108120 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.523+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f499410baa0 con 0x7f4994108120 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.525+0000 7f4991ffb700 1 -- 172.21.15.17:0/2204467855 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4988018be0 con 0x7f4994108120 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.527+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4984038ee0 msgr2=0x7f498403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.527+0000 7f499ab31700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4984038ee0 0x7f498403b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4980006fd0 tx=0x7f4980006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.527+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 msgr2=0x7f4994107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.527+0000 7f499ab31700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f4994107950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4988009130 tx=0x7f4988011b00 comp rx=0 tx=0).stop 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.528+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 shutdown_connections 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.528+0000 7f499ab31700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f4984038ee0 0x7f498403b3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:32.784 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.528+0000 7f499ab31700 1 --2- 172.21.15.17:0/2204467855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4994108120 0x7f4994107950 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:32.785 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.528+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 >> 172.21.15.17:0/2204467855 conn(0x7f4994101b90 msgr2=0x7f49941026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:32.785 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.528+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 shutdown_connections 2024-02-14T14:53:32.785 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:31.528+0000 7f499ab31700 1 -- 172.21.15.17:0/2204467855 wait complete. 2024-02-14T14:53:33.444 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:33 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:34.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.231+0000 7f1d7ead7700 1 Processor -- start 2024-02-14T14:53:34.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.231+0000 7f1d7ead7700 1 -- start start 2024-02-14T14:53:34.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.231+0000 7f1d7ead7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7810a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:34.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.231+0000 7f1d7ead7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7807ae70 con 0x7f1d78108140 2024-02-14T14:53:34.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.231+0000 7f1d7c873700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7810a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:34.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.231+0000 7f1d7c873700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7810a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49038/0 (socket says 172.21.15.17:49038) 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.231+0000 7f1d7c873700 1 -- 172.21.15.17:0/614053323 learned_addr learned my addr 172.21.15.17:0/614053323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.232+0000 7f1d7c873700 1 -- 172.21.15.17:0/614053323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7810aa70 con 0x7f1d78108140 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.232+0000 7f1d7c873700 1 --2- 172.21.15.17:0/614053323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7810a530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0098a0 tx=0x7f1d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776611c4be9ab4aa server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.232+0000 7f1d777fe700 1 -- 172.21.15.17:0/614053323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d6c004160 con 0x7f1d78108140 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.232+0000 7f1d777fe700 1 -- 172.21.15.17:0/614053323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d6c0042c0 con 0x7f1d78108140 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.232+0000 7f1d777fe700 1 -- 172.21.15.17:0/614053323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d6c011a80 con 0x7f1d78108140 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 -- 172.21.15.17:0/614053323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 msgr2=0x7f1d7810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 --2- 172.21.15.17:0/614053323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7810a530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0098a0 tx=0x7f1d6c009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 -- 172.21.15.17:0/614053323 shutdown_connections 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 --2- 172.21.15.17:0/614053323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7810a530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 -- 172.21.15.17:0/614053323 >> 172.21.15.17:0/614053323 conn(0x7f1d78101b90 msgr2=0x7f1d78103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 -- 172.21.15.17:0/614053323 shutdown_connections 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 -- 172.21.15.17:0/614053323 wait complete. 2024-02-14T14:53:34.021 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 Processor -- start 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 -- start start 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.233+0000 7f1d7ead7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7819b900 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d7c873700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d7c873700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49040/0 (socket says 172.21.15.17:49040) 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d7c873700 1 -- 172.21.15.17:0/3133468905 learned_addr learned my addr 172.21.15.17:0/3133468905 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d7c873700 1 -- 172.21.15.17:0/3133468905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c009550 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d7c873700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7819b3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c011740 tx=0x7f1d6c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d75ffb700 1 -- 172.21.15.17:0/3133468905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d6c003b20 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d75ffb700 1 -- 172.21.15.17:0/3133468905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d6c01a910 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d75ffb700 1 -- 172.21.15.17:0/3133468905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d6c024900 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7819bb00 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.234+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7819bfa0 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.235+0000 7f1d75ffb700 1 -- 172.21.15.17:0/3133468905 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f1d6c01aa80 con 0x7f1d78108140 2024-02-14T14:53:34.022 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.235+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7810b8b0 con 0x7f1d78108140 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.236+0000 7f1d75ffb700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1d68038ee0 0x7f1d6803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.236+0000 7f1d75ffb700 1 -- 172.21.15.17:0/3133468905 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1d6c04d230 con 0x7f1d78108140 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.236+0000 7f1d77fff700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1d68038ee0 0x7f1d6803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.236+0000 7f1d77fff700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1d68038ee0 0x7f1d6803b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006fd0 tx=0x7f1d64006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.241+0000 7f1d75ffb700 1 -- 172.21.15.17:0/3133468905 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d6c01ae10 con 0x7f1d78108140 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.443+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f1d7819e880 con 0x7f1d78108140 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.446+0000 7f1d75ffb700 1 -- 172.21.15.17:0/3133468905 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1d6c018350 con 0x7f1d78108140 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1d68038ee0 msgr2=0x7f1d6803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1d68038ee0 0x7f1d6803b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006fd0 tx=0x7f1d64006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 msgr2=0x7f1d7819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7819b3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c011740 tx=0x7f1d6c011a50 comp rx=0 tx=0).stop 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 shutdown_connections 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f1d68038ee0 0x7f1d6803b3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:34.023 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 --2- 172.21.15.17:0/3133468905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78108140 0x7f1d7819b3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:34.024 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 >> 172.21.15.17:0/3133468905 conn(0x7f1d78101b90 msgr2=0x7f1d78102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:34.024 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 shutdown_connections 2024-02-14T14:53:34.024 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:33.449+0000 7f1d7ead7700 1 -- 172.21.15.17:0/3133468905 wait complete. 2024-02-14T14:53:34.024 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling the dashboard module... 2024-02-14T14:53:34.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:34 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3133468905' entity='client.admin' 2024-02-14T14:53:34.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:34 smithi017 ceph-mon[32549]: from='mgr.14118 172.21.15.17:0/2237814527' entity='mgr.x' 2024-02-14T14:53:35.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:35 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2751938137' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-14T14:53:35.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:35 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: ignoring --setuser ceph since I am not root 2024-02-14T14:53:35.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:35 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: ignoring --setgroup ceph since I am not root 2024-02-14T14:53:36.056 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.491+0000 7f30f24d3700 1 Processor -- start 2024-02-14T14:53:36.056 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.491+0000 7f30f24d3700 1 -- start start 2024-02-14T14:53:36.056 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.491+0000 7f30f24d3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec10a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:36.056 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.491+0000 7f30f24d3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec07aeb0 con 0x7f30ec108120 2024-02-14T14:53:36.056 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.491+0000 7f30ebfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec10a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:36.056 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.491+0000 7f30ebfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec10a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49042/0 (socket says 172.21.15.17:49042) 2024-02-14T14:53:36.056 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.491+0000 7f30ebfff700 1 -- 172.21.15.17:0/1753676185 learned_addr learned my addr 172.21.15.17:0/1753676185 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.492+0000 7f30ebfff700 1 -- 172.21.15.17:0/1753676185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec10aa50 con 0x7f30ec108120 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.492+0000 7f30ebfff700 1 --2- 172.21.15.17:0/1753676185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec10a510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0098a0 tx=0x7f30dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d210b51ec5869fed server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.492+0000 7f30eaffd700 1 -- 172.21.15.17:0/1753676185 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30dc004160 con 0x7f30ec108120 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.492+0000 7f30eaffd700 1 -- 172.21.15.17:0/1753676185 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30dc0042c0 con 0x7f30ec108120 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.492+0000 7f30eaffd700 1 -- 172.21.15.17:0/1753676185 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30dc011a80 con 0x7f30ec108120 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 -- 172.21.15.17:0/1753676185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 msgr2=0x7f30ec10a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 --2- 172.21.15.17:0/1753676185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec10a510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0098a0 tx=0x7f30dc009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 -- 172.21.15.17:0/1753676185 shutdown_connections 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 --2- 172.21.15.17:0/1753676185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec10a510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 -- 172.21.15.17:0/1753676185 >> 172.21.15.17:0/1753676185 conn(0x7f30ec101bb0 msgr2=0x7f30ec103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 -- 172.21.15.17:0/1753676185 shutdown_connections 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 -- 172.21.15.17:0/1753676185 wait complete. 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 Processor -- start 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.493+0000 7f30f24d3700 1 -- start start 2024-02-14T14:53:36.057 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30f24d3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30f24d3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec19db70 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30ebfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30ebfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49052/0 (socket says 172.21.15.17:49052) 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30ebfff700 1 -- 172.21.15.17:0/2751938137 learned_addr learned my addr 172.21.15.17:0/2751938137 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30ebfff700 1 -- 172.21.15.17:0/2751938137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc009550 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30ebfff700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec19d630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30dc011740 tx=0x7f30dc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30dc003b20 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec19dd70 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30dc01a910 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30dc024900 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.494+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ec19e210 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.495+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f30dc024b70 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.495+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec10b8f0 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.496+0000 7f30e97fa700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f30d8038e90 0x7f30d803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.496+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f30dc04d3a0 con 0x7f30ec108120 2024-02-14T14:53:36.058 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.496+0000 7f30eb7fe700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f30d8038e90 0x7f30d803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.496+0000 7f30eb7fe700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f30d8038e90 0x7f30d803b350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f30d4006fd0 tx=0x7f30d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.505+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30dc024e20 con 0x7f30ec108120 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:34.773+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f30ec1a0af0 con 0x7f30ec108120 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.447+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f30dc018860 con 0x7f30ec108120 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.449+0000 7f30e97fa700 1 -- 172.21.15.17:0/2751938137 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f30dc00bcc0 con 0x7f30ec108120 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f30d8038e90 msgr2=0x7f30d803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f30d8038e90 0x7f30d803b350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f30d4006fd0 tx=0x7f30d4006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 msgr2=0x7f30ec19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec19d630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30dc011740 tx=0x7f30dc011a50 comp rx=0 tx=0).stop 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 shutdown_connections 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7f30d8038e90 0x7f30d803b350 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 --2- 172.21.15.17:0/2751938137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30ec108120 0x7f30ec19d630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 >> 172.21.15.17:0/2751938137 conn(0x7f30ec101bb0 msgr2=0x7f30ec102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 shutdown_connections 2024-02-14T14:53:36.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:35.451+0000 7f30f24d3700 1 -- 172.21.15.17:0/2751938137 wait complete. 2024-02-14T14:53:36.386 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:36 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:36.038+0000 7f0a6793d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T14:53:36.386 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:36 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:36.138+0000 7f0a6793d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T14:53:36.691 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:36 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2751938137' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-14T14:53:36.691 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:36 smithi017 ceph-mon[32549]: mgrmap e9: x(active, since 19s) 2024-02-14T14:53:37.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:36 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:36.831+0000 7f0a6793d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T14:53:37.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:36 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:36.926+0000 7f0a6793d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T14:53:37.363 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:37.072+0000 7f0a6793d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T14:53:37.363 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:37.171+0000 7f0a6793d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T14:53:37.363 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:37.256+0000 7f0a6793d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.485+0000 7fe3d2b28700 1 Processor -- start 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.485+0000 7fe3d2b28700 1 -- start start 2024-02-14T14:53:37.371 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.485+0000 7fe3d2b28700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc1069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.485+0000 7fe3d2b28700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc106f80 con 0x7fe3cc106590 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.485+0000 7fe3d08c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc1069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.485+0000 7fe3d08c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc1069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49080/0 (socket says 172.21.15.17:49080) 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.485+0000 7fe3d08c4700 1 -- 172.21.15.17:0/3467217881 learned_addr learned my addr 172.21.15.17:0/3467217881 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.486+0000 7fe3d08c4700 1 -- 172.21.15.17:0/3467217881 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc107790 con 0x7fe3cc106590 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.486+0000 7fe3d08c4700 1 --2- 172.21.15.17:0/3467217881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc1069b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00098a0 tx=0x7fe3c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e386fca9da4168 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.486+0000 7fe3cb7fe700 1 -- 172.21.15.17:0/3467217881 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0004160 con 0x7fe3cc106590 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.486+0000 7fe3cb7fe700 1 -- 172.21.15.17:0/3467217881 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c00042c0 con 0x7fe3cc106590 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.486+0000 7fe3cb7fe700 1 -- 172.21.15.17:0/3467217881 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0011a80 con 0x7fe3cc106590 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 -- 172.21.15.17:0/3467217881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 msgr2=0x7fe3cc1069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 --2- 172.21.15.17:0/3467217881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc1069b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00098a0 tx=0x7fe3c0009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:37.372 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 -- 172.21.15.17:0/3467217881 shutdown_connections 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 --2- 172.21.15.17:0/3467217881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc1069b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 -- 172.21.15.17:0/3467217881 >> 172.21.15.17:0/3467217881 conn(0x7fe3cc101c10 msgr2=0x7fe3cc104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 -- 172.21.15.17:0/3467217881 shutdown_connections 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 -- 172.21.15.17:0/3467217881 wait complete. 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 Processor -- start 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 -- start start 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc191700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.487+0000 7fe3d2b28700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc191c40 con 0x7fe3cc106590 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3d08c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc191700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3d08c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc191700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49088/0 (socket says 172.21.15.17:49088) 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3d08c4700 1 -- 172.21.15.17:0/104244864 learned_addr learned my addr 172.21.15.17:0/104244864 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3d08c4700 1 -- 172.21.15.17:0/104244864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009550 con 0x7fe3cc106590 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3d08c4700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc191700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c00 tx=0x7fe3c0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3c9ffb700 1 -- 172.21.15.17:0/104244864 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c00039e0 con 0x7fe3cc106590 2024-02-14T14:53:37.373 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3c9ffb700 1 -- 172.21.15.17:0/104244864 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0003b40 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc18fe30 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.488+0000 7fe3c9ffb700 1 -- 172.21.15.17:0/104244864 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0024820 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.489+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc1902d0 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.489+0000 7fe3c9ffb700 1 -- 172.21.15.17:0/104244864 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fe3c0024980 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.489+0000 7fe3c9ffb700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 0x7fe3bc03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.490+0000 7fe3c9ffb700 1 -- 172.21.15.17:0/104244864 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe3c004d070 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.490+0000 7fe3cbfff700 1 -- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 msgr2=0x7fe3bc03b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1044675107 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.490+0000 7fe3cbfff700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 0x7fe3bc03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.490+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc190ce0 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.495+0000 7fe3c9ffb700 1 -- 172.21.15.17:0/104244864 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3c0024e60 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.691+0000 7fe3cbfff700 1 -- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 msgr2=0x7fe3bc03b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1044675107 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.691+0000 7fe3cbfff700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 0x7fe3bc03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.772+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe3cc10a9c0 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.772+0000 7fe3c9ffb700 1 -- 172.21.15.17:0/104244864 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe3c0024c80 con 0x7fe3cc106590 2024-02-14T14:53:37.374 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 msgr2=0x7fe3bc03b3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 0x7fe3bc03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 msgr2=0x7fe3cc191700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc191700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c00 tx=0x7fe3c0011b00 comp rx=0 tx=0).stop 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 shutdown_connections 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7fe3bc038f30 0x7fe3bc03b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 --2- 172.21.15.17:0/104244864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3cc106590 0x7fe3cc191700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 >> 172.21.15.17:0/104244864 conn(0x7fe3cc101c10 msgr2=0x7fe3cc102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 shutdown_connections 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:36.776+0000 7fe3d2b28700 1 -- 172.21.15.17:0/104244864 wait complete. 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for the mgr to restart... 2024-02-14T14:53:37.375 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr epoch 9... 2024-02-14T14:53:37.654 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/104244864' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T14:53:37.655 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:37.537+0000 7f0a6793d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T14:53:37.655 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:37.654+0000 7f0a6793d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T14:53:37.989 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:37.744+0000 7f0a6793d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T14:53:37.989 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:37.844+0000 7f0a6793d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T14:53:38.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:38 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:38.023+0000 7f0a6793d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T14:53:39.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:39 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:39.022+0000 7f0a6793d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T14:53:39.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:39 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:39.157+0000 7f0a6793d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T14:53:39.814 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:39 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:39.517+0000 7f0a6793d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T14:53:39.814 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:39 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:39.610+0000 7f0a6793d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T14:53:39.814 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:39 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:39.813+0000 7f0a6793d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:53:40.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:40.005+0000 7f0a6793d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T14:53:40.565 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:40.301+0000 7f0a6793d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:53:40.565 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:40.564+0000 7f0a6793d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T14:53:41.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:40.810+0000 7f0a6793d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T14:53:42.010 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:41 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:41.645+0000 7f0a6793d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T14:53:42.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:42.232+0000 7f0a6793d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T14:53:42.612 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:42.334+0000 7f0a6793d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T14:53:42.612 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:53:42.556+0000 7f0a6793d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T14:53:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: Active manager daemon x restarted 2024-02-14T14:53:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: Activating manager daemon x 2024-02-14T14:53:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: osdmap e3: 0 total, 0 up, 0 in 2024-02-14T14:53:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: mgrmap e10: x(active, starting, since 0.00361514s) 2024-02-14T14:53:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:53:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T14:53:43.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T14:53:43.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T14:53:43.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T14:53:43.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: Manager daemon x is now available 2024-02-14T14:53:43.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:43.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:43.813 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-14T14:53:43.813 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-14T14:53:43.814 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:43 smithi017 ceph-mon[32549]: mgrmap e11: x(active, since 1.00675s) 2024-02-14T14:53:44.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.783+0000 7ff81cc34700 1 Processor -- start 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.783+0000 7ff81cc34700 1 -- start start 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.783+0000 7ff81cc34700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff818107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.783+0000 7ff81cc34700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8181080b0 con 0x7ff8181076c0 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.783+0000 7ff81659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff818107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.783+0000 7ff81659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff818107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49096/0 (socket says 172.21.15.17:49096) 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.783+0000 7ff81659c700 1 -- 172.21.15.17:0/2067075520 learned_addr learned my addr 172.21.15.17:0/2067075520 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.784+0000 7ff81659c700 1 -- 172.21.15.17:0/2067075520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818108870 con 0x7ff8181076c0 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.784+0000 7ff81659c700 1 --2- 172.21.15.17:0/2067075520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff818107ae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098a0 tx=0x7ff808009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38efad591b38e2e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.784+0000 7ff81559a700 1 -- 172.21.15.17:0/2067075520 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff808004160 con 0x7ff8181076c0 2024-02-14T14:53:44.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.784+0000 7ff81559a700 1 -- 172.21.15.17:0/2067075520 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8080042c0 con 0x7ff8181076c0 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.784+0000 7ff81559a700 1 -- 172.21.15.17:0/2067075520 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff808011a80 con 0x7ff8181076c0 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 -- 172.21.15.17:0/2067075520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 msgr2=0x7ff818107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 --2- 172.21.15.17:0/2067075520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff818107ae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098a0 tx=0x7ff808009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 -- 172.21.15.17:0/2067075520 shutdown_connections 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 --2- 172.21.15.17:0/2067075520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff818107ae0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 -- 172.21.15.17:0/2067075520 >> 172.21.15.17:0/2067075520 conn(0x7ff818102d20 msgr2=0x7ff818105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 -- 172.21.15.17:0/2067075520 shutdown_connections 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 -- 172.21.15.17:0/2067075520 wait complete. 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 Processor -- start 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.785+0000 7ff81cc34700 1 -- start start 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81cc34700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff81819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81cc34700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81819b9f0 con 0x7ff8181076c0 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff81819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff81819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49110/0 (socket says 172.21.15.17:49110) 2024-02-14T14:53:44.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81659c700 1 -- 172.21.15.17:0/52076229 learned_addr learned my addr 172.21.15.17:0/52076229 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81659c700 1 -- 172.21.15.17:0/52076229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808009550 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81659c700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff81819b4b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff808000c00 tx=0x7ff808011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8080039e0 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff808003b40 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff808024820 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.786+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81819bbf0 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.787+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81819c090 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.787+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7ff808024980 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.787+0000 7ff8077fe700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.787+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 --> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff80003bb60 con 0x7ff800038f90 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.787+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff80801ab80 con 0x7ff8181076c0 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.787+0000 7ff815d9b700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 msgr2=0x7ff80003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1044675107 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.788+0000 7ff815d9b700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.989+0000 7ff815d9b700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 msgr2=0x7ff80003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1044675107 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:37.989+0000 7ff815d9b700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T14:53:44.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:38.390+0000 7ff815d9b700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 msgr2=0x7ff80003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1044675107 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:38.390+0000 7ff815d9b700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:39.191+0000 7ff815d9b700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 msgr2=0x7ff80003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1044675107 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:39.191+0000 7ff815d9b700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:40.792+0000 7ff815d9b700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 msgr2=0x7ff80003b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1044675107 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:40.792+0000 7ff815d9b700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:42.571+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7ff80802da00 con 0x7ff8181076c0 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:42.571+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 msgr2=0x7ff80003b450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:42.571+0000 7ff8077fe700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.573+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7ff80804ea70 con 0x7ff8181076c0 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.573+0000 7ff8077fe700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff80003ca30 0x7ff80003ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.574+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff80003bb60 con 0x7ff80003ca30 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.574+0000 7ff815d9b700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff80003ca30 0x7ff80003ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.574+0000 7ff815d9b700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff80003ca30 0x7ff80003ee20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff80c003a10 tx=0x7ff80c009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.575+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff80003bb60 con 0x7ff80003ca30 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.588+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff81810baa0 con 0x7ff80003ca30 2024-02-14T14:53:44.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.588+0000 7ff8077fe700 1 -- 172.21.15.17:0/52076229 <== mgr.14150 v2:172.21.15.17:6800/3796717329 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff81810baa0 con 0x7ff80003ca30 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff80003ca30 msgr2=0x7ff80003ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff80003ca30 0x7ff80003ee20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff80c003a10 tx=0x7ff80c009270 comp rx=0 tx=0).stop 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 msgr2=0x7ff81819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff81819b4b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff808000c00 tx=0x7ff808011b00 comp rx=0 tx=0).stop 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 shutdown_connections 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8181076c0 0x7ff81819b4b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff80003ca30 0x7ff80003ee20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 --2- 172.21.15.17:0/52076229 >> [v2:172.21.15.17:6800/1044675107,v1:172.21.15.17:6801/1044675107] conn(0x7ff800038f90 0x7ff80003b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 >> 172.21.15.17:0/52076229 conn(0x7ff818102d20 msgr2=0x7ff818103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 shutdown_connections 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:43.589+0000 7ff81cc34700 1 -- 172.21.15.17:0/52076229 wait complete. 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:mgr epoch 9 is available 2024-02-14T14:53:44.520 INFO:teuthology.orchestra.run.smithi017.stdout:Generating a dashboard self-signed certificate... 2024-02-14T14:53:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:45 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:43] ENGINE Bus STARTING 2024-02-14T14:53:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:45 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:43] ENGINE Serving on http://172.21.15.17:8765 2024-02-14T14:53:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:45 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:43] ENGINE Serving on https://172.21.15.17:7150 2024-02-14T14:53:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:45 smithi017 ceph-mon[32549]: [14/Feb/2024:14:53:43] ENGINE Bus STARTED 2024-02-14T14:53:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.939+0000 7fc2190f3700 1 Processor -- start 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.939+0000 7fc2190f3700 1 -- start start 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.939+0000 7fc2190f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc214106390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.939+0000 7fc2190f3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21407ae30 con 0x7fc214105f70 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc212d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc214106390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc212d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc214106390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48080/0 (socket says 172.21.15.17:48080) 2024-02-14T14:53:45.869 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc212d9d700 1 -- 172.21.15.17:0/351960765 learned_addr learned my addr 172.21.15.17:0/351960765 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc212d9d700 1 -- 172.21.15.17:0/351960765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2141068d0 con 0x7fc214105f70 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc212d9d700 1 --2- 172.21.15.17:0/351960765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc214106390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098a0 tx=0x7fc204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445ee32f51b00550 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc211d9b700 1 -- 172.21.15.17:0/351960765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc204004160 con 0x7fc214105f70 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc211d9b700 1 -- 172.21.15.17:0/351960765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2040042c0 con 0x7fc214105f70 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.940+0000 7fc211d9b700 1 -- 172.21.15.17:0/351960765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc204011a80 con 0x7fc214105f70 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.941+0000 7fc2190f3700 1 -- 172.21.15.17:0/351960765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 msgr2=0x7fc214106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.941+0000 7fc2190f3700 1 --2- 172.21.15.17:0/351960765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc214106390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098a0 tx=0x7fc204009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.941+0000 7fc2190f3700 1 -- 172.21.15.17:0/351960765 shutdown_connections 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.941+0000 7fc2190f3700 1 --2- 172.21.15.17:0/351960765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc214106390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.941+0000 7fc2190f3700 1 -- 172.21.15.17:0/351960765 >> 172.21.15.17:0/351960765 conn(0x7fc214101c10 msgr2=0x7fc214104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.941+0000 7fc2190f3700 1 -- 172.21.15.17:0/351960765 shutdown_connections 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.941+0000 7fc2190f3700 1 -- 172.21.15.17:0/351960765 wait complete. 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.942+0000 7fc2190f3700 1 Processor -- start 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.942+0000 7fc2190f3700 1 -- start start 2024-02-14T14:53:45.870 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.942+0000 7fc2190f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc21419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.942+0000 7fc2190f3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419dc80 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.942+0000 7fc212d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc21419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.942+0000 7fc212d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc21419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48086/0 (socket says 172.21.15.17:48086) 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.942+0000 7fc212d9d700 1 -- 172.21.15.17:0/3406089652 learned_addr learned my addr 172.21.15.17:0/3406089652 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.943+0000 7fc212d9d700 1 -- 172.21.15.17:0/3406089652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc204009550 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.943+0000 7fc212d9d700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc21419d740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc204009130 tx=0x7fc204011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.943+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc204003aa0 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.943+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc204003c00 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.943+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21419de80 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.943+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2040248a0 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.943+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21419ea30 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.944+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7fc204024a40 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.944+0000 7fc1fbfff700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc1fc038e10 0x7fc1fc03b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.944+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc20404e090 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.944+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc214068df0 con 0x7fc214105f70 2024-02-14T14:53:45.871 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.944+0000 7fc21259c700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc1fc038e10 0x7fc1fc03b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.945+0000 7fc21259c700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc1fc038e10 0x7fc1fc03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc208006fd0 tx=0x7fc208006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:44.950+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc204024d40 con 0x7fc214105f70 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.164+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc21419ec70 con 0x7fc1fc038e10 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.190+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fc204018350 con 0x7fc214105f70 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.295+0000 7fc1fbfff700 1 -- 172.21.15.17:0/3406089652 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc21419ec70 con 0x7fc1fc038e10 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc1fc038e10 msgr2=0x7fc1fc03b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc1fc038e10 0x7fc1fc03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc208006fd0 tx=0x7fc208006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 msgr2=0x7fc21419d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc21419d740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc204009130 tx=0x7fc204011b00 comp rx=0 tx=0).stop 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 shutdown_connections 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc1fc038e10 0x7fc1fc03b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 --2- 172.21.15.17:0/3406089652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc214105f70 0x7fc21419d740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 >> 172.21.15.17:0/3406089652 conn(0x7fc214101c10 msgr2=0x7fc214194480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 shutdown_connections 2024-02-14T14:53:45.872 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:45.297+0000 7fc2190f3700 1 -- 172.21.15.17:0/3406089652 wait complete. 2024-02-14T14:53:45.873 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial admin user... 2024-02-14T14:53:46.263 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:46 smithi017 ceph-mon[32549]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:46.263 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:46 smithi017 ceph-mon[32549]: mgrmap e12: x(active, since 2s) 2024-02-14T14:53:46.263 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:46.263 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:46.264 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:47.290 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:47 smithi017 ceph-mon[32549]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:47.290 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:47.295 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9sqEKVLll4TrMQCTpXEw2eBnBVmlT9pxqkov89jmeqBAkB/OcB53y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707922426, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-14T14:53:47.295 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.262+0000 7f52a9283700 1 Processor -- start 2024-02-14T14:53:47.295 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.262+0000 7f52a9283700 1 -- start start 2024-02-14T14:53:47.295 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.262+0000 7f52a9283700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a410a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.262+0000 7f52a9283700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a407ae70 con 0x7f52a4108140 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.262+0000 7f52a2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a410a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.262+0000 7f52a2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a410a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48092/0 (socket says 172.21.15.17:48092) 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.262+0000 7f52a2ffd700 1 -- 172.21.15.17:0/3799065309 learned_addr learned my addr 172.21.15.17:0/3799065309 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.263+0000 7f52a2ffd700 1 -- 172.21.15.17:0/3799065309 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a410aa70 con 0x7f52a4108140 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.263+0000 7f52a2ffd700 1 --2- 172.21.15.17:0/3799065309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a410a530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52940098a0 tx=0x7f5294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51f8e662100b62c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.263+0000 7f52a1ffb700 1 -- 172.21.15.17:0/3799065309 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5294004160 con 0x7f52a4108140 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.263+0000 7f52a1ffb700 1 -- 172.21.15.17:0/3799065309 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52940042c0 con 0x7f52a4108140 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.263+0000 7f52a1ffb700 1 -- 172.21.15.17:0/3799065309 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5294004160 con 0x7f52a4108140 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.264+0000 7f52a9283700 1 -- 172.21.15.17:0/3799065309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 msgr2=0x7f52a410a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.264+0000 7f52a9283700 1 --2- 172.21.15.17:0/3799065309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a410a530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52940098a0 tx=0x7f5294009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.264+0000 7f52a9283700 1 -- 172.21.15.17:0/3799065309 shutdown_connections 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.264+0000 7f52a9283700 1 --2- 172.21.15.17:0/3799065309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a410a530 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:47.296 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.264+0000 7f52a9283700 1 -- 172.21.15.17:0/3799065309 >> 172.21.15.17:0/3799065309 conn(0x7f52a4101b90 msgr2=0x7f52a4103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.264+0000 7f52a9283700 1 -- 172.21.15.17:0/3799065309 shutdown_connections 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.264+0000 7f52a9283700 1 -- 172.21.15.17:0/3799065309 wait complete. 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.265+0000 7f52a9283700 1 Processor -- start 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.265+0000 7f52a9283700 1 -- start start 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a9283700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a9283700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a419b950 con 0x7f52a4108140 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48094/0 (socket says 172.21.15.17:48094) 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a2ffd700 1 -- 172.21.15.17:0/1722444859 learned_addr learned my addr 172.21.15.17:0/1722444859 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a2ffd700 1 -- 172.21.15.17:0/1722444859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5294009550 con 0x7f52a4108140 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a2ffd700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a419b410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52940119f0 tx=0x7f5294004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f528bfff700 1 -- 172.21.15.17:0/1722444859 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52940037d0 con 0x7f52a4108140 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a419bb50 con 0x7f52a4108140 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f528bfff700 1 -- 172.21.15.17:0/1722444859 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5294003930 con 0x7f52a4108140 2024-02-14T14:53:47.297 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.266+0000 7f528bfff700 1 -- 172.21.15.17:0/1722444859 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52940245a0 con 0x7f52a4108140 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.267+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a419bff0 con 0x7f52a4108140 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.267+0000 7f528bfff700 1 -- 172.21.15.17:0/1722444859 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f529402d3f0 con 0x7f52a4108140 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.268+0000 7f528bfff700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f528c038f20 0x7f528c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.268+0000 7f528bfff700 1 -- 172.21.15.17:0/1722444859 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f529404d240 con 0x7f52a4108140 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.268+0000 7f52a27fc700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f528c038f20 0x7f528c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.268+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a410b8b0 con 0x7f52a4108140 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.268+0000 7f52a27fc700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f528c038f20 0x7f528c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5298006fd0 tx=0x7f5298006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.275+0000 7f528bfff700 1 -- 172.21.15.17:0/1722444859 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5294010970 con 0x7f52a4108140 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.498+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f52a4063170 con 0x7f528c038f20 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.745+0000 7f528bfff700 1 -- 172.21.15.17:0/1722444859 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f52a4063170 con 0x7f528c038f20 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f528c038f20 msgr2=0x7f528c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f528c038f20 0x7f528c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5298006fd0 tx=0x7f5298006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 msgr2=0x7f52a419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:47.298 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a419b410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52940119f0 tx=0x7f5294004230 comp rx=0 tx=0).stop 2024-02-14T14:53:47.299 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 shutdown_connections 2024-02-14T14:53:47.299 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f528c038f20 0x7f528c03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:47.299 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 --2- 172.21.15.17:0/1722444859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52a4108140 0x7f52a419b410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:47.299 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 >> 172.21.15.17:0/1722444859 conn(0x7f52a4101b90 msgr2=0x7f52a4102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:47.299 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 shutdown_connections 2024-02-14T14:53:47.299 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:46.747+0000 7f52a9283700 1 -- 172.21.15.17:0/1722444859 wait complete. 2024-02-14T14:53:47.299 INFO:teuthology.orchestra.run.smithi017.stdout:Fetching dashboard port number... 2024-02-14T14:53:48.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:48 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/913716822' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-14T14:53:48.578 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 8443 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.701+0000 7f36d15fa700 1 Processor -- start 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.701+0000 7f36d15fa700 1 -- start start 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.701+0000 7f36d15fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.701+0000 7f36d15fa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc106390 con 0x7f36cc1059a0 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.701+0000 7f36caffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.701+0000 7f36caffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc105dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48102/0 (socket says 172.21.15.17:48102) 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.701+0000 7f36caffd700 1 -- 172.21.15.17:0/1642226920 learned_addr learned my addr 172.21.15.17:0/1642226920 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.702+0000 7f36caffd700 1 -- 172.21.15.17:0/1642226920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc108930 con 0x7f36cc1059a0 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.702+0000 7f36caffd700 1 --2- 172.21.15.17:0/1642226920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc105dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098a0 tx=0x7f36c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514c781a21915f81 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.702+0000 7f36c9ffb700 1 -- 172.21.15.17:0/1642226920 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c0004160 con 0x7f36cc1059a0 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.702+0000 7f36c9ffb700 1 -- 172.21.15.17:0/1642226920 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36c00042c0 con 0x7f36cc1059a0 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.702+0000 7f36c9ffb700 1 -- 172.21.15.17:0/1642226920 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c0011a80 con 0x7f36cc1059a0 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.702+0000 7f36d15fa700 1 -- 172.21.15.17:0/1642226920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 msgr2=0x7f36cc105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:48.579 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.702+0000 7f36d15fa700 1 --2- 172.21.15.17:0/1642226920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc105dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098a0 tx=0x7f36c0009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.703+0000 7f36d15fa700 1 -- 172.21.15.17:0/1642226920 shutdown_connections 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.703+0000 7f36d15fa700 1 --2- 172.21.15.17:0/1642226920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc105dc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.703+0000 7f36d15fa700 1 -- 172.21.15.17:0/1642226920 >> 172.21.15.17:0/1642226920 conn(0x7f36cc101350 msgr2=0x7f36cc103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.703+0000 7f36d15fa700 1 -- 172.21.15.17:0/1642226920 shutdown_connections 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.703+0000 7f36d15fa700 1 -- 172.21.15.17:0/1642226920 wait complete. 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.703+0000 7f36d15fa700 1 Processor -- start 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.703+0000 7f36d15fa700 1 -- start start 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36d15fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36d15fa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc19b970 con 0x7f36cc1059a0 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36caffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36caffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48106/0 (socket says 172.21.15.17:48106) 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36caffd700 1 -- 172.21.15.17:0/913716822 learned_addr learned my addr 172.21.15.17:0/913716822 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36caffd700 1 -- 172.21.15.17:0/913716822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009550 con 0x7f36cc1059a0 2024-02-14T14:53:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36caffd700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc19b430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36c0011740 tx=0x7f36c0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.704+0000 7f36b3fff700 1 -- 172.21.15.17:0/913716822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c0003b20 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.705+0000 7f36b3fff700 1 -- 172.21.15.17:0/913716822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36c001a910 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.705+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc19bb70 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.705+0000 7f36b3fff700 1 -- 172.21.15.17:0/913716822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c0024850 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.705+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc19c010 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.706+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36cc10bb60 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.706+0000 7f36b3fff700 1 -- 172.21.15.17:0/913716822 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f36c001a430 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.706+0000 7f36b3fff700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36b4038f90 0x7f36b403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.706+0000 7f36b3fff700 1 -- 172.21.15.17:0/913716822 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f36c004d410 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.706+0000 7f36ca7fc700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36b4038f90 0x7f36b403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.706+0000 7f36ca7fc700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36b4038f90 0x7f36b403b450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36bc006fd0 tx=0x7f36bc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.711+0000 7f36b3fff700 1 -- 172.21.15.17:0/913716822 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f36c0018ba0 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.916+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f36cc19e8f0 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.916+0000 7f36b3fff700 1 -- 172.21.15.17:0/913716822 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f36c0018ba0 con 0x7f36cc1059a0 2024-02-14T14:53:48.581 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.918+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36b4038f90 msgr2=0x7f36b403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.918+0000 7f36d15fa700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36b4038f90 0x7f36b403b450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36bc006fd0 tx=0x7f36bc006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.918+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 msgr2=0x7f36cc19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.918+0000 7f36d15fa700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc19b430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36c0011740 tx=0x7f36c0011b00 comp rx=0 tx=0).stop 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.919+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 shutdown_connections 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.919+0000 7f36d15fa700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36b4038f90 0x7f36b403b450 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.919+0000 7f36d15fa700 1 --2- 172.21.15.17:0/913716822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36cc1059a0 0x7f36cc19b430 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.919+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 >> 172.21.15.17:0/913716822 conn(0x7f36cc101350 msgr2=0x7f36cc101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.919+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 shutdown_connections 2024-02-14T14:53:48.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:47.919+0000 7f36d15fa700 1 -- 172.21.15.17:0/913716822 wait complete. 2024-02-14T14:53:48.588 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T14:53:48.588 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2024-02-14T14:53:48.596 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T14:53:48.596 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2024-02-14T14:53:48.596 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2024-02-14T14:53:48.596 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-14T14:53:48.598 INFO:teuthology.orchestra.run.smithi017.stdout:Ceph Dashboard is now available at: 2024-02-14T14:53:48.598 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:48.598 INFO:teuthology.orchestra.run.smithi017.stdout: URL: https://smithi017.front.sepia.ceph.com:8443/ 2024-02-14T14:53:48.598 INFO:teuthology.orchestra.run.smithi017.stdout: User: admin 2024-02-14T14:53:48.598 INFO:teuthology.orchestra.run.smithi017.stdout: Password: k0zvx1ilz0 2024-02-14T14:53:48.599 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:48.599 INFO:teuthology.orchestra.run.smithi017.stdout:Saving cluster configuration to /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config directory 2024-02-14T14:53:48.599 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling autotune for osd_memory_target 2024-02-14T14:53:49.762 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.055+0000 7fa90753b700 1 Processor -- start 2024-02-14T14:53:49.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.056+0000 7fa90753b700 1 -- start start 2024-02-14T14:53:49.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.056+0000 7fa90753b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa900107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:49.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.056+0000 7fa90753b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9001080a0 con 0x7fa9001076b0 2024-02-14T14:53:49.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.056+0000 7fa9052d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa900107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:49.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.056+0000 7fa9052d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa900107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48114/0 (socket says 172.21.15.17:48114) 2024-02-14T14:53:49.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.056+0000 7fa9052d7700 1 -- 172.21.15.17:0/2256096552 learned_addr learned my addr 172.21.15.17:0/2256096552 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:49.763 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.056+0000 7fa9052d7700 1 -- 172.21.15.17:0/2256096552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900108860 con 0x7fa9001076b0 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa9052d7700 1 --2- 172.21.15.17:0/2256096552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa900107ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098a0 tx=0x7fa8fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cac9af67d58fcf server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa8f7fff700 1 -- 172.21.15.17:0/2256096552 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8fc004160 con 0x7fa9001076b0 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa8f7fff700 1 -- 172.21.15.17:0/2256096552 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc0042c0 con 0x7fa9001076b0 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa8f7fff700 1 -- 172.21.15.17:0/2256096552 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8fc011a80 con 0x7fa9001076b0 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa90753b700 1 -- 172.21.15.17:0/2256096552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 msgr2=0x7fa900107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa90753b700 1 --2- 172.21.15.17:0/2256096552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa900107ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098a0 tx=0x7fa8fc009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa90753b700 1 -- 172.21.15.17:0/2256096552 shutdown_connections 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa90753b700 1 --2- 172.21.15.17:0/2256096552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa900107ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:49.764 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa90753b700 1 -- 172.21.15.17:0/2256096552 >> 172.21.15.17:0/2256096552 conn(0x7fa900102d50 msgr2=0x7fa900105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa90753b700 1 -- 172.21.15.17:0/2256096552 shutdown_connections 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.057+0000 7fa90753b700 1 -- 172.21.15.17:0/2256096552 wait complete. 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.058+0000 7fa90753b700 1 Processor -- start 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.058+0000 7fa90753b700 1 -- start start 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.058+0000 7fa90753b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa90019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.058+0000 7fa90753b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90019b9c0 con 0x7fa9001076b0 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.058+0000 7fa9052d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa90019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.058+0000 7fa9052d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa90019b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48124/0 (socket says 172.21.15.17:48124) 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.058+0000 7fa9052d7700 1 -- 172.21.15.17:0/1739993415 learned_addr learned my addr 172.21.15.17:0/1739993415 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.059+0000 7fa9052d7700 1 -- 172.21.15.17:0/1739993415 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc009550 con 0x7fa9001076b0 2024-02-14T14:53:49.765 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.059+0000 7fa9052d7700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa90019b480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc011740 tx=0x7fa8fc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.059+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8fc0039e0 con 0x7fa9001076b0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.059+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc003b40 con 0x7fa9001076b0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.059+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90019bbc0 con 0x7fa9001076b0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.059+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8fc024740 con 0x7fa9001076b0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.059+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90019c060 con 0x7fa9001076b0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.060+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fa8fc02d3f0 con 0x7fa9001076b0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.060+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90010ba90 con 0x7fa9001076b0 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.060+0000 7fa8f67fc700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8ec038f90 0x7fa8ec03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:49.766 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.060+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa8fc04d3c0 con 0x7fa9001076b0 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.060+0000 7fa904ad6700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8ec038f90 0x7fa8ec03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.060+0000 7fa904ad6700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8ec038f90 0x7fa8ec03b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0006fd0 tx=0x7fa8f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.065+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8fc010ca0 con 0x7fa9001076b0 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.195+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fa8fc0194b0 con 0x7fa9001076b0 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.266+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa90010bca0 con 0x7fa9001076b0 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.267+0000 7fa8f67fc700 1 -- 172.21.15.17:0/1739993415 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa8fc010ca0 con 0x7fa9001076b0 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8ec038f90 msgr2=0x7fa8ec03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8ec038f90 0x7fa8ec03b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0006fd0 tx=0x7fa8f0006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 msgr2=0x7fa90019b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa90019b480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc011740 tx=0x7fa8fc011b00 comp rx=0 tx=0).stop 2024-02-14T14:53:49.767 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 shutdown_connections 2024-02-14T14:53:49.768 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8ec038f90 0x7fa8ec03b450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:49.768 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 --2- 172.21.15.17:0/1739993415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9001076b0 0x7fa90019b480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:49.768 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 >> 172.21.15.17:0/1739993415 conn(0x7fa900102d50 msgr2=0x7fa900103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:49.768 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 shutdown_connections 2024-02-14T14:53:49.768 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:49.269+0000 7fa90753b700 1 -- 172.21.15.17:0/1739993415 wait complete. 2024-02-14T14:53:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:50 smithi017 ceph-mon[32549]: mgrmap e13: x(active, since 6s) 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.140+0000 7f2d4a6b4700 1 Processor -- start 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.140+0000 7f2d4a6b4700 1 -- start start 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.140+0000 7f2d4a6b4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.140+0000 7f2d4a6b4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4407ae30 con 0x7f2d441081a0 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.140+0000 7f2d43fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.141+0000 7f2d43fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48132/0 (socket says 172.21.15.17:48132) 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.141+0000 7f2d43fff700 1 -- 172.21.15.17:0/2154693018 learned_addr learned my addr 172.21.15.17:0/2154693018 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.141+0000 7f2d43fff700 1 -- 172.21.15.17:0/2154693018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4410aad0 con 0x7f2d441081a0 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.141+0000 7f2d43fff700 1 --2- 172.21.15.17:0/2154693018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4410a590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d340098a0 tx=0x7f2d34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291525e5b2821247 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.141+0000 7f2d42ffd700 1 -- 172.21.15.17:0/2154693018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d34004160 con 0x7f2d441081a0 2024-02-14T14:53:51.017 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.141+0000 7f2d42ffd700 1 -- 172.21.15.17:0/2154693018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d340042c0 con 0x7f2d441081a0 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.141+0000 7f2d42ffd700 1 -- 172.21.15.17:0/2154693018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d34011a80 con 0x7f2d441081a0 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/2154693018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 msgr2=0x7f2d4410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 --2- 172.21.15.17:0/2154693018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4410a590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d340098a0 tx=0x7f2d34009bb0 comp rx=0 tx=0).stop 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/2154693018 shutdown_connections 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 --2- 172.21.15.17:0/2154693018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4410a590 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/2154693018 >> 172.21.15.17:0/2154693018 conn(0x7f2d44101bf0 msgr2=0x7f2d44104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/2154693018 shutdown_connections 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/2154693018 wait complete. 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 Processor -- start 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.142+0000 7f2d4a6b4700 1 -- start start 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d4a6b4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d4a6b4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4419dc00 con 0x7f2d441081a0 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d43fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:51.018 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d43fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48142/0 (socket says 172.21.15.17:48142) 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d43fff700 1 -- 172.21.15.17:0/1783404774 learned_addr learned my addr 172.21.15.17:0/1783404774 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d43fff700 1 -- 172.21.15.17:0/1783404774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d34009550 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d43fff700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4419d6c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009130 tx=0x7f2d34011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d417fa700 1 -- 172.21.15.17:0/1783404774 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d34003aa0 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d417fa700 1 -- 172.21.15.17:0/1783404774 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d34003c00 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d417fa700 1 -- 172.21.15.17:0/1783404774 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d340248a0 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4419de00 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.143+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4419e2a0 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.144+0000 7f2d417fa700 1 -- 172.21.15.17:0/1783404774 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f2d34024a60 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.144+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4410b8c0 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.144+0000 7f2d417fa700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d2c038fd0 0x7f2d2c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.144+0000 7f2d417fa700 1 -- 172.21.15.17:0/1783404774 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2d3404d310 con 0x7f2d441081a0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.145+0000 7f2d437fe700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d2c038fd0 0x7f2d2c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.145+0000 7f2d437fe700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d2c038fd0 0x7f2d2c03b490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006fd0 tx=0x7f2d38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:51.019 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.149+0000 7f2d417fa700 1 -- 172.21.15.17:0/1783404774 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d3402dc90 con 0x7f2d441081a0 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.471+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f2d441a0b30 con 0x7f2d441081a0 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.473+0000 7f2d417fa700 1 -- 172.21.15.17:0/1783404774 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f2d34018ba0 con 0x7f2d441081a0 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d2c038fd0 msgr2=0x7f2d2c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d2c038fd0 0x7f2d2c03b490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006fd0 tx=0x7f2d38006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 msgr2=0x7f2d4419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4419d6c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009130 tx=0x7f2d34011b00 comp rx=0 tx=0).stop 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 shutdown_connections 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d2c038fd0 0x7f2d2c03b490 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 --2- 172.21.15.17:0/1783404774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d441081a0 0x7f2d4419d6c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 >> 172.21.15.17:0/1783404774 conn(0x7f2d44101bf0 msgr2=0x7f2d441026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 shutdown_connections 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2024-02-14T14:53:50.475+0000 7f2d4a6b4700 1 -- 172.21.15.17:0/1783404774 wait complete. 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-14T14:53:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout:Or, if you are only running a single cluster on this host: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: ceph telemetry on 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout:For more information see: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:53:51.021 INFO:teuthology.orchestra.run.smithi017.stdout:Bootstrap complete. 2024-02-14T14:53:51.056 INFO:tasks.cephadm:Fetching config... 2024-02-14T14:53:51.057 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:53:51.057 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-14T14:53:51.072 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-14T14:53:51.072 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:53:51.072 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-14T14:53:51.127 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-14T14:53:51.127 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:53:51.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-14T14:53:51.195 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-14T14:53:51.195 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:53:51.195 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-14T14:53:51.250 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-14T14:53:51.250 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoIoh4vAnlcLiBiiCWlGRa9E9wGk51HmfAoHvFQjZjg8XNwd6X+J9mllrxoNwI5Raomo6ya6hK1aWJ1dsJM321QiKrEFU0cQg8u2k1d3QKlWRteb6kzX2ASm7Ew9rGygno1lO9mfjzqYfWChlYHt6PjKhMAK8mALhaQQdJ0S6lkmnVpkABKIu3imMz50gzugCPtMaBDAEzgz5GFawJC2W8aGKMbNMGOx7CINVMmDfh3ONqg6Ry+nRNnWkt77wtSfMljNyMvP7uZCH71mOFROMIm9ZX/0HPmxtO1UdeXkoHeoljXqqwAPztdgeJnETZzyp5ewUGrHU+udtPTOT3VgteHzHWDFeO74eaBQW1u+2tArqctbsFwD8LhhedTJhL3F30mgwS1sCo1n+JMGjvX0Dhs7GMyq4N3VwuuX/VsaDHsQLn2NvXCz125YKvmidtYNhE0FLQPW5aMEuHW8YAFT4UiBDEVx2hlhrWyBM/lBWjjNh2DtSAwOwmbARjNyU+z78= ceph-867ff6ea-cb48-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T14:53:51.338 INFO:teuthology.orchestra.run.smithi017.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoIoh4vAnlcLiBiiCWlGRa9E9wGk51HmfAoHvFQjZjg8XNwd6X+J9mllrxoNwI5Raomo6ya6hK1aWJ1dsJM321QiKrEFU0cQg8u2k1d3QKlWRteb6kzX2ASm7Ew9rGygno1lO9mfjzqYfWChlYHt6PjKhMAK8mALhaQQdJ0S6lkmnVpkABKIu3imMz50gzugCPtMaBDAEzgz5GFawJC2W8aGKMbNMGOx7CINVMmDfh3ONqg6Ry+nRNnWkt77wtSfMljNyMvP7uZCH71mOFROMIm9ZX/0HPmxtO1UdeXkoHeoljXqqwAPztdgeJnETZzyp5ewUGrHU+udtPTOT3VgteHzHWDFeO74eaBQW1u+2tArqctbsFwD8LhhedTJhL3F30mgwS1sCo1n+JMGjvX0Dhs7GMyq4N3VwuuX/VsaDHsQLn2NvXCz125YKvmidtYNhE0FLQPW5aMEuHW8YAFT4UiBDEVx2hlhrWyBM/lBWjjNh2DtSAwOwmbARjNyU+z78= ceph-867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:53:51.353 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoIoh4vAnlcLiBiiCWlGRa9E9wGk51HmfAoHvFQjZjg8XNwd6X+J9mllrxoNwI5Raomo6ya6hK1aWJ1dsJM321QiKrEFU0cQg8u2k1d3QKlWRteb6kzX2ASm7Ew9rGygno1lO9mfjzqYfWChlYHt6PjKhMAK8mALhaQQdJ0S6lkmnVpkABKIu3imMz50gzugCPtMaBDAEzgz5GFawJC2W8aGKMbNMGOx7CINVMmDfh3ONqg6Ry+nRNnWkt77wtSfMljNyMvP7uZCH71mOFROMIm9ZX/0HPmxtO1UdeXkoHeoljXqqwAPztdgeJnETZzyp5ewUGrHU+udtPTOT3VgteHzHWDFeO74eaBQW1u+2tArqctbsFwD8LhhedTJhL3F30mgwS1sCo1n+JMGjvX0Dhs7GMyq4N3VwuuX/VsaDHsQLn2NvXCz125YKvmidtYNhE0FLQPW5aMEuHW8YAFT4UiBDEVx2hlhrWyBM/lBWjjNh2DtSAwOwmbARjNyU+z78= ceph-867ff6ea-cb48-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T14:53:51.406 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoIoh4vAnlcLiBiiCWlGRa9E9wGk51HmfAoHvFQjZjg8XNwd6X+J9mllrxoNwI5Raomo6ya6hK1aWJ1dsJM321QiKrEFU0cQg8u2k1d3QKlWRteb6kzX2ASm7Ew9rGygno1lO9mfjzqYfWChlYHt6PjKhMAK8mALhaQQdJ0S6lkmnVpkABKIu3imMz50gzugCPtMaBDAEzgz5GFawJC2W8aGKMbNMGOx7CINVMmDfh3ONqg6Ry+nRNnWkt77wtSfMljNyMvP7uZCH71mOFROMIm9ZX/0HPmxtO1UdeXkoHeoljXqqwAPztdgeJnETZzyp5ewUGrHU+udtPTOT3VgteHzHWDFeO74eaBQW1u+2tArqctbsFwD8LhhedTJhL3F30mgwS1sCo1n+JMGjvX0Dhs7GMyq4N3VwuuX/VsaDHsQLn2NvXCz125YKvmidtYNhE0FLQPW5aMEuHW8YAFT4UiBDEVx2hlhrWyBM/lBWjjNh2DtSAwOwmbARjNyU+z78= ceph-867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:53:51.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoIoh4vAnlcLiBiiCWlGRa9E9wGk51HmfAoHvFQjZjg8XNwd6X+J9mllrxoNwI5Raomo6ya6hK1aWJ1dsJM321QiKrEFU0cQg8u2k1d3QKlWRteb6kzX2ASm7Ew9rGygno1lO9mfjzqYfWChlYHt6PjKhMAK8mALhaQQdJ0S6lkmnVpkABKIu3imMz50gzugCPtMaBDAEzgz5GFawJC2W8aGKMbNMGOx7CINVMmDfh3ONqg6Ry+nRNnWkt77wtSfMljNyMvP7uZCH71mOFROMIm9ZX/0HPmxtO1UdeXkoHeoljXqqwAPztdgeJnETZzyp5ewUGrHU+udtPTOT3VgteHzHWDFeO74eaBQW1u+2tArqctbsFwD8LhhedTJhL3F30mgwS1sCo1n+JMGjvX0Dhs7GMyq4N3VwuuX/VsaDHsQLn2NvXCz125YKvmidtYNhE0FLQPW5aMEuHW8YAFT4UiBDEVx2hlhrWyBM/lBWjjNh2DtSAwOwmbARjNyU+z78= ceph-867ff6ea-cb48-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T14:53:51.468 INFO:teuthology.orchestra.run.smithi076.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoIoh4vAnlcLiBiiCWlGRa9E9wGk51HmfAoHvFQjZjg8XNwd6X+J9mllrxoNwI5Raomo6ya6hK1aWJ1dsJM321QiKrEFU0cQg8u2k1d3QKlWRteb6kzX2ASm7Ew9rGygno1lO9mfjzqYfWChlYHt6PjKhMAK8mALhaQQdJ0S6lkmnVpkABKIu3imMz50gzugCPtMaBDAEzgz5GFawJC2W8aGKMbNMGOx7CINVMmDfh3ONqg6Ry+nRNnWkt77wtSfMljNyMvP7uZCH71mOFROMIm9ZX/0HPmxtO1UdeXkoHeoljXqqwAPztdgeJnETZzyp5ewUGrHU+udtPTOT3VgteHzHWDFeO74eaBQW1u+2tArqctbsFwD8LhhedTJhL3F30mgwS1sCo1n+JMGjvX0Dhs7GMyq4N3VwuuX/VsaDHsQLn2NvXCz125YKvmidtYNhE0FLQPW5aMEuHW8YAFT4UiBDEVx2hlhrWyBM/lBWjjNh2DtSAwOwmbARjNyU+z78= ceph-867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:53:51.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-14T14:53:51.720 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:53:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:51 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1783404774' entity='client.admin' 2024-02-14T14:53:54.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.191+0000 7f244d785700 1 -- 172.21.15.17:0/2749649801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 msgr2=0x7f2448100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:54.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.191+0000 7f244d785700 1 --2- 172.21.15.17:0/2749649801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448100a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2438009910 tx=0x7f2438009c20 comp rx=0 tx=0).stop 2024-02-14T14:53:54.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.191+0000 7f244d785700 1 -- 172.21.15.17:0/2749649801 shutdown_connections 2024-02-14T14:53:54.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.191+0000 7f244d785700 1 --2- 172.21.15.17:0/2749649801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448100a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.191+0000 7f244d785700 1 -- 172.21.15.17:0/2749649801 >> 172.21.15.17:0/2749649801 conn(0x7f24480fbbf0 msgr2=0x7f24480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.192+0000 7f244d785700 1 -- 172.21.15.17:0/2749649801 shutdown_connections 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.192+0000 7f244d785700 1 -- 172.21.15.17:0/2749649801 wait complete. 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.192+0000 7f244d785700 1 Processor -- start 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.192+0000 7f244d785700 1 -- start start 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.192+0000 7f244d785700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.192+0000 7f244d785700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2448199950 con 0x7f2448100650 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f2446ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f2446ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32930/0 (socket says 172.21.15.17:32930) 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f2446ffd700 1 -- 172.21.15.17:0/3738447190 learned_addr learned my addr 172.21.15.17:0/3738447190 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f2446ffd700 1 -- 172.21.15.17:0/3738447190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24380095f0 con 0x7f2448100650 2024-02-14T14:53:54.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f2446ffd700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448069bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2438004f40 tx=0x7f2438005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f242ffff700 1 -- 172.21.15.17:0/3738447190 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f243801d070 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f242ffff700 1 -- 172.21.15.17:0/3738447190 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24380044a0 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2448199a90 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f242ffff700 1 -- 172.21.15.17:0/3738447190 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2438003a40 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.193+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2448199e70 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.194+0000 7f242ffff700 1 -- 172.21.15.17:0/3738447190 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f2438003d00 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.194+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244818f4b0 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.194+0000 7f242ffff700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2430038fe0 0x7f243003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.194+0000 7f242ffff700 1 -- 172.21.15.17:0/3738447190 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f243804cf50 con 0x7f2448100650 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.194+0000 7f24467fc700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2430038fe0 0x7f243003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:54.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.195+0000 7f24467fc700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2430038fe0 0x7f243003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f243c006fd0 tx=0x7f243c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.199+0000 7f242ffff700 1 -- 172.21.15.17:0/3738447190 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2438029950 con 0x7f2448100650 2024-02-14T14:53:54.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.403+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f2448063350 con 0x7f2448100650 2024-02-14T14:53:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.406+0000 7f242ffff700 1 -- 172.21.15.17:0/3738447190 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2438026030 con 0x7f2448100650 2024-02-14T14:53:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2430038fe0 msgr2=0x7f243003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2430038fe0 0x7f243003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f243c006fd0 tx=0x7f243c006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 msgr2=0x7f2448069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448069bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2438004f40 tx=0x7f2438005020 comp rx=0 tx=0).stop 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 shutdown_connections 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2430038fe0 0x7f243003b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 --2- 172.21.15.17:0/3738447190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2448100650 0x7f2448069bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 >> 172.21.15.17:0/3738447190 conn(0x7f24480fbbf0 msgr2=0x7f24480fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 shutdown_connections 2024-02-14T14:53:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:54.408+0000 7f244d785700 1 -- 172.21.15.17:0/3738447190 wait complete. 2024-02-14T14:53:55.068 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-14T14:53:55.069 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-14T14:53:55.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:55 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3738447190' entity='client.admin' 2024-02-14T14:53:55.648 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:53:56.806 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:56.806 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:56.806 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi017", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:53:56.806 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:56.806 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:56.806 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:57.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.011+0000 7f7244fbd700 1 -- 172.21.15.17:0/2706509529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 msgr2=0x7f7240101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:57.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.011+0000 7f7244fbd700 1 --2- 172.21.15.17:0/2706509529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240101b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f723800b0a0 tx=0x7f723800b3b0 comp rx=0 tx=0).stop 2024-02-14T14:53:57.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.011+0000 7f7244fbd700 1 -- 172.21.15.17:0/2706509529 shutdown_connections 2024-02-14T14:53:57.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.011+0000 7f7244fbd700 1 --2- 172.21.15.17:0/2706509529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240101b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:57.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.011+0000 7f7244fbd700 1 -- 172.21.15.17:0/2706509529 >> 172.21.15.17:0/2706509529 conn(0x7f72400fcd10 msgr2=0x7f72400ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:57.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.011+0000 7f7244fbd700 1 -- 172.21.15.17:0/2706509529 shutdown_connections 2024-02-14T14:53:57.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.011+0000 7f7244fbd700 1 -- 172.21.15.17:0/2706509529 wait complete. 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f7244fbd700 1 Processor -- start 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f7244fbd700 1 -- start start 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f7244fbd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f7244fbd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7240199400 con 0x7f7240101770 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f723e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f723e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32944/0 (socket says 172.21.15.17:32944) 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f723e59c700 1 -- 172.21.15.17:0/2374039126 learned_addr learned my addr 172.21.15.17:0/2374039126 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f723e59c700 1 -- 172.21.15.17:0/2374039126 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7238009d00 con 0x7f7240101770 2024-02-14T14:53:57.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f723e59c700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240198ec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72380093e0 tx=0x7f723800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.012+0000 7f72377fe700 1 -- 172.21.15.17:0/2374039126 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7238021070 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.013+0000 7f72377fe700 1 -- 172.21.15.17:0/2374039126 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7238003c10 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.013+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7240199600 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.013+0000 7f72377fe700 1 -- 172.21.15.17:0/2374039126 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f723800e040 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.013+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7240199aa0 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.013+0000 7f72377fe700 1 -- 172.21.15.17:0/2374039126 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f7238019070 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.014+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72401929d0 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.014+0000 7f72377fe700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f722c038fd0 0x7f722c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.014+0000 7f72377fe700 1 -- 172.21.15.17:0/2374039126 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f723804cbb0 con 0x7f7240101770 2024-02-14T14:53:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.014+0000 7f723dd9b700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f722c038fd0 0x7f722c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.014+0000 7f723dd9b700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f722c038fd0 0x7f722c03b490 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7228006fd0 tx=0x7f7228006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.019+0000 7f72377fe700 1 -- 172.21.15.17:0/2374039126 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7238025550 con 0x7f7240101770 2024-02-14T14:53:57.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.226+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f724002d7e0 con 0x7f722c038fd0 2024-02-14T14:53:57.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.230+0000 7f72377fe700 1 -- 172.21.15.17:0/2374039126 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f724002d7e0 con 0x7f722c038fd0 2024-02-14T14:53:57.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f722c038fd0 msgr2=0x7f722c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f722c038fd0 0x7f722c03b490 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7228006fd0 tx=0x7f7228006e40 comp rx=0 tx=0).stop 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 msgr2=0x7f7240198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240198ec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72380093e0 tx=0x7f723800bd40 comp rx=0 tx=0).stop 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 shutdown_connections 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f722c038fd0 0x7f722c03b490 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 --2- 172.21.15.17:0/2374039126 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7240101770 0x7f7240198ec0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 >> 172.21.15.17:0/2374039126 conn(0x7f72400fcd10 msgr2=0x7f72400fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 shutdown_connections 2024-02-14T14:53:57.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:57.232+0000 7f7244fbd700 1 -- 172.21.15.17:0/2374039126 wait complete. 2024-02-14T14:53:57.947 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi050 2024-02-14T14:53:57.947 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:53:57.947 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.conf 2024-02-14T14:53:57.967 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:53:57.967 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:53:58.026 INFO:tasks.cephadm:Adding host smithi050 to orchestrator... 2024-02-14T14:53:58.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch host add smithi050 2024-02-14T14:53:58.269 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:53:58.495 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:53:58.495 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:58.495 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:53:58.495 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:53:58.495 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:53:58.495 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: Updating smithi017:/etc/ceph/ceph.conf 2024-02-14T14:53:58.495 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:53:58.496 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:58.496 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:58.496 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:53:59.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:59 smithi017 ceph-mon[32549]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:53:59.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:53:59 smithi017 ceph-mon[32549]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.652+0000 7fded1ca4700 1 -- 172.21.15.17:0/658029352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 msgr2=0x7fdecc102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.652+0000 7fded1ca4700 1 --2- 172.21.15.17:0/658029352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc102cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009910 tx=0x7fdebc009c20 comp rx=0 tx=0).stop 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.652+0000 7fded1ca4700 1 -- 172.21.15.17:0/658029352 shutdown_connections 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.652+0000 7fded1ca4700 1 --2- 172.21.15.17:0/658029352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc102cf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.652+0000 7fded1ca4700 1 -- 172.21.15.17:0/658029352 >> 172.21.15.17:0/658029352 conn(0x7fdecc077260 msgr2=0x7fdecc0fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.652+0000 7fded1ca4700 1 -- 172.21.15.17:0/658029352 shutdown_connections 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.652+0000 7fded1ca4700 1 -- 172.21.15.17:0/658029352 wait complete. 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.653+0000 7fded1ca4700 1 Processor -- start 2024-02-14T14:53:59.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.653+0000 7fded1ca4700 1 -- start start 2024-02-14T14:53:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.653+0000 7fded1ca4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.653+0000 7fded1ca4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc197580 con 0x7fdecc102910 2024-02-14T14:53:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.653+0000 7fdecb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.653+0000 7fdecb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32956/0 (socket says 172.21.15.17:32956) 2024-02-14T14:53:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.653+0000 7fdecb7fe700 1 -- 172.21.15.17:0/4194624893 learned_addr learned my addr 172.21.15.17:0/4194624893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.654+0000 7fdecb7fe700 1 -- 172.21.15.17:0/4194624893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0095f0 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.654+0000 7fdecb7fe700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc197040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005850 tx=0x7fdebc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.654+0000 7fdec8ff9700 1 -- 172.21.15.17:0/4194624893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdebc01d070 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.654+0000 7fdec8ff9700 1 -- 172.21.15.17:0/4194624893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdebc00bc30 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.654+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc197780 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.654+0000 7fdec8ff9700 1 -- 172.21.15.17:0/4194624893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdebc003a20 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.654+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc197ba0 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.655+0000 7fdec8ff9700 1 -- 172.21.15.17:0/4194624893 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fdebc003c90 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.655+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc063350 con 0x7fdecc102910 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.655+0000 7fdec8ff9700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdeb8038f80 0x7fdeb803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:53:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.655+0000 7fdec8ff9700 1 -- 172.21.15.17:0/4194624893 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdebc04ce90 con 0x7fdecc102910 2024-02-14T14:53:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.655+0000 7fdecaffd700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdeb8038f80 0x7fdeb803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:53:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.656+0000 7fdecaffd700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdeb8038f80 0x7fdeb803b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4006fd0 tx=0x7fdeb4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:53:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.660+0000 7fdec8ff9700 1 -- 172.21.15.17:0/4194624893 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdebc00b930 con 0x7fdecc102910 2024-02-14T14:53:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:53:59.867+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}) v1 -- 0x7fdecc10bc20 con 0x7fdeb8038f80 2024-02-14T14:54:01.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:01 smithi017 ceph-mon[32549]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:02.755 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:02 smithi017 ceph-mon[32549]: Deploying cephadm binary to smithi050 2024-02-14T14:54:04.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:03 smithi017 ceph-mon[32549]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:04.255 INFO:teuthology.orchestra.run.smithi017.stdout:Added host 'smithi050' with addr '172.21.15.50' 2024-02-14T14:54:04.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.254+0000 7fdec8ff9700 1 -- 172.21.15.17:0/4194624893 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fdecc10bc20 con 0x7fdeb8038f80 2024-02-14T14:54:04.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdeb8038f80 msgr2=0x7fdeb803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdeb8038f80 0x7fdeb803b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4006fd0 tx=0x7fdeb4006e40 comp rx=0 tx=0).stop 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 msgr2=0x7fdecc197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc197040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005850 tx=0x7fdebc004ff0 comp rx=0 tx=0).stop 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 shutdown_connections 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdeb8038f80 0x7fdeb803b440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 --2- 172.21.15.17:0/4194624893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc102910 0x7fdecc197040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.256+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 >> 172.21.15.17:0/4194624893 conn(0x7fdecc077260 msgr2=0x7fdecc1036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.257+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 shutdown_connections 2024-02-14T14:54:04.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:04.257+0000 7fded1ca4700 1 -- 172.21.15.17:0/4194624893 wait complete. 2024-02-14T14:54:04.837 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-14T14:54:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:54:05.474 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:05 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:05.474 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:05 smithi017 ceph-mon[32549]: Added host smithi050 2024-02-14T14:54:05.474 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:05 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:05.474 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:05 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:06.513 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:06 smithi017 ceph-mon[32549]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:06.514 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:06 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.549+0000 7fb1733be700 1 -- 172.21.15.17:0/3042066355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 msgr2=0x7fb16c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.549+0000 7fb1733be700 1 --2- 172.21.15.17:0/3042066355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c106c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb168009910 tx=0x7fb168009c20 comp rx=0 tx=0).stop 2024-02-14T14:54:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.549+0000 7fb1733be700 1 -- 172.21.15.17:0/3042066355 shutdown_connections 2024-02-14T14:54:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.549+0000 7fb1733be700 1 --2- 172.21.15.17:0/3042066355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c106c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.549+0000 7fb1733be700 1 -- 172.21.15.17:0/3042066355 >> 172.21.15.17:0/3042066355 conn(0x7fb16c0fb870 msgr2=0x7fb16c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.549+0000 7fb1733be700 1 -- 172.21.15.17:0/3042066355 shutdown_connections 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.549+0000 7fb1733be700 1 -- 172.21.15.17:0/3042066355 wait complete. 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb1733be700 1 Processor -- start 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb1733be700 1 -- start start 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb1733be700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb1733be700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c100be0 con 0x7fb16c1068b0 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb17115a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb17115a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57146/0 (socket says 172.21.15.17:57146) 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb17115a700 1 -- 172.21.15.17:0/89398375 learned_addr learned my addr 172.21.15.17:0/89398375 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb17115a700 1 -- 172.21.15.17:0/89398375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680095f0 con 0x7fb16c1068b0 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.550+0000 7fb17115a700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c198050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb1680092e0 tx=0x7fb168004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.551+0000 7fb1627fc700 1 -- 172.21.15.17:0/89398375 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb16801d070 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.551+0000 7fb1627fc700 1 -- 172.21.15.17:0/89398375 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb16800b750 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.551+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c198650 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.551+0000 7fb1627fc700 1 -- 172.21.15.17:0/89398375 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb168003930 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.551+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c198ad0 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.552+0000 7fb1627fc700 1 -- 172.21.15.17:0/89398375 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fb168003a90 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.552+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c063350 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.552+0000 7fb1627fc700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb158038fe0 0x7fb15803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.552+0000 7fb1627fc700 1 -- 172.21.15.17:0/89398375 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb16804cd20 con 0x7fb16c1068b0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.552+0000 7fb170959700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb158038fe0 0x7fb15803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.552+0000 7fb170959700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb158038fe0 0x7fb15803b4a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb15c006fd0 tx=0x7fb15c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:06.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.557+0000 7fb1627fc700 1 -- 172.21.15.17:0/89398375 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb16802ae20 con 0x7fb16c1068b0 2024-02-14T14:54:06.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.763+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb16c198d20 con 0x7fb158038fe0 2024-02-14T14:54:06.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.764+0000 7fb1627fc700 1 -- 172.21.15.17:0/89398375 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fb16c198d20 con 0x7fb158038fe0 2024-02-14T14:54:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:54:06.766 INFO:teuthology.orchestra.run.smithi017.stdout:[{"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}] 2024-02-14T14:54:06.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb158038fe0 msgr2=0x7fb15803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb158038fe0 0x7fb15803b4a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb15c006fd0 tx=0x7fb15c006e40 comp rx=0 tx=0).stop 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 msgr2=0x7fb16c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c198050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb1680092e0 tx=0x7fb168004ff0 comp rx=0 tx=0).stop 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 shutdown_connections 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb158038fe0 0x7fb15803b4a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 --2- 172.21.15.17:0/89398375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb16c1068b0 0x7fb16c198050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 >> 172.21.15.17:0/89398375 conn(0x7fb16c0fb870 msgr2=0x7fb16c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 shutdown_connections 2024-02-14T14:54:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:06.767+0000 7fb1733be700 1 -- 172.21.15.17:0/89398375 wait complete. 2024-02-14T14:54:07.421 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi076 2024-02-14T14:54:07.421 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:54:07.422 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.conf 2024-02-14T14:54:07.440 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:54:07.440 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:07.498 INFO:tasks.cephadm:Adding host smithi076 to orchestrator... 2024-02-14T14:54:07.498 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch host add smithi076 2024-02-14T14:54:07.738 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:54:08.153 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:07 smithi017 ceph-mon[32549]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:08.153 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:07 smithi017 ceph-mon[32549]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T14:54:08.153 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:07 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.205+0000 7efcc2086700 1 -- 172.21.15.17:0/3891531360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 msgr2=0x7efcbc102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.205+0000 7efcc2086700 1 --2- 172.21.15.17:0/3891531360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc102ce0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efcac009910 tx=0x7efcac009c20 comp rx=0 tx=0).stop 2024-02-14T14:54:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.205+0000 7efcc2086700 1 -- 172.21.15.17:0/3891531360 shutdown_connections 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.205+0000 7efcc2086700 1 --2- 172.21.15.17:0/3891531360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc102ce0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.205+0000 7efcc2086700 1 -- 172.21.15.17:0/3891531360 >> 172.21.15.17:0/3891531360 conn(0x7efcbc0fc0d0 msgr2=0x7efcbc0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.205+0000 7efcc2086700 1 -- 172.21.15.17:0/3891531360 shutdown_connections 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.205+0000 7efcc2086700 1 -- 172.21.15.17:0/3891531360 wait complete. 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcc2086700 1 Processor -- start 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcc2086700 1 -- start start 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcc2086700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcc2086700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc197630 con 0x7efcbc102900 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57170/0 (socket says 172.21.15.17:57170) 2024-02-14T14:54:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcbb7fe700 1 -- 172.21.15.17:0/3947203528 learned_addr learned my addr 172.21.15.17:0/3947203528 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.206+0000 7efcbb7fe700 1 -- 172.21.15.17:0/3947203528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0095f0 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.207+0000 7efcbb7fe700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc1970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcac006010 tx=0x7efcac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.207+0000 7efcb8ff9700 1 -- 172.21.15.17:0/3947203528 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcac01d070 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.207+0000 7efcb8ff9700 1 -- 172.21.15.17:0/3947203528 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcac00bc30 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.207+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc197830 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.207+0000 7efcb8ff9700 1 -- 172.21.15.17:0/3947203528 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcac0039a0 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.207+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc197c50 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.208+0000 7efcb8ff9700 1 -- 172.21.15.17:0/3947203528 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7efcac003c20 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.208+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc06a220 con 0x7efcbc102900 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.208+0000 7efcb8ff9700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efca4038f80 0x7efca403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.208+0000 7efcb8ff9700 1 -- 172.21.15.17:0/3947203528 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efcac04cff0 con 0x7efcbc102900 2024-02-14T14:54:09.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.208+0000 7efcbaffd700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efca4038f80 0x7efca403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:09.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.208+0000 7efcbaffd700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efca4038f80 0x7efca403b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efcb0006fd0 tx=0x7efcb0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:09.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.213+0000 7efcb8ff9700 1 -- 172.21.15.17:0/3947203528 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efcac00fd00 con 0x7efcbc102900 2024-02-14T14:54:09.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:09.421+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}) v1 -- 0x7efcbc10bbe0 con 0x7efca4038f80 2024-02-14T14:54:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:09 smithi017 ceph-mon[32549]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:09 smithi017 ceph-mon[32549]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:54:11.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:11.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:11.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-14T14:54:11.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: Deploying cephadm binary to smithi076 2024-02-14T14:54:11.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:10 smithi017 ceph-mon[32549]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:12.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:12 smithi017 ceph-mon[32549]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:12.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:12 smithi017 ceph-mon[32549]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:13.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.778+0000 7efcb8ff9700 1 -- 172.21.15.17:0/3947203528 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7efcbc10bbe0 con 0x7efca4038f80 2024-02-14T14:54:13.779 INFO:teuthology.orchestra.run.smithi017.stdout:Added host 'smithi076' with addr '172.21.15.76' 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efca4038f80 msgr2=0x7efca403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efca4038f80 0x7efca403b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efcb0006fd0 tx=0x7efcb0006e40 comp rx=0 tx=0).stop 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 msgr2=0x7efcbc1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc1970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcac006010 tx=0x7efcac004ff0 comp rx=0 tx=0).stop 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 shutdown_connections 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efca4038f80 0x7efca403b440 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 --2- 172.21.15.17:0/3947203528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcbc102900 0x7efcbc1970f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 >> 172.21.15.17:0/3947203528 conn(0x7efcbc0fc0d0 msgr2=0x7efcbc1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 shutdown_connections 2024-02-14T14:54:13.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:13.780+0000 7efcc2086700 1 -- 172.21.15.17:0/3947203528 wait complete. 2024-02-14T14:54:14.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-14T14:54:14.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:14 smithi017 ceph-mon[32549]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:14.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:14 smithi017 ceph-mon[32549]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T14:54:14.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:14.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:14.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:14.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:14.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:14.559 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:54:15.375 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:15 smithi017 ceph-mon[32549]: Added host smithi076 2024-02-14T14:54:15.375 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:15.375 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.997+0000 7f41fa097700 1 -- 172.21.15.17:0/3170592629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 msgr2=0x7f41f4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.997+0000 7f41fa097700 1 --2- 172.21.15.17:0/3170592629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f4107ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009910 tx=0x7f41e4009c20 comp rx=0 tx=0).stop 2024-02-14T14:54:15.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.998+0000 7f41fa097700 1 -- 172.21.15.17:0/3170592629 shutdown_connections 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.998+0000 7f41fa097700 1 --2- 172.21.15.17:0/3170592629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f4107ae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.998+0000 7f41fa097700 1 -- 172.21.15.17:0/3170592629 >> 172.21.15.17:0/3170592629 conn(0x7f41f40fd230 msgr2=0x7f41f40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.998+0000 7f41fa097700 1 -- 172.21.15.17:0/3170592629 shutdown_connections 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.998+0000 7f41fa097700 1 -- 172.21.15.17:0/3170592629 wait complete. 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.998+0000 7f41fa097700 1 Processor -- start 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.998+0000 7f41fa097700 1 -- start start 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41fa097700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41fa097700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41975e0 con 0x7f41f4107700 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f41970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42598/0 (socket says 172.21.15.17:42598) 2024-02-14T14:54:16.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f37fe700 1 -- 172.21.15.17:0/4069604559 learned_addr learned my addr 172.21.15.17:0/4069604559 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f37fe700 1 -- 172.21.15.17:0/4069604559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e40095f0 con 0x7f41f4107700 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f37fe700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f41970a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006010 tx=0x7f41e4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f0ff9700 1 -- 172.21.15.17:0/4069604559 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41e401d070 con 0x7f41f4107700 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f0ff9700 1 -- 172.21.15.17:0/4069604559 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41e400bbd0 con 0x7f41f4107700 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f4197840 con 0x7f41f4107700 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:15.999+0000 7f41f0ff9700 1 -- 172.21.15.17:0/4069604559 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41e4003880 con 0x7f41f4107700 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.000+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f4197c00 con 0x7f41f4107700 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.000+0000 7f41f0ff9700 1 -- 172.21.15.17:0/4069604559 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f41e400b6f0 con 0x7f41f4107700 2024-02-14T14:54:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.001+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f4102e40 con 0x7f41f4107700 2024-02-14T14:54:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.001+0000 7f41f0ff9700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f41dc038f80 0x7f41dc03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.001+0000 7f41f0ff9700 1 -- 172.21.15.17:0/4069604559 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f41e404ce30 con 0x7f41f4107700 2024-02-14T14:54:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.001+0000 7f41f2ffd700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f41dc038f80 0x7f41dc03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.001+0000 7f41f2ffd700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f41dc038f80 0x7f41dc03b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006fd0 tx=0x7f41e8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.006+0000 7f41f0ff9700 1 -- 172.21.15.17:0/4069604559 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f41e4029540 con 0x7f41f4107700 2024-02-14T14:54:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.216+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41f410bbe0 con 0x7f41dc038f80 2024-02-14T14:54:16.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.218+0000 7f41f0ff9700 1 -- 172.21.15.17:0/4069604559 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f41f410bbe0 con 0x7f41dc038f80 2024-02-14T14:54:16.218 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:54:16.218 INFO:teuthology.orchestra.run.smithi017.stdout:[{"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}] 2024-02-14T14:54:16.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.219+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f41dc038f80 msgr2=0x7f41dc03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:16.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.219+0000 7f41fa097700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f41dc038f80 0x7f41dc03b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006fd0 tx=0x7f41e8006e40 comp rx=0 tx=0).stop 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.219+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 msgr2=0x7f41f41970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.220+0000 7f41fa097700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f41970a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006010 tx=0x7f41e4004ff0 comp rx=0 tx=0).stop 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.220+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 shutdown_connections 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.220+0000 7f41fa097700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f41dc038f80 0x7f41dc03b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.220+0000 7f41fa097700 1 --2- 172.21.15.17:0/4069604559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41f4107700 0x7f41f41970a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.220+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 >> 172.21.15.17:0/4069604559 conn(0x7f41f40fd230 msgr2=0x7f41f4109260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.220+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 shutdown_connections 2024-02-14T14:54:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:16.220+0000 7f41fa097700 1 -- 172.21.15.17:0/4069604559 wait complete. 2024-02-14T14:54:16.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:16 smithi017 ceph-mon[32549]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:16.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:16.945 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-14T14:54:16.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd crush tunables default 2024-02-14T14:54:17.186 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:54:17.559 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:17 smithi017 ceph-mon[32549]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T14:54:18.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:18 smithi017 ceph-mon[32549]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:18.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.760+0000 7f8fd9262700 1 -- 172.21.15.17:0/4173705546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 msgr2=0x7f8fd40744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:18.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.760+0000 7f8fd9262700 1 --2- 172.21.15.17:0/4173705546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd40744d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009910 tx=0x7f8fc4009c20 comp rx=0 tx=0).stop 2024-02-14T14:54:18.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.760+0000 7f8fd9262700 1 -- 172.21.15.17:0/4173705546 shutdown_connections 2024-02-14T14:54:18.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.760+0000 7f8fd9262700 1 --2- 172.21.15.17:0/4173705546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd40744d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:18.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.760+0000 7f8fd9262700 1 -- 172.21.15.17:0/4173705546 >> 172.21.15.17:0/4173705546 conn(0x7f8fd40fd230 msgr2=0x7f8fd40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:18.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.760+0000 7f8fd9262700 1 -- 172.21.15.17:0/4173705546 shutdown_connections 2024-02-14T14:54:18.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.760+0000 7f8fd9262700 1 -- 172.21.15.17:0/4173705546 wait complete. 2024-02-14T14:54:18.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.761+0000 7f8fd9262700 1 Processor -- start 2024-02-14T14:54:18.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.761+0000 7f8fd9262700 1 -- start start 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.761+0000 7f8fd9262700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.761+0000 7f8fd9262700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4199790 con 0x7f8fd40740f0 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.761+0000 7f8fd2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fd2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42624/0 (socket says 172.21.15.17:42624) 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fd2ffd700 1 -- 172.21.15.17:0/1967025556 learned_addr learned my addr 172.21.15.17:0/1967025556 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fd2ffd700 1 -- 172.21.15.17:0/1967025556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40095f0 con 0x7f8fd40740f0 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fd2ffd700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd4199250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005850 tx=0x7f8fc4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fbbfff700 1 -- 172.21.15.17:0/1967025556 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc401d070 con 0x7f8fd40740f0 2024-02-14T14:54:18.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fbbfff700 1 -- 172.21.15.17:0/1967025556 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc400bc30 con 0x7f8fd40740f0 2024-02-14T14:54:18.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd4199990 con 0x7f8fd40740f0 2024-02-14T14:54:18.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fbbfff700 1 -- 172.21.15.17:0/1967025556 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4003a20 con 0x7f8fd40740f0 2024-02-14T14:54:18.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.762+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd4199db0 con 0x7f8fd40740f0 2024-02-14T14:54:18.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.763+0000 7f8fbbfff700 1 -- 172.21.15.17:0/1967025556 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f8fc4003b80 con 0x7f8fd40740f0 2024-02-14T14:54:18.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.763+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd4063350 con 0x7f8fd40740f0 2024-02-14T14:54:18.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.763+0000 7f8fbbfff700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8fbc038fe0 0x7f8fbc03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:18.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.763+0000 7f8fbbfff700 1 -- 172.21.15.17:0/1967025556 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8fc404ced0 con 0x7f8fd40740f0 2024-02-14T14:54:18.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.763+0000 7f8fd27fc700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8fbc038fe0 0x7f8fbc03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:18.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.764+0000 7f8fd27fc700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8fbc038fe0 0x7f8fbc03b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8006fd0 tx=0x7f8fc8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:18.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.768+0000 7f8fbbfff700 1 -- 172.21.15.17:0/1967025556 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fc4029910 con 0x7f8fd40740f0 2024-02-14T14:54:18.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:18.974+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8fd410de00 con 0x7f8fd40740f0 2024-02-14T14:54:19.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.454+0000 7f8fbbfff700 1 -- 172.21.15.17:0/1967025556 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8fc4026070 con 0x7f8fd40740f0 2024-02-14T14:54:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.457+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8fbc038fe0 msgr2=0x7f8fbc03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.457+0000 7f8fd9262700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8fbc038fe0 0x7f8fbc03b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8006fd0 tx=0x7f8fc8006e40 comp rx=0 tx=0).stop 2024-02-14T14:54:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.457+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 msgr2=0x7f8fd4199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.457+0000 7f8fd9262700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd4199250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005850 tx=0x7f8fc4004ff0 comp rx=0 tx=0).stop 2024-02-14T14:54:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.457+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 shutdown_connections 2024-02-14T14:54:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.458+0000 7f8fd9262700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8fbc038fe0 0x7f8fbc03b4a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:19.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.458+0000 7f8fd9262700 1 --2- 172.21.15.17:0/1967025556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fd40740f0 0x7f8fd4199250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:19.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.458+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 >> 172.21.15.17:0/1967025556 conn(0x7f8fd40fd230 msgr2=0x7f8fd4105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:19.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.458+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 shutdown_connections 2024-02-14T14:54:19.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:19.458+0000 7f8fd9262700 1 -- 172.21.15.17:0/1967025556 wait complete. 2024-02-14T14:54:19.460 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2024-02-14T14:54:19.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:19 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1967025556' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T14:54:20.131 INFO:tasks.cephadm:Adding mon.a on smithi017 2024-02-14T14:54:20.131 INFO:tasks.cephadm:Adding mon.b on smithi050 2024-02-14T14:54:20.131 INFO:tasks.cephadm:Adding mon.c on smithi076 2024-02-14T14:54:20.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch apply mon '3;smithi017:172.21.15.17=a;smithi050:172.21.15.50=b;smithi076:172.21.15.76=c' 2024-02-14T14:54:20.362 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:20.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:20.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1967025556' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-14T14:54:20.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:20.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:20.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:21.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:21 smithi017 ceph-mon[32549]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:21.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:21 smithi017 ceph-mon[32549]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:21.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:21 smithi017 ceph-mon[32549]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T14:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.766+0000 7efc36b86700 1 -- 172.21.15.76:0/3822174688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 msgr2=0x7efc30107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.766+0000 7efc36b86700 1 --2- 172.21.15.76:0/3822174688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30107b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b0a0 tx=0x7efc2800b3b0 comp rx=0 tx=0).stop 2024-02-14T14:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.766+0000 7efc36b86700 1 -- 172.21.15.76:0/3822174688 shutdown_connections 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.766+0000 7efc36b86700 1 --2- 172.21.15.76:0/3822174688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30107b90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.766+0000 7efc36b86700 1 -- 172.21.15.76:0/3822174688 >> 172.21.15.76:0/3822174688 conn(0x7efc300fb8a0 msgr2=0x7efc300fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.766+0000 7efc36b86700 1 -- 172.21.15.76:0/3822174688 shutdown_connections 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.766+0000 7efc36b86700 1 -- 172.21.15.76:0/3822174688 wait complete. 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc36b86700 1 Processor -- start 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc36b86700 1 -- start start 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc36b86700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30100d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc36b86700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc30102c50 con 0x7efc301077b0 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc34922700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30100d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc34922700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30100d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.76:59898/0 (socket says 172.21.15.76:59898) 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc34922700 1 -- 172.21.15.76:0/647274925 learned_addr learned my addr 172.21.15.76:0/647274925 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.767+0000 7efc34922700 1 -- 172.21.15.76:0/647274925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc28009d00 con 0x7efc301077b0 2024-02-14T14:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.768+0000 7efc34922700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30100d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc280062a0 tx=0x7efc2800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.768+0000 7efc2dffb700 1 -- 172.21.15.76:0/647274925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc2800e040 con 0x7efc301077b0 2024-02-14T14:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.768+0000 7efc2dffb700 1 -- 172.21.15.76:0/647274925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc28003c10 con 0x7efc301077b0 2024-02-14T14:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.768+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc301012a0 con 0x7efc301077b0 2024-02-14T14:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.768+0000 7efc2dffb700 1 -- 172.21.15.76:0/647274925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc2801d600 con 0x7efc301077b0 2024-02-14T14:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.768+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc30101720 con 0x7efc301077b0 2024-02-14T14:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.769+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc30102290 con 0x7efc301077b0 2024-02-14T14:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.769+0000 7efc2dffb700 1 -- 172.21.15.76:0/647274925 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7efc28019070 con 0x7efc301077b0 2024-02-14T14:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.769+0000 7efc2dffb700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efc1c038fd0 0x7efc1c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.769+0000 7efc2dffb700 1 -- 172.21.15.76:0/647274925 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc2804d200 con 0x7efc301077b0 2024-02-14T14:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.770+0000 7efc2ffff700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efc1c038fd0 0x7efc1c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.770+0000 7efc2ffff700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efc1c038fd0 0x7efc1c03b490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc18006fd0 tx=0x7efc18006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:21.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.775+0000 7efc2dffb700 1 -- 172.21.15.76:0/647274925 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc28003d80 con 0x7efc301077b0 2024-02-14T14:54:21.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.983+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi050:172.21.15.50=b;smithi076:172.21.15.76=c", "target": ["mon-mgr", ""]}) v1 -- 0x7efc30101a50 con 0x7efc1c038fd0 2024-02-14T14:54:21.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.990+0000 7efc2dffb700 1 -- 172.21.15.76:0/647274925 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efc30101a50 con 0x7efc1c038fd0 2024-02-14T14:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout:Scheduled mon update... 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efc1c038fd0 msgr2=0x7efc1c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efc1c038fd0 0x7efc1c03b490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc18006fd0 tx=0x7efc18006e40 comp rx=0 tx=0).stop 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 msgr2=0x7efc30100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30100d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc280062a0 tx=0x7efc2800bd40 comp rx=0 tx=0).stop 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 shutdown_connections 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efc1c038fd0 0x7efc1c03b490 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 --2- 172.21.15.76:0/647274925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc301077b0 0x7efc30100d60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 >> 172.21.15.76:0/647274925 conn(0x7efc300fb8a0 msgr2=0x7efc300fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 shutdown_connections 2024-02-14T14:54:21.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:21.992+0000 7efc36b86700 1 -- 172.21.15.76:0/647274925 wait complete. 2024-02-14T14:54:22.568 DEBUG:teuthology.orchestra.run.smithi050:mon.b> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.b.service 2024-02-14T14:54:22.571 DEBUG:teuthology.orchestra.run.smithi076:mon.c> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.c.service 2024-02-14T14:54:22.573 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T14:54:22.573 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T14:54:22.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:22.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:22.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:22.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:22.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:22.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:22.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:22.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:23.737 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:23.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:23 smithi017 ceph-mon[32549]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi050:172.21.15.50=b;smithi076:172.21.15.76=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:23.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:23 smithi017 ceph-mon[32549]: Saving service mon spec with placement smithi017:172.21.15.17=a;smithi050:172.21.15.50=b;smithi076:172.21.15.76=c;count:3 2024-02-14T14:54:23.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:23 smithi017 ceph-mon[32549]: Deploying daemon mon.c on smithi076 2024-02-14T14:54:24.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:24 smithi017 ceph-mon[32549]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:26.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 -- 172.21.15.76:0/972800574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 msgr2=0x7f58e4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:26.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 --2- 172.21.15.76:0/972800574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4107af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009910 tx=0x7f58e0009c20 comp rx=0 tx=0).stop 2024-02-14T14:54:26.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 -- 172.21.15.76:0/972800574 shutdown_connections 2024-02-14T14:54:26.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 --2- 172.21.15.76:0/972800574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4107af0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:26.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 -- 172.21.15.76:0/972800574 >> 172.21.15.76:0/972800574 conn(0x7f58e40fd220 msgr2=0x7f58e40ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:26.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 -- 172.21.15.76:0/972800574 shutdown_connections 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 -- 172.21.15.76:0/972800574 wait complete. 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 Processor -- start 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.296+0000 7f58eb550700 1 -- start start 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.297+0000 7f58eb550700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.297+0000 7f58eb550700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e4075f10 con 0x7f58e4107710 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.297+0000 7f58e92ec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.297+0000 7f58e92ec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.76:59918/0 (socket says 172.21.15.76:59918) 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.297+0000 7f58e92ec700 1 -- 172.21.15.76:0/796193854 learned_addr learned my addr 172.21.15.76:0/796193854 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.297+0000 7f58e92ec700 1 -- 172.21.15.76:0/796193854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e00095f0 con 0x7f58e4107710 2024-02-14T14:54:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.298+0000 7f58e92ec700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4074020 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f58e0004f40 tx=0x7f58e0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.298+0000 7f58da7fc700 1 -- 172.21.15.76:0/796193854 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58e001d070 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.298+0000 7f58da7fc700 1 -- 172.21.15.76:0/796193854 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58e000bbd0 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.298+0000 7f58da7fc700 1 -- 172.21.15.76:0/796193854 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58e0003900 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.298+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e4074560 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.298+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e40749e0 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.299+0000 7f58da7fc700 1 -- 172.21.15.76:0/796193854 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f58e0003bc0 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.299+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e4063350 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.299+0000 7f58da7fc700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f58d0038f80 0x7f58d003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.299+0000 7f58da7fc700 1 -- 172.21.15.76:0/796193854 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f58e004ce60 con 0x7f58e4107710 2024-02-14T14:54:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.299+0000 7f58e8aeb700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f58d0038f80 0x7f58d003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.300+0000 7f58e8aeb700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f58d0038f80 0x7f58d003b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58d4006fd0 tx=0x7f58d4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.304+0000 7f58da7fc700 1 -- 172.21.15.76:0/796193854 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58e0029910 con 0x7f58e4107710 2024-02-14T14:54:26.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.594+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58e4074c70 con 0x7f58e4107710 2024-02-14T14:54:26.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.595+0000 7f58da7fc700 1 -- 172.21.15.76:0/796193854 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f58e0026070 con 0x7f58e4107710 2024-02-14T14:54:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:54:26.596 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":1,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","modified":"2024-02-14T14:52:43.130422Z","created":"2024-02-14T14:52:43.130422Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-14T14:54:26.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f58d0038f80 msgr2=0x7f58d003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:26.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f58d0038f80 0x7f58d003b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58d4006fd0 tx=0x7f58d4006e40 comp rx=0 tx=0).stop 2024-02-14T14:54:26.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 msgr2=0x7f58e4074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:26.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4074020 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f58e0004f40 tx=0x7f58e0005020 comp rx=0 tx=0).stop 2024-02-14T14:54:26.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 shutdown_connections 2024-02-14T14:54:26.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f58d0038f80 0x7f58d003b440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:26.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 --2- 172.21.15.76:0/796193854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58e4107710 0x7f58e4074020 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:26.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.597+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 >> 172.21.15.76:0/796193854 conn(0x7f58e40fd220 msgr2=0x7f58e4109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:26.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.598+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 shutdown_connections 2024-02-14T14:54:26.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:26.598+0000 7f58eb550700 1 -- 172.21.15.76:0/796193854 wait complete. 2024-02-14T14:54:26.599 INFO:teuthology.orchestra.run.smithi076.stderr:dumped monmap epoch 1 2024-02-14T14:54:26.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:26 smithi017 ceph-mon[32549]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:26.940 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:26 smithi076 ceph-mon[37057]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-14T14:54:27.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:27 smithi076 bash[37023]: 8a2c4279921bffff61a05488fdeda40fda2cec5b6d3254403379f6e9308702cc 2024-02-14T14:54:27.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:27 smithi076 systemd[1]: Started Ceph mon.c for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T14:54:28.335 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T14:54:28.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T14:54:28.576 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:54:32.173 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:32.173 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: mon.a calling monitor election 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: mon.c calling monitor election 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.174 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: fsmap 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: mgrmap e13: x(active, since 49s) 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: overall HEALTH_OK 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:32.175 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:54:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: mon.a calling monitor election 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: mon.c calling monitor election 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T14:54:32.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:32.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: fsmap 2024-02-14T14:54:32.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:32.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: mgrmap e13: x(active, since 49s) 2024-02-14T14:54:32.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: overall HEALTH_OK 2024-02-14T14:54:32.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:32.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.890+0000 7f78a9f8f700 1 -- 172.21.15.76:0/269764186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f788c0035c0 msgr2=0x7f788c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.890+0000 7f78a9f8f700 1 --2- 172.21.15.76:0/269764186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f788c0035c0 0x7f788c005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78a40696b0 tx=0x7f789c009e80 comp rx=0 tx=0).stop 2024-02-14T14:54:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.890+0000 7f78a9f8f700 1 -- 172.21.15.76:0/269764186 shutdown_connections 2024-02-14T14:54:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.890+0000 7f78a9f8f700 1 --2- 172.21.15.76:0/269764186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f788c0035c0 0x7f788c005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.890+0000 7f78a9f8f700 1 -- 172.21.15.76:0/269764186 >> 172.21.15.76:0/269764186 conn(0x7f78a40fc1d0 msgr2=0x7f78a40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.890+0000 7f78a9f8f700 1 -- 172.21.15.76:0/269764186 shutdown_connections 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.890+0000 7f78a9f8f700 1 -- 172.21.15.76:0/269764186 wait complete. 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a9f8f700 1 Processor -- start 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a9f8f700 1 -- start start 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a9f8f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 0x7f78a4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a9f8f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78a4197580 0x7f78a419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a9f8f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4197b10 con 0x7f78a4197580 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a9f8f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4197c80 con 0x7f788c0035c0 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a37fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 0x7f78a4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a37fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 0x7f78a4197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40184/0 (socket says 172.21.15.76:40184) 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a37fe700 1 -- 172.21.15.76:0/1838979771 learned_addr learned my addr 172.21.15.76:0/1838979771 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:54:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.891+0000 7f78a2ffd700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78a4197580 0x7f78a419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a37fe700 1 -- 172.21.15.76:0/1838979771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 msgr2=0x7f78a4197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a37fe700 1 -- 172.21.15.76:0/1838979771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 msgr2=0x7f78a4197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a37fe700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 0x7f78a4197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a37fe700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 0x7f78a4197040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a2ffd700 1 -- 172.21.15.76:0/1838979771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 msgr2=0x7f78a4197040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a2ffd700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 0x7f78a4197040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a2ffd700 1 -- 172.21.15.76:0/1838979771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c0098d0 con 0x7f78a4197580 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a2ffd700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78a4197580 0x7f78a419b910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f789800b4d0 tx=0x7f789800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a0ff9700 1 -- 172.21.15.76:0/1838979771 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7898011630 con 0x7f78a4197580 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a0ff9700 1 -- 172.21.15.76:0/1838979771 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7898011c70 con 0x7f78a4197580 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a419beb0 con 0x7f78a4197580 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a0ff9700 1 -- 172.21.15.76:0/1838979771 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7898010de0 con 0x7f78a4197580 2024-02-14T14:54:32.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.892+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a419c400 con 0x7f78a4197580 2024-02-14T14:54:32.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.893+0000 7f78a0ff9700 1 -- 172.21.15.76:0/1838979771 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f789801a400 con 0x7f78a4197580 2024-02-14T14:54:32.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.893+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a4191010 con 0x7f78a4197580 2024-02-14T14:54:32.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.894+0000 7f78a0ff9700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7890038d70 0x7f789003b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:32.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.894+0000 7f78a0ff9700 1 -- 172.21.15.76:0/1838979771 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f789804b780 con 0x7f78a4197580 2024-02-14T14:54:32.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.894+0000 7f78a37fe700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7890038d70 0x7f789003b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:32.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.894+0000 7f78a37fe700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7890038d70 0x7f789003b230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f789c009160 tx=0x7f789c0090b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:32.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:32.899+0000 7f78a0ff9700 1 -- 172.21.15.76:0/1838979771 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78980178b0 con 0x7f78a4197580 2024-02-14T14:54:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:32 smithi076 ceph-mon[37057]: Deploying daemon mon.b on smithi050 2024-02-14T14:54:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:33.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.172+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f78a41983a0 con 0x7f78a4197580 2024-02-14T14:54:33.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.172+0000 7f78a0ff9700 1 -- 172.21.15.76:0/1838979771 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7f78980172c0 con 0x7f78a4197580 2024-02-14T14:54:33.174 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:54:33.174 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":2,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","modified":"2024-02-14T14:54:26.746405Z","created":"2024-02-14T14:52:43.130422Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-14T14:54:33.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7890038d70 msgr2=0x7f789003b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7890038d70 0x7f789003b230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f789c009160 tx=0x7f789c0090b0 comp rx=0 tx=0).stop 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78a4197580 msgr2=0x7f78a419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78a4197580 0x7f78a419b910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f789800b4d0 tx=0x7f789800b890 comp rx=0 tx=0).stop 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 shutdown_connections 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7890038d70 0x7f789003b230 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f788c0035c0 0x7f78a4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 --2- 172.21.15.76:0/1838979771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78a4197580 0x7f78a419b910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 >> 172.21.15.76:0/1838979771 conn(0x7f78a40fc1d0 msgr2=0x7f78a40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 shutdown_connections 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:33.174+0000 7f78a9f8f700 1 -- 172.21.15.76:0/1838979771 wait complete. 2024-02-14T14:54:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:dumped monmap epoch 2 2024-02-14T14:54:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:32 smithi017 ceph-mon[32549]: Deploying daemon mon.b on smithi050 2024-02-14T14:54:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:33 smithi076 ceph-mon[37057]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:33 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/1838979771' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T14:54:34.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:33 smithi017 ceph-mon[32549]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:34.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:33 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/1838979771' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T14:54:34.782 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T14:54:34.782 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T14:54:35.035 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:54:35.976 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 podman[36492]: 2024-02-14 14:54:35.951645815 +0000 UTC m=+0.342359092 container start e78f83618c287769235beadc946b3fcd78c6ca926923f5d1420c4a02dfad3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T14:54:35.977 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T14:54:35.977 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev), process ceph-mon, pid 2 2024-02-14T14:54:35.977 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: pidfile_write: ignore empty --pid-file 2024-02-14T14:54:36.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: load: jerasure load: lrc 2024-02-14T14:54:36.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: RocksDB version: 7.9.2 2024-02-14T14:54:36.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Git sha 0 2024-02-14T14:54:36.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Compile date 2024-02-06 05:38:06 2024-02-14T14:54:36.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: DB SUMMARY 2024-02-14T14:54:36.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: DB Session ID: DW5D7HMBO8QPV73SHP2Y 2024-02-14T14:54:36.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: CURRENT file: CURRENT 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.error_if_exists: 0 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.create_if_missing: 0 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.paranoid_checks: 1 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.env: 0x5607bf2edae0 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T14:54:36.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.info_log: 0x5607c12558c0 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.statistics: (nil) 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.use_fsync: 0 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_log_file_size: 0 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.allow_fallocate: 1 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T14:54:36.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.use_direct_reads: 0 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.db_log_dir: 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.wal_dir: 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T14:54:36.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.write_buffer_manager: 0x5607c1229400 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.unordered_write: 0 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T14:54:36.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.row_cache: None 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.wal_filter: None 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.two_write_queues: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.wal_compression: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.atomic_flush: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.log_readahead_size: 0 2024-02-14T14:54:36.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_background_jobs: 2 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_background_compactions: -1 2024-02-14T14:54:36.399 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_subcompactions: 1 2024-02-14T14:54:36.401 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T14:54:36.401 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T14:54:36.401 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T14:54:36.401 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T14:54:36.401 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T14:54:36.401 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_open_files: -1 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_background_flushes: -1 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Compression algorithms supported: 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kZSTD supported: 0 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kXpressCompression supported: 0 2024-02-14T14:54:36.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kZlibCompression supported: 1 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kSnappyCompression supported: 1 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kLZ4Compression supported: 1 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: kBZip2Compression supported: 0 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.merge_operator: 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_filter: None 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T14:54:36.403 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5607c12559c0) 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: cache_index_and_filter_blocks: 1 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: pin_top_level_index_and_filter: 1 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_type: 0 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: data_block_index_type: 0 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_shortening: 1 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: checksum: 4 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: no_block_cache: 0 2024-02-14T14:54:36.404 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache: 0x5607c1259090 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_name: BinnedLRUCache 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_options: 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: capacity : 536870912 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: num_shard_bits : 4 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: strict_capacity_limit : 0 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: high_pri_pool_ratio: 0.000 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_compressed: (nil) 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: persistent_cache: (nil) 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_size: 4096 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_size_deviation: 10 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_restart_interval: 16 2024-02-14T14:54:36.405 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_block_restart_interval: 1 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: metadata_block_size: 4096 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: partition_filters: 0 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: use_delta_encoding: 1 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: filter_policy: bloomfilter 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: whole_key_filtering: 1 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: verify_compression: 0 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: read_amp_bytes_per_bit: 0 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: format_version: 5 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: enable_index_compression: 1 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_align: 0 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: max_auto_readahead_size: 262144 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: prepopulate_block_cache: 0 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: initial_auto_readahead_size: 8192 2024-02-14T14:54:36.406 INFO:journalctl@ceph.mon.b.smithi050.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression: NoCompression 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.num_levels: 7 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T14:54:36.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T14:54:36.408 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T14:54:36.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T14:54:36.410 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.table_properties_collectors: 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.inplace_update_support: 0 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.bloom_locality: 0 2024-02-14T14:54:36.411 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.max_successive_merges: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.ttl: 2592000 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.enable_blob_files: false 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.min_blob_size: 0 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T14:54:36.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eb60d8ef-4237-4920-ad68-f2bffcab2fe7 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707922475982148, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707922475982561, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707922475, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eb60d8ef-4237-4920-ad68-f2bffcab2fe7", "db_session_id": "DW5D7HMBO8QPV73SHP2Y", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707922475982643, "job": 1, "event": "recovery_finished"} 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T14:54:36.413 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5607c134a000 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: DB pointer 0x5607c1334000 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** DB Stats ** 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-14T14:54:36.414 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-14T14:54:36.415 INFO:journalctl@ceph.mon.b.smithi050.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache BinnedLRUCache@0x5607c1259090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-14T14:54:36.416 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: using public_addr v2:172.21.15.50:0/0 -> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: starting mon.b rank -1 at public addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] at bind addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: mon.b@-1(???) e0 preinit fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).mds e1 new map 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:35 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout: e1 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout: legacy client fscid: -1 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout: No filesystems configured 2024-02-14T14:54:36.417 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.418 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Deploying cephadm binary to smithi076 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.419 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Added host smithi076 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.420 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1967025556' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1967025556' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.421 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi050:172.21.15.50=b;smithi076:172.21.15.76=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:36.422 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Saving service mon spec with placement smithi017:172.21.15.17=a;smithi050:172.21.15.50=b;smithi076:172.21.15.76=c;count:3 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Deploying daemon mon.c on smithi076 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.a calling monitor election 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.c calling monitor election 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.423 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: fsmap 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mgrmap e13: x(active, since 49s) 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: overall HEALTH_OK 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: Deploying daemon mon.b on smithi050 2024-02-14T14:54:36.424 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:36.425 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:36.425 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/1838979771' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T14:54:36.425 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 ceph-mon[36512]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-14T14:54:36.425 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 bash[36492]: e78f83618c287769235beadc946b3fcd78c6ca926923f5d1420c4a02dfad3741 2024-02-14T14:54:36.425 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:36 smithi050 systemd[1]: Started Ceph mon.b for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T14:54:41.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: mon.c calling monitor election 2024-02-14T14:54:41.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:54:41.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: mon.a calling monitor election 2024-02-14T14:54:41.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: fsmap 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: mgrmap e13: x(active, since 58s) 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-14T14:54:41.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T14:54:41.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: mon.b (rank 2) addr [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] is down (out of quorum) 2024-02-14T14:54:41.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:41.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:41.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: mon.c calling monitor election 2024-02-14T14:54:41.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:54:41.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: mon.a calling monitor election 2024-02-14T14:54:41.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:41.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: fsmap 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: mgrmap e13: x(active, since 58s) 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: mon.b (rank 2) addr [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] is down (out of quorum) 2024-02-14T14:54:41.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:41.518 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:42.446 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:42 smithi076 ceph-mon[37057]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:42.447 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:42.447 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:42.447 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:42.447 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:42.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.445+0000 7fa32c74d700 1 -- 172.21.15.76:0/2213181445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3100062a0 msgr2=0x7fa310008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:42.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.445+0000 7fa32c74d700 1 --2- 172.21.15.76:0/2213181445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3100062a0 0x7fa310008710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa3241028b0 tx=0x7fa32000fe90 comp rx=0 tx=0).stop 2024-02-14T14:54:42.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.445+0000 7fa32c74d700 1 -- 172.21.15.76:0/2213181445 shutdown_connections 2024-02-14T14:54:42.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.445+0000 7fa32c74d700 1 --2- 172.21.15.76:0/2213181445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3100062a0 0x7fa310008710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.445+0000 7fa32c74d700 1 -- 172.21.15.76:0/2213181445 >> 172.21.15.76:0/2213181445 conn(0x7fa3240fd320 msgr2=0x7fa3240ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.445+0000 7fa32c74d700 1 -- 172.21.15.76:0/2213181445 shutdown_connections 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.445+0000 7fa32c74d700 1 -- 172.21.15.76:0/2213181445 wait complete. 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 Processor -- start 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 -- start start 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3100062a0 0x7fa3241973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 0x7fa32419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa32419c1c0 0x7fa32419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa324197fe0 con 0x7fa32419c1c0 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241a3f30 con 0x7fa3241978f0 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32c74d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241a4070 con 0x7fa3100062a0 2024-02-14T14:54:42.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32a4e9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3100062a0 0x7fa3241973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32a4e9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3100062a0 0x7fa3241973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45294/0 (socket says 172.21.15.76:45294) 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32a4e9700 1 -- 172.21.15.76:0/3298657200 learned_addr learned my addr 172.21.15.76:0/3298657200 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa32acea700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa32419c1c0 0x7fa32419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.446+0000 7fa329ce8700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 0x7fa32419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32a4e9700 1 -- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 msgr2=0x7fa32419bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa329ce8700 1 -- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 msgr2=0x7fa32419bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa329ce8700 1 -- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 msgr2=0x7fa32419bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa329ce8700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 0x7fa32419bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa329ce8700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 0x7fa32419bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T14:54:42.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32a4e9700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 0x7fa32419bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32a4e9700 1 -- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa32419c1c0 msgr2=0x7fa32419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32a4e9700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa32419c1c0 0x7fa32419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32a4e9700 1 -- 172.21.15.76:0/3298657200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32000f8e0 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32a4e9700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3100062a0 0x7fa3241973b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa320013f60 tx=0x7fa320013f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa3177fe700 1 -- 172.21.15.76:0/3298657200 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320011e60 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa3177fe700 1 -- 172.21.15.76:0/3298657200 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa32001d410 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3241a42f0 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa3177fe700 1 -- 172.21.15.76:0/3298657200 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200038e0 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.447+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3241a47e0 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.448+0000 7fa3177fe700 1 -- 172.21.15.76:0/3298657200 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fa320003a80 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.448+0000 7fa3177fe700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa308039410 0x7fa30803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.448+0000 7fa3177fe700 1 -- 172.21.15.76:0/3298657200 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa32004efb0 con 0x7fa3100062a0 2024-02-14T14:54:42.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.448+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa324191010 con 0x7fa3100062a0 2024-02-14T14:54:42.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.448+0000 7fa329ce8700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa308039410 0x7fa30803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:42.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.449+0000 7fa329ce8700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa308039410 0x7fa30803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa318009850 tx=0x7fa318008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:42.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.454+0000 7fa3177fe700 1 -- 172.21.15.76:0/3298657200 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa320039350 con 0x7fa3100062a0 2024-02-14T14:54:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:42 smithi017 ceph-mon[32549]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:42.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.745+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa324198710 con 0x7fa3100062a0 2024-02-14T14:54:42.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.746+0000 7fa3177fe700 1 -- 172.21.15.76:0/3298657200 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1275 (secure 0 0 0) 0x7fa320040090 con 0x7fa3100062a0 2024-02-14T14:54:42.747 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-02-14T14:54:42.748 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":3,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","modified":"2024-02-14T14:54:36.005163Z","created":"2024-02-14T14:52:43.130422Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa308039410 msgr2=0x7fa30803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa308039410 0x7fa30803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa318009850 tx=0x7fa318008040 comp rx=0 tx=0).stop 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3100062a0 msgr2=0x7fa3241973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3100062a0 0x7fa3241973b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa320013f60 tx=0x7fa320013f90 comp rx=0 tx=0).stop 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 shutdown_connections 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa308039410 0x7fa30803b8d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa32419c1c0 0x7fa32419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3100062a0 0x7fa3241973b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:42.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 --2- 172.21.15.76:0/3298657200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3241978f0 0x7fa32419bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:42.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 >> 172.21.15.76:0/3298657200 conn(0x7fa3240fd320 msgr2=0x7fa324106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:42.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 shutdown_connections 2024-02-14T14:54:42.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:42.748+0000 7fa32c74d700 1 -- 172.21.15.76:0/3298657200 wait complete. 2024-02-14T14:54:42.751 INFO:teuthology.orchestra.run.smithi076.stderr:dumped monmap epoch 3 2024-02-14T14:54:43.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: mon.c calling monitor election 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: mon.a calling monitor election 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-14T14:54:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: fsmap 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: mgrmap e13: x(active, since 58s) 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: mon.b (rank 2) addr [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] is down (out of quorum) 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:43.409 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-14T14:54:43.409 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph config generate-minimal-conf 2024-02-14T14:54:43.657 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:54:44.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: mon.b calling monitor election 2024-02-14T14:54:44.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:44.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: mon.b calling monitor election 2024-02-14T14:54:44.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: mon.c calling monitor election 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: mon.a calling monitor election 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: fsmap 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: mgrmap e13: x(active, since 60s) 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: Cluster is now healthy 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: overall HEALTH_OK 2024-02-14T14:54:44.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T14:54:44.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:44.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:44.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:44.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:44 smithi017 ceph-mon[32549]: Reconfiguring daemon mon.a on smithi017 2024-02-14T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: mon.b calling monitor election 2024-02-14T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: mon.b calling monitor election 2024-02-14T14:54:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: mon.c calling monitor election 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: mon.a calling monitor election 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: fsmap 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: mgrmap e13: x(active, since 60s) 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: Cluster is now healthy 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: overall HEALTH_OK 2024-02-14T14:54:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T14:54:44.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:44.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:44 smithi050 ceph-mon[36512]: Reconfiguring daemon mon.a on smithi017 2024-02-14T14:54:44.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: mon.b calling monitor election 2024-02-14T14:54:44.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:44.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: mon.b calling monitor election 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: mon.c calling monitor election 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: mon.a calling monitor election 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: fsmap 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: mgrmap e13: x(active, since 60s) 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: Cluster is now healthy 2024-02-14T14:54:44.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: overall HEALTH_OK 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:44.486 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:44 smithi076 ceph-mon[37057]: Reconfiguring daemon mon.a on smithi017 2024-02-14T14:54:45.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 14:54:45 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T14:54:45.004+0000 7f0a1db41700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-14T14:54:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:45.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:45.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:45.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:54:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 -- 172.21.15.17:0/3058064969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4102c10 msgr2=0x7f1ec4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/3058064969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4102c10 0x7f1ec4102ff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009990 tx=0x7f1ebc009ca0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 -- 172.21.15.17:0/3058064969 shutdown_connections 2024-02-14T14:54:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/3058064969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ec4101680 0x7f1ec410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/3058064969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4102c10 0x7f1ec4102ff0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/3058064969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec4100cc0 0x7f1ec4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 -- 172.21.15.17:0/3058064969 >> 172.21.15.17:0/3058064969 conn(0x7f1ec4076c70 msgr2=0x7f1ec4077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 -- 172.21.15.17:0/3058064969 shutdown_connections 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 -- 172.21.15.17:0/3058064969 wait complete. 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.212+0000 7f1ec97e5700 1 Processor -- start 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec97e5700 1 -- start start 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec97e5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ec4100cc0 0x7f1ec4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec97e5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4101680 0x7f1ec41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec97e5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec4102c10 0x7f1ec419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec97e5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419c170 con 0x7f1ec4101680 2024-02-14T14:54:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec97e5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419c2e0 con 0x7f1ec4100cc0 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec97e5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419c4a0 con 0x7f1ec4102c10 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4101680 0x7f1ec41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4101680 0x7f1ec41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55754/0 (socket says 172.21.15.17:55754) 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 -- 172.21.15.17:0/1896079360 learned_addr learned my addr 172.21.15.17:0/1896079360 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec2ffd700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ec4100cc0 0x7f1ec4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 -- 172.21.15.17:0/1896079360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec4102c10 msgr2=0x7f1ec419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec4102c10 0x7f1ec419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 -- 172.21.15.17:0/1896079360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ec4100cc0 msgr2=0x7f1ec4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ec4100cc0 0x7f1ec4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 -- 172.21.15.17:0/1896079360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0095f0 con 0x7f1ec4101680 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.213+0000 7f1ec27fc700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4101680 0x7f1ec41977d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00d950 tx=0x7f1eac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.214+0000 7f1ebbfff700 1 -- 172.21.15.17:0/1896079360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac009940 con 0x7f1ec4101680 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.214+0000 7f1ebbfff700 1 -- 172.21.15.17:0/1896079360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1eac010460 con 0x7f1ec4101680 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.214+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec419c780 con 0x7f1ec4101680 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.214+0000 7f1ebbfff700 1 -- 172.21.15.17:0/1896079360 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00f620 con 0x7f1ec4101680 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.214+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec41a3fe0 con 0x7f1ec4101680 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.214+0000 7f1ebbfff700 1 -- 172.21.15.17:0/1896079360 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f1eac00f780 con 0x7f1ec4101680 2024-02-14T14:54:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.215+0000 7f1ebbfff700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1eb00391c0 0x7f1eb003b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.215+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec41092a0 con 0x7f1ec4101680 2024-02-14T14:54:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.215+0000 7f1ebbfff700 1 -- 172.21.15.17:0/1896079360 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1eac04baf0 con 0x7f1ec4101680 2024-02-14T14:54:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.215+0000 7f1ec2ffd700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1eb00391c0 0x7f1eb003b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.215+0000 7f1ec2ffd700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1eb00391c0 0x7f1eb003b680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00b5c0 tx=0x7f1ebc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.220+0000 7f1ebbfff700 1 -- 172.21.15.17:0/1896079360 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1eac0226c0 con 0x7f1ec4101680 2024-02-14T14:54:46.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.421+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1ec41985f0 con 0x7f1ec4101680 2024-02-14T14:54:46.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.422+0000 7f1ebbfff700 1 -- 172.21.15.17:0/1896079360 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7f1eac016070 con 0x7f1ec4101680 2024-02-14T14:54:46.424 INFO:teuthology.orchestra.run.smithi017.stdout:# minimal ceph.conf for 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:54:46.424 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2024-02-14T14:54:46.424 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T14:54:46.424 INFO:teuthology.orchestra.run.smithi017.stdout: mon_host = [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] 2024-02-14T14:54:46.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1eb00391c0 msgr2=0x7f1eb003b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1eb00391c0 0x7f1eb003b680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00b5c0 tx=0x7f1ebc005fd0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4101680 msgr2=0x7f1ec41977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4101680 0x7f1ec41977d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00d950 tx=0x7f1eac00dc60 comp rx=0 tx=0).stop 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 shutdown_connections 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1eb00391c0 0x7f1eb003b680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ec4102c10 0x7f1ec419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ec4100cc0 0x7f1ec4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 --2- 172.21.15.17:0/1896079360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ec4101680 0x7f1ec41977d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:46.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.424+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 >> 172.21.15.17:0/1896079360 conn(0x7f1ec4076c70 msgr2=0x7f1ec40ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:46.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.425+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 shutdown_connections 2024-02-14T14:54:46.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:46.425+0000 7f1ec97e5700 1 -- 172.21.15.17:0/1896079360 wait complete. 2024-02-14T14:54:46.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: Reconfiguring mon.b (monmap changed)... 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: Reconfiguring daemon mon.b on smithi050 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1896079360' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:46.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: Reconfiguring mon.b (monmap changed)... 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: Reconfiguring daemon mon.b on smithi050 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1896079360' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:46.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: Reconfiguring mon.b (monmap changed)... 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: Reconfiguring daemon mon.b on smithi050 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1896079360' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T14:54:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T14:54:46.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:47.023 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-14T14:54:47.023 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:54:47.023 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T14:54:47.054 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:54:47.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:47.121 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:54:47.121 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T14:54:47.161 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:54:47.161 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:47.238 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:54:47.238 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T14:54:47.267 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:54:47.267 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T14:54:47.334 INFO:tasks.cephadm:Adding mgr.x on smithi017 2024-02-14T14:54:47.335 INFO:tasks.cephadm:Adding mgr.y on smithi050 2024-02-14T14:54:47.335 INFO:tasks.cephadm:Adding mgr.z on smithi076 2024-02-14T14:54:47.335 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch apply mgr '3;smithi017=x;smithi050=y;smithi076=z' 2024-02-14T14:54:47.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:47 smithi076 ceph-mon[37057]: Reconfiguring mon.c (monmap changed)... 2024-02-14T14:54:47.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:47 smithi076 ceph-mon[37057]: Reconfiguring daemon mon.c on smithi076 2024-02-14T14:54:47.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:47 smithi017 ceph-mon[32549]: Reconfiguring mon.c (monmap changed)... 2024-02-14T14:54:47.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:47 smithi017 ceph-mon[32549]: Reconfiguring daemon mon.c on smithi076 2024-02-14T14:54:47.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:47 smithi050 ceph-mon[36512]: Reconfiguring mon.c (monmap changed)... 2024-02-14T14:54:47.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:47 smithi050 ceph-mon[36512]: Reconfiguring daemon mon.c on smithi076 2024-02-14T14:54:48.037 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:54:48.518 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:48 smithi017 ceph-mon[32549]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:48.519 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:48.519 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:48.519 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:48.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:48 smithi076 ceph-mon[37057]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:48.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:48.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:48.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:48.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:48 smithi050 ceph-mon[36512]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:48.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:48.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:48.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 -- 172.21.15.76:0/2511173592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c105180 msgr2=0x7f603c107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 --2- 172.21.15.76:0/2511173592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c105180 0x7f603c107570 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6030009990 tx=0x7f6030009ca0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 -- 172.21.15.76:0/2511173592 shutdown_connections 2024-02-14T14:54:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 --2- 172.21.15.76:0/2511173592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f603c107ab0 0x7f603c109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 --2- 172.21.15.76:0/2511173592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f603c102850 0x7f603c104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 --2- 172.21.15.76:0/2511173592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c105180 0x7f603c107570 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 -- 172.21.15.76:0/2511173592 >> 172.21.15.76:0/2511173592 conn(0x7f603c0fc280 msgr2=0x7f603c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 -- 172.21.15.76:0/2511173592 shutdown_connections 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 -- 172.21.15.76:0/2511173592 wait complete. 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 Processor -- start 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.508+0000 7f6041f02700 1 -- start start 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6041f02700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c102850 0x7f603c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6041f02700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f603c105180 0x7f603c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6041f02700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f603c107ab0 0x7f603c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6041f02700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c19f820 con 0x7f603c105180 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6041f02700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c19f960 con 0x7f603c107ab0 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6041f02700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c19fb20 con 0x7f603c102850 2024-02-14T14:54:49.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c102850 0x7f603c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c102850 0x7f603c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45312/0 (socket says 172.21.15.76:45312) 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 -- 172.21.15.76:0/3204785216 learned_addr learned my addr 172.21.15.76:0/3204785216 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603affd700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f603c105180 0x7f603c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603bfff700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f603c107ab0 0x7f603c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 -- 172.21.15.76:0/3204785216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f603c107ab0 msgr2=0x7f603c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f603c107ab0 0x7f603c19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 -- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f603c105180 msgr2=0x7f603c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f603c105180 0x7f603c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 -- 172.21.15.76:0/3204785216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300095f0 con 0x7f603c102850 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f603b7fe700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c102850 0x7f603c197ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f602c00c9b0 tx=0x7f602c00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6038ff9700 1 -- 172.21.15.76:0/3204785216 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c007b00 con 0x7f603c102850 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6038ff9700 1 -- 172.21.15.76:0/3204785216 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f602c00ce50 con 0x7f603c102850 2024-02-14T14:54:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6038ff9700 1 -- 172.21.15.76:0/3204785216 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c018660 con 0x7f603c102850 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.509+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c19fe00 con 0x7f603c102850 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.510+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c1a0270 con 0x7f603c102850 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.510+0000 7f6038ff9700 1 -- 172.21.15.76:0/3204785216 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f602c004830 con 0x7f603c102850 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.510+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c0fde00 con 0x7f603c102850 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.511+0000 7f6038ff9700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f60240391c0 0x7f602403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.511+0000 7f6038ff9700 1 -- 172.21.15.76:0/3204785216 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f602c04ba60 con 0x7f603c102850 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.511+0000 7f603affd700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f60240391c0 0x7f602403b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.511+0000 7f603affd700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f60240391c0 0x7f602403b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f603c1991b0 tx=0x7f6030003860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:49.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.516+0000 7f6038ff9700 1 -- 172.21.15.76:0/3204785216 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f602c00d930 con 0x7f603c102850 2024-02-14T14:54:49.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.721+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi050=y;smithi076=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f603c02d9a0 con 0x7f60240391c0 2024-02-14T14:54:49.722 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:49 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:49.722 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:49 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:49.722 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:49 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:49.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.727+0000 7f6038ff9700 1 -- 172.21.15.76:0/3204785216 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f603c02d9a0 con 0x7f60240391c0 2024-02-14T14:54:49.728 INFO:teuthology.orchestra.run.smithi076.stdout:Scheduled mgr update... 2024-02-14T14:54:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f60240391c0 msgr2=0x7f602403b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f60240391c0 0x7f602403b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f603c1991b0 tx=0x7f6030003860 comp rx=0 tx=0).stop 2024-02-14T14:54:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c102850 msgr2=0x7f603c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c102850 0x7f603c197ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f602c00c9b0 tx=0x7f602c00ccc0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 shutdown_connections 2024-02-14T14:54:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f60240391c0 0x7f602403b680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f603c107ab0 0x7f603c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c102850 0x7f603c197ca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 --2- 172.21.15.76:0/3204785216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f603c105180 0x7f603c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:49.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 >> 172.21.15.76:0/3204785216 conn(0x7f603c0fc280 msgr2=0x7f603c103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:49.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 shutdown_connections 2024-02-14T14:54:49.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:54:49.729+0000 7f6041f02700 1 -- 172.21.15.76:0/3204785216 wait complete. 2024-02-14T14:54:49.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:49 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:49.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:49 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:49.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:49 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:49.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:49 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:49.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:49 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:49.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:49 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:50.285 DEBUG:teuthology.orchestra.run.smithi050:mgr.y> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.y.service 2024-02-14T14:54:50.288 DEBUG:teuthology.orchestra.run.smithi076:mgr.z> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.z.service 2024-02-14T14:54:50.290 INFO:tasks.cephadm:Deploying OSDs... 2024-02-14T14:54:50.290 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T14:54:50.290 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T14:54:50.309 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T14:54:50.309 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2024-02-14T14:54:50.367 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T14:54:50.367 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:50.367 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-14T14:54:50.367 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:50.367 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:50.367 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-02-14 14:53:54.374386984 +0000 2024-02-14T14:54:50.368 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-02-14 14:48:34.521920833 +0000 2024-02-14T14:54:50.368 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-02-14 14:48:34.521920833 +0000 2024-02-14T14:54:50.368 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-02-14 14:48:34.521920833 +0000 2024-02-14T14:54:50.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T14:54:50.438 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-02-14T14:54:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-02-14T14:54:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000126257 s, 4.1 MB/s 2024-02-14T14:54:50.440 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T14:54:50.499 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-02-14 14:53:54.410386609 +0000 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-02-14 14:48:34.837917510 +0000 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-02-14 14:48:34.837917510 +0000 2024-02-14T14:54:50.556 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-02-14 14:48:34.837917510 +0000 2024-02-14T14:54:50.557 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T14:54:50.559 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:50.559 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:50.559 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:50.559 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:50.559 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:50.560 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:50.560 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:50.560 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T14:54:50.560 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:50.560 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-02-14T14:54:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-02-14T14:54:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000115805 s, 4.4 MB/s 2024-02-14T14:54:50.625 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T14:54:50.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:50.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:50.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:50.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T14:54:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:50.682 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2024-02-14T14:54:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T14:54:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:50.739 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-14T14:54:50.739 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:50.739 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:50.739 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-02-14 14:53:54.452386171 +0000 2024-02-14T14:54:50.740 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-02-14 14:48:35.160914114 +0000 2024-02-14T14:54:50.740 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-02-14 14:48:35.160914114 +0000 2024-02-14T14:54:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-02-14 14:48:35.160914114 +0000 2024-02-14T14:54:50.740 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T14:54:50.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:50.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:50.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:50.793 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-02-14T14:54:50.793 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-02-14T14:54:50.794 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 8.988e-05 s, 5.7 MB/s 2024-02-14T14:54:50.795 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T14:54:50.851 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2024-02-14T14:54:50.907 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T14:54:50.907 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:50.908 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-14T14:54:50.908 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:50.908 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:50.908 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-02-14 14:53:54.485385826 +0000 2024-02-14T14:54:50.908 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-02-14 14:48:35.487910676 +0000 2024-02-14T14:54:50.908 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-02-14 14:48:35.487910676 +0000 2024-02-14T14:54:50.908 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-02-14 14:48:35.487910676 +0000 2024-02-14T14:54:50.908 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T14:54:50.974 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-02-14T14:54:50.974 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-02-14T14:54:50.974 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.00011142 s, 4.6 MB/s 2024-02-14T14:54:50.975 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T14:54:51.031 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-14T14:54:51.032 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T14:54:51.052 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T14:54:51.052 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-02-14T14:54:51.112 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T14:54:51.112 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:51.112 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-02-14T14:54:51.112 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:51.112 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:51.112 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-14 14:54:35.417474773 +0000 2024-02-14T14:54:51.113 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-14 14:49:02.256334488 +0000 2024-02-14T14:54:51.113 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-14 14:49:02.256334488 +0000 2024-02-14T14:54:51.113 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-14 14:49:02.256334488 +0000 2024-02-14T14:54:51.113 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T14:54:51.189 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-14T14:54:51.189 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-14T14:54:51.189 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000133795 s, 3.8 MB/s 2024-02-14T14:54:51.191 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T14:54:51.249 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-02-14T14:54:51.307 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T14:54:51.307 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:51.307 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-14T14:54:51.308 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:51.308 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:51.308 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-14 14:54:35.417474773 +0000 2024-02-14T14:54:51.308 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-14 14:49:02.590325988 +0000 2024-02-14T14:54:51.308 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-14 14:49:02.590325988 +0000 2024-02-14T14:54:51.308 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-14 14:49:02.590325988 +0000 2024-02-14T14:54:51.308 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T14:54:51.381 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-14T14:54:51.381 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-14T14:54:51.381 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000126549 s, 4.0 MB/s 2024-02-14T14:54:51.383 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T14:54:51.440 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-02-14T14:54:51.497 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T14:54:51.497 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:51.497 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-02-14T14:54:51.498 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:51.498 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:51.498 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-14 14:54:35.417474773 +0000 2024-02-14T14:54:51.498 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-14 14:49:02.931317309 +0000 2024-02-14T14:54:51.498 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-14 14:49:02.931317309 +0000 2024-02-14T14:54:51.498 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-14 14:49:02.931317309 +0000 2024-02-14T14:54:51.498 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T14:54:51.569 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-14T14:54:51.569 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-14T14:54:51.569 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.00012171 s, 4.2 MB/s 2024-02-14T14:54:51.570 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T14:54:51.628 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-02-14T14:54:51.684 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T14:54:51.684 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:51.684 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-02-14T14:54:51.684 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:51.684 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:51.684 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-14 14:54:35.418474749 +0000 2024-02-14T14:54:51.685 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-14 14:49:03.265308808 +0000 2024-02-14T14:54:51.685 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-14 14:49:03.265308808 +0000 2024-02-14T14:54:51.685 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-14 14:49:03.265308808 +0000 2024-02-14T14:54:51.685 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T14:54:51.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:51 smithi076 ceph-mon[37057]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi050=y;smithi076=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:51.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:51 smithi076 ceph-mon[37057]: Saving service mgr spec with placement smithi017=x;smithi050=y;smithi076=z;count:3 2024-02-14T14:54:51.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:51 smithi076 ceph-mon[37057]: Deploying daemon mgr.z on smithi076 2024-02-14T14:54:51.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:51 smithi076 systemd[1]: Starting Ceph mgr.z for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T14:54:51.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:51 smithi050 ceph-mon[36512]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi050=y;smithi076=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:51.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:51 smithi050 ceph-mon[36512]: Saving service mgr spec with placement smithi017=x;smithi050=y;smithi076=z;count:3 2024-02-14T14:54:51.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:51 smithi050 ceph-mon[36512]: Deploying daemon mgr.z on smithi076 2024-02-14T14:54:51.755 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-14T14:54:51.756 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-14T14:54:51.756 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.6349e-05 s, 5.3 MB/s 2024-02-14T14:54:51.757 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T14:54:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:51 smithi017 ceph-mon[32549]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi050=y;smithi076=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:51 smithi017 ceph-mon[32549]: Saving service mgr spec with placement smithi017=x;smithi050=y;smithi076=z;count:3 2024-02-14T14:54:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:51 smithi017 ceph-mon[32549]: Deploying daemon mgr.z on smithi076 2024-02-14T14:54:51.814 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-02-14T14:54:51.814 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T14:54:51.830 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T14:54:51.830 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-02-14T14:54:51.884 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T14:54:51.884 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:51.885 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 655 Links: 1 2024-02-14T14:54:51.885 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:51.885 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:51.885 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-14 14:54:51.542258473 +0000 2024-02-14T14:54:51.885 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-14 14:48:46.840802028 +0000 2024-02-14T14:54:51.885 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-14 14:48:46.840802028 +0000 2024-02-14T14:54:51.885 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-14 14:48:46.840802028 +0000 2024-02-14T14:54:51.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T14:54:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-14T14:54:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-14T14:54:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00034331 s, 1.5 MB/s 2024-02-14T14:54:51.953 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T14:54:52.008 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-02-14T14:54:52.063 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T14:54:52.063 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:52.063 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 676 Links: 1 2024-02-14T14:54:52.063 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:52.063 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:52.063 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-14 14:54:51.543258441 +0000 2024-02-14T14:54:52.063 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-14 14:48:47.176791194 +0000 2024-02-14T14:54:52.064 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-14 14:48:47.176791194 +0000 2024-02-14T14:54:52.064 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-14 14:48:47.176791194 +0000 2024-02-14T14:54:52.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T14:54:52.110 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:51 smithi076 podman[38700]: 2024-02-14 14:54:51.723717204 +0000 UTC m=+0.017868247 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T14:54:52.110 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:51 smithi076 podman[38700]: 2024-02-14 14:54:51.866501474 +0000 UTC m=+0.160652520 container create e8a8f5f084dc1ac4a96578086af5a6031f7c499e73875e08111de6e4073e0df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-14T14:54:52.110 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:52 smithi076 podman[38700]: 2024-02-14 14:54:52.110177407 +0000 UTC m=+0.404328435 container init e8a8f5f084dc1ac4a96578086af5a6031f7c499e73875e08111de6e4073e0df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.build-date=20240131) 2024-02-14T14:54:52.130 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-14T14:54:52.130 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-14T14:54:52.130 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000254936 s, 2.0 MB/s 2024-02-14T14:54:52.131 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T14:54:52.187 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-02-14T14:54:52.242 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T14:54:52.242 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:52.242 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 694 Links: 1 2024-02-14T14:54:52.242 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:52.242 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:52.242 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-14 14:54:51.543258441 +0000 2024-02-14T14:54:52.243 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-14 14:48:47.537779554 +0000 2024-02-14T14:54:52.243 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-14 14:48:47.537779554 +0000 2024-02-14T14:54:52.243 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-14 14:48:47.537779554 +0000 2024-02-14T14:54:52.243 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T14:54:52.308 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-14T14:54:52.308 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-14T14:54:52.309 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00019527 s, 2.6 MB/s 2024-02-14T14:54:52.310 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T14:54:52.365 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 712 Links: 1 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-02-14 14:54:51.543258441 +0000 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-02-14 14:48:47.876768623 +0000 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-02-14 14:48:47.876768623 +0000 2024-02-14T14:54:52.420 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-02-14 14:48:47.876768623 +0000 2024-02-14T14:54:52.421 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T14:54:52.474 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:52 smithi076 podman[38700]: 2024-02-14 14:54:52.113504248 +0000 UTC m=+0.407655283 container start e8a8f5f084dc1ac4a96578086af5a6031f7c499e73875e08111de6e4073e0df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-14T14:54:52.474 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mgr[38815]: -- 172.21.15.76:0/975985922 <== mon.1 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d766741520 con 0x55d766716800 2024-02-14T14:54:52.474 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:52 smithi076 bash[38700]: e8a8f5f084dc1ac4a96578086af5a6031f7c499e73875e08111de6e4073e0df1 2024-02-14T14:54:52.474 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:52 smithi076 systemd[1]: Started Ceph mgr.z for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T14:54:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-02-14T14:54:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-02-14T14:54:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000191885 s, 2.7 MB/s 2024-02-14T14:54:52.488 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T14:54:52.543 INFO:tasks.cephadm:Deploying osd.0 on smithi017 with /dev/vg_nvme/lv_4... 2024-02-14T14:54:52.543 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-14T14:54:52.733 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:52.674+0000 7f11e1485200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:52.791 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:52.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:53.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:52 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:52.767+0000 7f11e1485200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T14:54:53.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:53 smithi076 ceph-mon[37057]: Deploying daemon mgr.y on smithi050 2024-02-14T14:54:53.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:53.399+0000 7f11e1485200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T14:54:53.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:53.493+0000 7f11e1485200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T14:54:53.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:53.635+0000 7f11e1485200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T14:54:53.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:53 smithi017 ceph-mon[32549]: Deploying daemon mgr.y on smithi050 2024-02-14T14:54:53.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:53 smithi050 ceph-mon[36512]: Deploying daemon mgr.y on smithi050 2024-02-14T14:54:53.990 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:53.733+0000 7f11e1485200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T14:54:53.990 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:53.819+0000 7f11e1485200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T14:54:54.286 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:54.089+0000 7f11e1485200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T14:54:54.286 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:54.199+0000 7f11e1485200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T14:54:54.286 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:54.285+0000 7f11e1485200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T14:54:54.481 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:54 smithi050 podman[37922]: 2024-02-14 14:54:54.442140592 +0000 UTC m=+0.123388133 container create a8c20e957e06ecb987229afb400ade2bb7c10641ddb94cd6525197045cdeb26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-14T14:54:54.557 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:54 smithi076 ceph-mon[37057]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:54.557 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:54.380+0000 7f11e1485200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T14:54:54.557 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:54.556+0000 7f11e1485200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T14:54:54.698 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-14T14:54:54.699 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-14T14:54:54.699 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-02-14T14:54:54.699 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-02-14T14:54:54.699 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00851744 s, 1.2 GB/s 2024-02-14T14:54:54.699 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-02-14T14:54:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:54 smithi017 ceph-mon[32549]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:54.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:54 smithi050 ceph-mon[36512]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:54.892 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:54 smithi050 podman[37922]: 2024-02-14 14:54:54.627764207 +0000 UTC m=+0.309011768 container init a8c20e957e06ecb987229afb400ade2bb7c10641ddb94cd6525197045cdeb26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T14:54:54.892 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:54 smithi050 podman[37922]: 2024-02-14 14:54:54.630418213 +0000 UTC m=+0.311665755 container start a8c20e957e06ecb987229afb400ade2bb7c10641ddb94cd6525197045cdeb26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-14T14:54:54.893 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:54 smithi050 ceph-mgr[37941]: -- 172.21.15.50:0/731682202 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d03e947520 con 0x55d03e91cc00 2024-02-14T14:54:54.893 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:54 smithi050 bash[37922]: a8c20e957e06ecb987229afb400ade2bb7c10641ddb94cd6525197045cdeb26e 2024-02-14T14:54:54.893 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:54 smithi050 systemd[1]: Started Ceph mgr.y for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T14:54:55.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_4 2024-02-14T14:54:55.557 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:54:55.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:55.196+0000 7f1d0854c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T14:54:55.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:55.295+0000 7f1d0854c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T14:54:55.914 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:55.568+0000 7f11e1485200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T14:54:55.915 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:55.703+0000 7f11e1485200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T14:54:55.915 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-mon[37057]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:55.915 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.915 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.915 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.915 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.915 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:55.995 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:55 smithi017 ceph-mon[32549]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:55.995 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.995 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.995 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.995 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:55.995 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:56.087 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:55.989+0000 7f1d0854c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T14:54:56.087 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-mon[36512]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:56.087 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:56.087 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:56.087 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:56.087 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:56.088 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:56.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:56.044+0000 7f11e1485200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T14:54:56.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:56.133+0000 7f11e1485200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T14:54:56.358 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:56.085+0000 7f1d0854c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T14:54:56.358 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:56.242+0000 7f1d0854c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T14:54:56.358 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:56.356+0000 7f1d0854c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T14:54:56.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:56.442+0000 7f1d0854c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T14:54:56.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:56.342+0000 7f11e1485200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:54:56.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:56.524+0000 7f11e1485200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T14:54:57.021 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:56.719+0000 7f1d0854c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T14:54:57.021 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:56.832+0000 7f1d0854c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T14:54:57.021 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:56.922+0000 7f1d0854c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T14:54:57.022 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:57.020+0000 7f1d0854c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T14:54:57.059 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:56.804+0000 7f11e1485200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:54:57.059 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:57.059+0000 7f11e1485200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T14:54:57.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 -- 172.21.15.17:0/1490715243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 msgr2=0x7f14001022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:57.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 --2- 172.21.15.17:0/1490715243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f14001022b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13f40099c0 tx=0x7f13f4009cd0 comp rx=0 tx=0).stop 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 -- 172.21.15.17:0/1490715243 shutdown_connections 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 --2- 172.21.15.17:0/1490715243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14001027f0 0x7f14001090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 --2- 172.21.15.17:0/1490715243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 0x7f1400101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 --2- 172.21.15.17:0/1490715243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f14001022b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 -- 172.21.15.17:0/1490715243 >> 172.21.15.17:0/1490715243 conn(0x7f14000fc1f0 msgr2=0x7f14000fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 -- 172.21.15.17:0/1490715243 shutdown_connections 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 -- 172.21.15.17:0/1490715243 wait complete. 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 Processor -- start 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.121+0000 7f14063ba700 1 -- start start 2024-02-14T14:54:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f14063ba700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 0x7f140006a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f14063ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f140019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f14063ba700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14001027f0 0x7f140019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f14063ba700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019f860 con 0x7f1400101e50 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f14063ba700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019f9d0 con 0x7f1400100c50 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f14063ba700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140019fb90 con 0x7f14001027f0 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f13ff7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f140019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f13fffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 0x7f140006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f13fffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 0x7f140006a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:38744/0 (socket says 172.21.15.17:38744) 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f13fffff700 1 -- 172.21.15.17:0/426973440 learned_addr learned my addr 172.21.15.17:0/426973440 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:54:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f13ff7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f140019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34650/0 (socket says 172.21.15.17:34650) 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f1404957700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14001027f0 0x7f140019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f1404957700 1 -- 172.21.15.17:0/426973440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 msgr2=0x7f140006a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f1404957700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 0x7f140006a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f1404957700 1 -- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 msgr2=0x7f140019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f1404957700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f140019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f1404957700 1 -- 172.21.15.17:0/426973440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40095f0 con 0x7f14001027f0 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f13ff7fe700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f140019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.122+0000 7f13fffff700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 0x7f140006a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.123+0000 7f1404957700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14001027f0 0x7f140019f0c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13f800efc0 tx=0x7f13f800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:57.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.123+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8018070 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.123+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13f800f040 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.123+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140019fe10 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.123+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f800cd10 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.123+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14001a0230 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.124+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f13f8004000 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.124+0000 7f13fd7fa700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f13e80391c0 0x7f13e803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.124+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f13f804b870 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.124+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1400108010 con 0x7f14001027f0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.124+0000 7f13fffff700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f13e80391c0 0x7f13e803b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:54:57.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.124+0000 7f13fffff700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f13e80391c0 0x7f13e803b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13f00105c0 tx=0x7f13f000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:54:57.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.133+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13f8005000 con 0x7f14001027f0 2024-02-14T14:54:57.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:57.349+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f140002d880 con 0x7f13e80391c0 2024-02-14T14:54:57.392 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:57.201+0000 7f1d0854c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T14:54:57.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: Reconfiguring mgr.x (unknown last config time)... 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:57.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:57 smithi050 ceph-mon[36512]: Reconfiguring daemon mgr.x on smithi017 2024-02-14T14:54:57.484 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:57.298+0000 7f11e1485200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T14:54:57.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:57.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:57.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: Reconfiguring mgr.x (unknown last config time)... 2024-02-14T14:54:57.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:57.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:57.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:57.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:57 smithi076 ceph-mon[37057]: Reconfiguring daemon mgr.x on smithi017 2024-02-14T14:54:57.487 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.487 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.487 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.487 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.487 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:57.488 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:57.488 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:57.488 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: Reconfiguring mgr.x (unknown last config time)... 2024-02-14T14:54:57.488 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T14:54:57.488 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T14:54:57.488 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:57.488 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:57 smithi017 ceph-mon[32549]: Reconfiguring daemon mgr.x on smithi017 2024-02-14T14:54:58.345 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:58.092+0000 7f11e1485200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.415 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.416 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:58.416 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:58.416 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:58.416 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:58.229+0000 7f1d0854c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T14:54:58.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:58.365+0000 7f1d0854c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T14:54:58.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:58.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.659 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:54:58.659 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:54:58.659 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:54:58.659 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:54:58.659 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:58.660 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.660 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.660 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.660 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:54:58.660 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:54:58.660 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:54:58.660 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:54:58.970 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:58.658+0000 7f11e1485200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T14:54:58.970 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:58.755+0000 7f11e1485200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T14:54:58.970 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 14:54:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T14:54:58.970+0000 7f11e1485200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T14:54:59.020 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:58.724+0000 7f1d0854c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T14:54:59.020 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:58.817+0000 7f1d0854c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T14:54:59.020 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:59.019+0000 7f1d0854c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:54:59.344 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:59.207+0000 7f1d0854c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T14:54:59.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:54:59.352+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f13f8004ad0 con 0x7f14001027f0 2024-02-14T14:54:59.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:59 smithi017 ceph-mon[32549]: Standby manager daemon z started 2024-02-14T14:54:59.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:59 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T14:54:59.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:59 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T14:54:59.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:59 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T14:54:59.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:54:59 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T14:54:59.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:59.498+0000 7f1d0854c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T14:54:59.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-mon[36512]: Standby manager daemon z started 2024-02-14T14:54:59.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T14:54:59.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T14:54:59.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T14:54:59.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T14:54:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:59 smithi076 ceph-mon[37057]: Standby manager daemon z started 2024-02-14T14:54:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:59 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T14:54:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:59 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T14:54:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:59 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T14:54:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:54:59 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3305870525' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T14:55:00.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:54:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:54:59.766+0000 7f1d0854c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T14:55:00.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:55:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:55:00.007+0000 7f1d0854c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T14:55:00.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:00 smithi050 ceph-mon[36512]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:00.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:00 smithi050 ceph-mon[36512]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-14T14:55:00.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T14:55:00.711 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:00 smithi017 ceph-mon[32549]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:00.715 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:00 smithi017 ceph-mon[32549]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-14T14:55:00.715 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T14:55:00.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:00 smithi076 ceph-mon[37057]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:00.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:00 smithi076 ceph-mon[37057]: mgrmap e14: x(active, since 76s), standbys: z 2024-02-14T14:55:00.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T14:55:01.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:55:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:55:00.839+0000 7f1d0854c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T14:55:01.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1519525095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "423af2fd-89fb-4e39-913c-be53474fe012"}]: dispatch 2024-02-14T14:55:01.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1519525095' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "423af2fd-89fb-4e39-913c-be53474fe012"}]': finished 2024-02-14T14:55:01.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-mon[36512]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T14:55:01.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:01.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/764813420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:01.643 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:55:01.431+0000 7f1d0854c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T14:55:01.643 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:55:01.532+0000 7f1d0854c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T14:55:01.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:01 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1519525095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "423af2fd-89fb-4e39-913c-be53474fe012"}]: dispatch 2024-02-14T14:55:01.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:01 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1519525095' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "423af2fd-89fb-4e39-913c-be53474fe012"}]': finished 2024-02-14T14:55:01.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:01 smithi076 ceph-mon[37057]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T14:55:01.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:01 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:01.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:01 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/764813420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:01.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:01 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1519525095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "423af2fd-89fb-4e39-913c-be53474fe012"}]: dispatch 2024-02-14T14:55:01.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:01 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1519525095' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "423af2fd-89fb-4e39-913c-be53474fe012"}]': finished 2024-02-14T14:55:01.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:01 smithi017 ceph-mon[32549]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T14:55:01.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:01 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:01.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:01 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/764813420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:02.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 14:55:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T14:55:01.752+0000 7f1d0854c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T14:55:02.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:02.469+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mon.1 v2:172.21.15.76:3300/0 8 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f13f8007500 con 0x7f14001027f0 2024-02-14T14:55:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:02 smithi076 ceph-mon[37057]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:02 smithi076 ceph-mon[37057]: Standby manager daemon y started 2024-02-14T14:55:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:02 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-14T14:55:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:02 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T14:55:02.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:02 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-14T14:55:02.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:02 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T14:55:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:02 smithi017 ceph-mon[32549]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:02 smithi017 ceph-mon[32549]: Standby manager daemon y started 2024-02-14T14:55:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:02 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-14T14:55:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:02 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T14:55:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:02 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-14T14:55:02.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:02 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T14:55:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:02 smithi050 ceph-mon[36512]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:02 smithi050 ceph-mon[36512]: Standby manager daemon y started 2024-02-14T14:55:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:02 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-14T14:55:02.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:02 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T14:55:02.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:02 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-14T14:55:02.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:02 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/3120185194' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T14:55:03.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:03 smithi076 ceph-mon[37057]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-02-14T14:55:03.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:03 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T14:55:03.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:03 smithi017 ceph-mon[32549]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-02-14T14:55:03.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:03 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T14:55:03.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:03 smithi050 ceph-mon[36512]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-02-14T14:55:03.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:03 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T14:55:04.720 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:04 smithi017 ceph-mon[32549]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:04.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:04 smithi076 ceph-mon[37057]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:04.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:04 smithi050 ceph-mon[36512]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:06.716 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:06 smithi017 ceph-mon[32549]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:06.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:06 smithi076 ceph-mon[37057]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:06.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:06 smithi050 ceph-mon[36512]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:08.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:08 smithi076 ceph-mon[37057]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:08.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T14:55:08.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:08.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:08 smithi017 ceph-mon[32549]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:08.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T14:55:08.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:08.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:08 smithi050 ceph-mon[36512]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:08.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T14:55:08.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:09.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:09 smithi017 ceph-mon[32549]: Deploying daemon osd.0 on smithi017 2024-02-14T14:55:09.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:09 smithi076 ceph-mon[37057]: Deploying daemon osd.0 on smithi017 2024-02-14T14:55:09.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:09 smithi050 ceph-mon[36512]: Deploying daemon osd.0 on smithi017 2024-02-14T14:55:10.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:10 smithi076 ceph-mon[37057]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:10.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:10 smithi017 ceph-mon[32549]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:10.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:10 smithi050 ceph-mon[36512]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:12.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:12 smithi017 ceph-mon[32549]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:12.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:12 smithi050 ceph-mon[36512]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:12 smithi076 ceph-mon[37057]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:14.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:14 smithi017 ceph-mon[32549]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:14.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:14.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:14.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:14.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:14 smithi050 ceph-mon[36512]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:14.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:14.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:14.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:14.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:14 smithi076 ceph-mon[37057]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:14.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:14.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:14.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:16.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:16 smithi017 ceph-mon[32549]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:16.621 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:16.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:16.622 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.706 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 0 on host 'smithi017' 2024-02-14T14:55:16.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.705+0000 7f13fd7fa700 1 -- 172.21.15.17:0/426973440 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f140002d880 con 0x7f13e80391c0 2024-02-14T14:55:16.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f13e80391c0 msgr2=0x7f13e803b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f13e80391c0 0x7f13e803b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13f00105c0 tx=0x7f13f000b4a0 comp rx=0 tx=0).stop 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14001027f0 msgr2=0x7f140019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14001027f0 0x7f140019f0c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13f800efc0 tx=0x7f13f800c3a0 comp rx=0 tx=0).stop 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 shutdown_connections 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f13e80391c0 0x7f13e803b680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14001027f0 0x7f140019f0c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1400100c50 0x7f140006a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 --2- 172.21.15.17:0/426973440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1400101e50 0x7f140019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.707+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 >> 172.21.15.17:0/426973440 conn(0x7f14000fc1f0 msgr2=0x7f1400105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:55:16.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.708+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 shutdown_connections 2024-02-14T14:55:16.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:55:16.708+0000 7f14063ba700 1 -- 172.21.15.17:0/426973440 wait complete. 2024-02-14T14:55:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:16 smithi076 ceph-mon[37057]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:16 smithi050 ceph-mon[36512]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:17.339 DEBUG:teuthology.orchestra.run.smithi017:osd.0> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.0.service 2024-02-14T14:55:17.341 INFO:tasks.cephadm:Deploying osd.1 on smithi050 with /dev/vg_nvme/lv_4... 2024-02-14T14:55:17.341 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-14T14:55:17.599 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:55:17.942 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:17 smithi050 ceph-mon[36512]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:17.943 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:17.943 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:17.943 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:17.943 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:17 smithi050 ceph-mon[36512]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T14:55:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:17 smithi076 ceph-mon[37057]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:17 smithi076 ceph-mon[37057]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T14:55:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:17 smithi017 ceph-mon[32549]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:18.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:18.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:17 smithi017 ceph-mon[32549]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T14:55:18.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:18 smithi017 ceph-mon[32549]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T14:55:18.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:18 smithi017 ceph-mon[32549]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T14:55:18.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:18 smithi017 ceph-mon[32549]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:55:18.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:18 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:18.767 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:55:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0[47600]: 2024-02-14T14:55:18.669+0000 7f4e39ad4700 -1 osd.0 0 waiting for initial osdmap 2024-02-14T14:55:18.767 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:55:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0[47600]: 2024-02-14T14:55:18.673+0000 7f4e358ce700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:55:18.850 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:18 smithi050 ceph-mon[36512]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T14:55:18.850 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:18 smithi050 ceph-mon[36512]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T14:55:18.850 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:18 smithi050 ceph-mon[36512]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:55:18.850 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:18 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:18 smithi076 ceph-mon[37057]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T14:55:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:18 smithi076 ceph-mon[37057]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T14:55:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:18 smithi076 ceph-mon[37057]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:55:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:18 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:19.429 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-14T14:55:19.429 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-14T14:55:19.430 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-14T14:55:19.430 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-14T14:55:19.430 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00924325 s, 1.1 GB/s 2024-02-14T14:55:19.430 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-14T14:55:19.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:19 smithi017 ceph-mon[32549]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:19.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:19 smithi017 ceph-mon[32549]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:55:19.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:19 smithi017 ceph-mon[32549]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T14:55:19.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:19 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:19.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:19 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:19 smithi076 ceph-mon[37057]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:19 smithi076 ceph-mon[37057]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:55:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:19 smithi076 ceph-mon[37057]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T14:55:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:19 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:19 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:20.036 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:19 smithi050 ceph-mon[36512]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:20.036 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:19 smithi050 ceph-mon[36512]: from='osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:55:20.036 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:19 smithi050 ceph-mon[36512]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T14:55:20.037 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:19 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:20.037 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:19 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:20.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-02-14T14:55:20.318 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:55:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] boot 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi017 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:55:20.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi017 to 17093M 2024-02-14T14:55:20.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:20.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:20.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:20.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] boot 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi017 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi017 to 17093M 2024-02-14T14:55:21.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:21.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:21.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: osd.0 [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] boot 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi017 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.118 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:55:21.118 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi017 to 17093M 2024-02-14T14:55:21.118 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.118 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:21.118 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:21.118 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 -- 172.21.15.50:0/2515461600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c102810 msgr2=0x7f8c2c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 --2- 172.21.15.50:0/2515461600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c102810 0x7f8c2c104c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009990 tx=0x7f8c20009ca0 comp rx=0 tx=0).stop 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 -- 172.21.15.50:0/2515461600 shutdown_connections 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 --2- 172.21.15.50:0/2515461600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c2c107a70 0x7f8c2c109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 --2- 172.21.15.50:0/2515461600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c102810 0x7f8c2c104c00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 --2- 172.21.15.50:0/2515461600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c2c105140 0x7f8c2c107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 -- 172.21.15.50:0/2515461600 >> 172.21.15.50:0/2515461600 conn(0x7f8c2c0fc240 msgr2=0x7f8c2c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 -- 172.21.15.50:0/2515461600 shutdown_connections 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 -- 172.21.15.50:0/2515461600 wait complete. 2024-02-14T14:55:21.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 Processor -- start 2024-02-14T14:55:21.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.881+0000 7f8c3233a700 1 -- start start 2024-02-14T14:55:21.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c3233a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c2c102810 0x7f8c2c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:21.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c3233a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c105140 0x7f8c2c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:21.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c3233a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c2c107a70 0x7f8c2c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:21.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c3233a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19f7c0 con 0x7f8c2c107a70 2024-02-14T14:55:21.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c3233a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19f900 con 0x7f8c2c105140 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c3233a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19fac0 con 0x7f8c2c102810 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c105140 0x7f8c2c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c105140 0x7f8c2c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54300/0 (socket says 172.21.15.50:54300) 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 -- 172.21.15.50:0/389082328 learned_addr learned my addr 172.21.15.50:0/389082328 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c308d7700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c2c107a70 0x7f8c2c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2bfff700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c2c102810 0x7f8c2c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 -- 172.21.15.50:0/389082328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c2c102810 msgr2=0x7f8c2c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c2c102810 0x7f8c2c197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 -- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c2c107a70 msgr2=0x7f8c2c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c2c107a70 0x7f8c2c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:21.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 -- 172.21.15.50:0/389082328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200095f0 con 0x7f8c2c105140 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2bfff700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c2c102810 0x7f8c2c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c308d7700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c2c107a70 0x7f8c2c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.882+0000 7f8c2b7fe700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c105140 0x7f8c2c1981b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00c9b0 tx=0x7f8c1c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.883+0000 7f8c297fa700 1 -- 172.21.15.50:0/389082328 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c007b00 con 0x7f8c2c105140 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.883+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c19fda0 con 0x7f8c2c105140 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.883+0000 7f8c297fa700 1 -- 172.21.15.50:0/389082328 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c007c60 con 0x7f8c2c105140 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.883+0000 7f8c297fa700 1 -- 172.21.15.50:0/389082328 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c018730 con 0x7f8c2c105140 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.883+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c1a0210 con 0x7f8c2c105140 2024-02-14T14:55:21.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.884+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c0689e0 con 0x7f8c2c105140 2024-02-14T14:55:21.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.885+0000 7f8c297fa700 1 -- 172.21.15.50:0/389082328 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f8c1c018890 con 0x7f8c2c105140 2024-02-14T14:55:21.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.885+0000 7f8c297fa700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8c140a2140 0x7f8c140a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:21.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.886+0000 7f8c297fa700 1 -- 172.21.15.50:0/389082328 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f8c1c0cc660 con 0x7f8c2c105140 2024-02-14T14:55:21.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.886+0000 7f8c2bfff700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8c140a2140 0x7f8c140a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:21.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.886+0000 7f8c2bfff700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8c140a2140 0x7f8c140a4600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000c00 tx=0x7f8c20005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:55:21.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:21.889+0000 7f8c297fa700 1 -- 172.21.15.50:0/389082328 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c1c097090 con 0x7f8c2c105140 2024-02-14T14:55:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:21 smithi076 ceph-mon[37057]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:21 smithi076 ceph-mon[37057]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T14:55:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:21 smithi017 ceph-mon[32549]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:21 smithi017 ceph-mon[32549]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T14:55:22.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:22.100+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c2c02d960 con 0x7f8c140a2140 2024-02-14T14:55:22.102 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:21 smithi050 ceph-mon[36512]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T14:55:22.102 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:21 smithi050 ceph-mon[36512]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T14:55:22.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:22 smithi076 ceph-mon[37057]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:55:22.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:22 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:55:22.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:22 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:55:22.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:22 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:22.990 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:22 smithi050 ceph-mon[36512]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:55:22.990 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:22 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:55:22.990 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:22 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:55:22.991 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:22 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:22 smithi017 ceph-mon[32549]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:55:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:55:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:55:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:23.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:23 smithi050 ceph-mon[36512]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:23.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:23 smithi076 ceph-mon[37057]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:24.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:23 smithi017 ceph-mon[32549]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:24 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/1439611589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]: dispatch 2024-02-14T14:55:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:24 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]: dispatch 2024-02-14T14:55:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:24 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]': finished 2024-02-14T14:55:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:24 smithi076 ceph-mon[37057]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T14:55:24.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:24 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/1439611589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]: dispatch 2024-02-14T14:55:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:24 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]: dispatch 2024-02-14T14:55:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:24 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]': finished 2024-02-14T14:55:25.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:24 smithi017 ceph-mon[32549]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T14:55:25.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:24 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/1439611589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]: dispatch 2024-02-14T14:55:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:24 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]: dispatch 2024-02-14T14:55:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:24 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f59afa4-e5b3-462c-89a0-e9552f262730"}]': finished 2024-02-14T14:55:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:24 smithi050 ceph-mon[36512]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T14:55:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:25 smithi076 ceph-mon[37057]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:25 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/3930294854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:26.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:25 smithi017 ceph-mon[32549]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:26.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:25 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/3930294854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:25 smithi050 ceph-mon[36512]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:25 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/3930294854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:27 smithi076 ceph-mon[37057]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:27 smithi017 ceph-mon[32549]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:28.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:27 smithi050 ceph-mon[36512]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:29.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:29 smithi050 ceph-mon[36512]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:29 smithi076 ceph-mon[37057]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:30.004 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:29 smithi017 ceph-mon[32549]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:32.727 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:32 smithi050 ceph-mon[36512]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:32.727 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T14:55:32.727 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:32 smithi076 ceph-mon[37057]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T14:55:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:32.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:32 smithi017 ceph-mon[32549]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:32.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T14:55:32.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:33 smithi076 ceph-mon[37057]: Deploying daemon osd.1 on smithi050 2024-02-14T14:55:33.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:33 smithi017 ceph-mon[32549]: Deploying daemon osd.1 on smithi050 2024-02-14T14:55:33.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:33 smithi050 ceph-mon[36512]: Deploying daemon osd.1 on smithi050 2024-02-14T14:55:34.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:34 smithi076 ceph-mon[37057]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:34.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:34 smithi017 ceph-mon[32549]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:34.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:34 smithi050 ceph-mon[36512]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:36.704 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:36 smithi050 ceph-mon[36512]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:36.733 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:36 smithi076 ceph-mon[37057]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:36.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:36 smithi017 ceph-mon[32549]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:38.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:38 smithi050 ceph-mon[36512]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:38.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:38.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:38.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:38 smithi076 ceph-mon[37057]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:38 smithi017 ceph-mon[32549]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:40.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:40 smithi050 ceph-mon[36512]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:40.725 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:40.725 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:40.725 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:40.725 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:40.725 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:40 smithi076 ceph-mon[37057]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:40 smithi017 ceph-mon[32549]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:41.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.066+0000 7f8c297fa700 1 -- 172.21.15.50:0/389082328 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8c2c02d960 con 0x7f8c140a2140 2024-02-14T14:55:41.068 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 1 on host 'smithi050' 2024-02-14T14:55:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8c140a2140 msgr2=0x7f8c140a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8c140a2140 0x7f8c140a4600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000c00 tx=0x7f8c20005fb0 comp rx=0 tx=0).stop 2024-02-14T14:55:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c105140 msgr2=0x7f8c2c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c105140 0x7f8c2c1981b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00c9b0 tx=0x7f8c1c00cd70 comp rx=0 tx=0).stop 2024-02-14T14:55:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 shutdown_connections 2024-02-14T14:55:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8c140a2140 0x7f8c140a4600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c2c107a70 0x7f8c2c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c2c102810 0x7f8c2c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 --2- 172.21.15.50:0/389082328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c2c105140 0x7f8c2c1981b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 >> 172.21.15.50:0/389082328 conn(0x7f8c2c0fc240 msgr2=0x7f8c2c1030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:55:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 shutdown_connections 2024-02-14T14:55:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:55:41.069+0000 7f8c3233a700 1 -- 172.21.15.50:0/389082328 wait complete. 2024-02-14T14:55:41.695 DEBUG:teuthology.orchestra.run.smithi050:osd.1> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.1.service 2024-02-14T14:55:41.698 INFO:tasks.cephadm:Deploying osd.2 on smithi076 with /dev/vg_nvme/lv_4... 2024-02-14T14:55:41.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-14T14:55:41.955 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:55:42.322 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:42 smithi076 ceph-mon[37057]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:42.322 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:42.322 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:42.322 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:42.322 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:42 smithi076 ceph-mon[37057]: from='osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T14:55:42.322 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:42 smithi076 ceph-mon[37057]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T14:55:42.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:42 smithi050 ceph-mon[36512]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:42.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:42.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:42.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:42.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:42 smithi050 ceph-mon[36512]: from='osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T14:55:42.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:42 smithi050 ceph-mon[36512]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T14:55:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:42 smithi017 ceph-mon[32549]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:42.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:42.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:55:42.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:42 smithi017 ceph-mon[32549]: from='osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T14:55:42.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:42 smithi017 ceph-mon[32549]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T14:55:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:43 smithi050 ceph-mon[36512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T14:55:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:43 smithi050 ceph-mon[36512]: from='osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:55:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:43 smithi050 ceph-mon[36512]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T14:55:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:43 smithi050 ceph-mon[36512]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:55:43.143 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:55:43 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1[41151]: 2024-02-14T14:55:43.052+0000 7faa6d7ce700 -1 osd.1 0 waiting for initial osdmap 2024-02-14T14:55:43.143 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:55:43 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1[41151]: 2024-02-14T14:55:43.055+0000 7faa695c8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:55:43.294 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:43 smithi076 ceph-mon[37057]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T14:55:43.294 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:43 smithi076 ceph-mon[37057]: from='osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:55:43.294 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:43 smithi076 ceph-mon[37057]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T14:55:43.294 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:43.295 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:43 smithi076 ceph-mon[37057]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:55:43.483 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:43 smithi017 ceph-mon[32549]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T14:55:43.484 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:43 smithi017 ceph-mon[32549]: from='osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:55:43.484 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:43 smithi017 ceph-mon[32549]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T14:55:43.484 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:43.484 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:43 smithi017 ceph-mon[32549]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:55:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-14T14:55:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-14T14:55:43.845 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-02-14T14:55:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-02-14T14:55:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130231 s, 805 MB/s 2024-02-14T14:55:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-02-14T14:55:44.308 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:44 smithi050 ceph-mon[36512]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:44.309 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:44 smithi050 ceph-mon[36512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:55:44.309 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:44 smithi050 ceph-mon[36512]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T14:55:44.309 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:44.309 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:44.414 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:44 smithi076 ceph-mon[37057]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:44.414 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:44 smithi076 ceph-mon[37057]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:55:44.414 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:44 smithi076 ceph-mon[37057]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T14:55:44.414 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:44.415 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:44.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_4 2024-02-14T14:55:44.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:44 smithi017 ceph-mon[32549]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:44.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:44 smithi017 ceph-mon[32549]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:55:44.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:44 smithi017 ceph-mon[32549]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T14:55:44.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:44.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:44.688 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:55:45.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] boot 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:55:45.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:45.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:45.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:55:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:55:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] boot 2024-02-14T14:55:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T14:55:45.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:45.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:55:45.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:45.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:45.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:55:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:55:45.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: osd.1 [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] boot 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:55:45.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:55:46.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.274+0000 7fa89ba6a700 1 -- 172.21.15.76:0/4087855001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894101d50 msgr2=0x7fa894102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.274+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/4087855001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894101d50 0x7fa894102170 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa884009990 tx=0x7fa884009ca0 comp rx=0 tx=0).stop 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.274+0000 7fa89ba6a700 1 -- 172.21.15.76:0/4087855001 shutdown_connections 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.274+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/4087855001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8941038f0 0x7fa89410a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.274+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/4087855001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa894101d50 0x7fa894102170 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.274+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/4087855001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa894102f50 0x7fa8941033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.274+0000 7fa89ba6a700 1 -- 172.21.15.76:0/4087855001 >> 172.21.15.76:0/4087855001 conn(0x7fa8940fd330 msgr2=0x7fa8940ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 -- 172.21.15.76:0/4087855001 shutdown_connections 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 -- 172.21.15.76:0/4087855001 wait complete. 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 Processor -- start 2024-02-14T14:55:46.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 -- start start 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894101d50 0x7fa894197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa894102f50 0x7fa894198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8941038f0 0x7fa89419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419f870 con 0x7fa894102f50 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419f9b0 con 0x7fa894101d50 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89ba6a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419fb70 con 0x7fa8941038f0 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89a007700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8941038f0 0x7fa89419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89a007700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8941038f0 0x7fa89419f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51078/0 (socket says 172.21.15.76:51078) 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.275+0000 7fa89a007700 1 -- 172.21.15.76:0/59050917 learned_addr learned my addr 172.21.15.76:0/59050917 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:55:46.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa899005700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa894102f50 0x7fa894198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa899806700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894101d50 0x7fa894197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89a007700 1 -- 172.21.15.76:0/59050917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894101d50 msgr2=0x7fa894197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89a007700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894101d50 0x7fa894197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89a007700 1 -- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa894102f50 msgr2=0x7fa894198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89a007700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa894102f50 0x7fa894198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89a007700 1 -- 172.21.15.76:0/59050917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840095f0 con 0x7fa8941038f0 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa899806700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894101d50 0x7fa894197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89a007700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8941038f0 0x7fa89419f2a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00ada0 tx=0x7fa88c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa88affd700 1 -- 172.21.15.76:0/59050917 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c00f070 con 0x7fa8941038f0 2024-02-14T14:55:46.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa88affd700 1 -- 172.21.15.76:0/59050917 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa88c007b30 con 0x7fa8941038f0 2024-02-14T14:55:46.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89419fe50 con 0x7fa8941038f0 2024-02-14T14:55:46.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa88affd700 1 -- 172.21.15.76:0/59050917 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0053c0 con 0x7fa8941038f0 2024-02-14T14:55:46.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.276+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8941a0210 con 0x7fa8941038f0 2024-02-14T14:55:46.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.277+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8940689e0 con 0x7fa8941038f0 2024-02-14T14:55:46.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.279+0000 7fa88affd700 1 -- 172.21.15.76:0/59050917 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7fa88c005620 con 0x7fa8941038f0 2024-02-14T14:55:46.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.279+0000 7fa88affd700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8800a20c0 0x7fa8800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:55:46.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.280+0000 7fa88affd700 1 -- 172.21.15.76:0/59050917 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fa88c0cc020 con 0x7fa8941038f0 2024-02-14T14:55:46.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.280+0000 7fa899806700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8800a20c0 0x7fa8800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:55:46.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.280+0000 7fa899806700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8800a20c0 0x7fa8800a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa884000c00 tx=0x7fa884005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:55:46.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.283+0000 7fa88affd700 1 -- 172.21.15.76:0/59050917 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa88c0968d0 con 0x7fa8941038f0 2024-02-14T14:55:46.284 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:46 smithi076 ceph-mon[37057]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:46.284 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:46 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi050 2024-02-14T14:55:46.284 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:46 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi050 to 17084M 2024-02-14T14:55:46.284 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:46 smithi076 ceph-mon[37057]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T14:55:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:46 smithi050 ceph-mon[36512]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:46 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi050 2024-02-14T14:55:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:46 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi050 to 17084M 2024-02-14T14:55:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:46 smithi050 ceph-mon[36512]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T14:55:46.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:55:46.482+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa89402d8b0 con 0x7fa8800a20c0 2024-02-14T14:55:46.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:46 smithi017 ceph-mon[32549]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T14:55:46.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:46 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi050 2024-02-14T14:55:46.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:46 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi050 to 17084M 2024-02-14T14:55:46.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:46 smithi017 ceph-mon[32549]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T14:55:47.375 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:47 smithi076 ceph-mon[37057]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:55:47.375 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:55:47.376 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:55:47.376 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:47 smithi050 ceph-mon[36512]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:55:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:55:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:55:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:47.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:47 smithi017 ceph-mon[32549]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:55:47.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:55:47.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:55:47.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:48.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:48 smithi076 ceph-mon[37057]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:48.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:48 smithi050 ceph-mon[36512]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:48.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:48 smithi017 ceph-mon[32549]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:49.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:49 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/3324802247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]: dispatch 2024-02-14T14:55:49.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:49 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]: dispatch 2024-02-14T14:55:49.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:49 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]': finished 2024-02-14T14:55:49.419 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:49 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/3324802247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]: dispatch 2024-02-14T14:55:49.419 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:49 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]: dispatch 2024-02-14T14:55:49.419 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:49 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]': finished 2024-02-14T14:55:49.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:49 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/3324802247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]: dispatch 2024-02-14T14:55:49.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:49 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]: dispatch 2024-02-14T14:55:49.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:49 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1d2809-6bb1-46aa-9ba8-dd08192a591b"}]': finished 2024-02-14T14:55:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:50 smithi050 ceph-mon[36512]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:50 smithi050 ceph-mon[36512]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T14:55:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:55:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:50 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/1538223422' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:50.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:50 smithi076 ceph-mon[37057]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:50.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:50 smithi076 ceph-mon[37057]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T14:55:50.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:55:50.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:50 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/1538223422' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:50 smithi017 ceph-mon[32549]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:50 smithi017 ceph-mon[32549]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T14:55:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:55:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:50 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/1538223422' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:55:52.723 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:52 smithi076 ceph-mon[37057]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:52 smithi017 ceph-mon[32549]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:52 smithi050 ceph-mon[36512]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:54.683 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:54 smithi076 ceph-mon[37057]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:54 smithi017 ceph-mon[32549]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:54.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:54 smithi050 ceph-mon[36512]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:56 smithi076 ceph-mon[37057]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T14:55:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:56 smithi017 ceph-mon[32549]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T14:55:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:56 smithi050 ceph-mon[36512]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T14:55:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:55:57.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:57 smithi076 ceph-mon[37057]: Deploying daemon osd.2 on smithi076 2024-02-14T14:55:57.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:57 smithi017 ceph-mon[32549]: Deploying daemon osd.2 on smithi076 2024-02-14T14:55:57.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:57 smithi050 ceph-mon[36512]: Deploying daemon osd.2 on smithi076 2024-02-14T14:55:58.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:55:58 smithi076 ceph-mon[37057]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:58.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:55:58 smithi017 ceph-mon[32549]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:55:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:55:58 smithi050 ceph-mon[36512]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:00.674 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:00 smithi076 ceph-mon[37057]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:00.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:00 smithi017 ceph-mon[32549]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:00 smithi050 ceph-mon[36512]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:02.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:02 smithi017 ceph-mon[32549]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:02.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:02 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:02.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:02 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:02.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:02 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:02.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:02 smithi050 ceph-mon[36512]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:02.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:02 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:02.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:02 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:02.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:02 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:02 smithi076 ceph-mon[37057]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:02 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:02 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:02.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:02 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:04.391 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:04 smithi076 ceph-mon[37057]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:04.391 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.391 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.391 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:04.391 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:04.391 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:04 smithi017 ceph-mon[32549]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:04.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:04.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:04.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:04 smithi050 ceph-mon[36512]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:04.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:04.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:04.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:04.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.800+0000 7fa88affd700 1 -- 172.21.15.76:0/59050917 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa89402d8b0 con 0x7fa8800a20c0 2024-02-14T14:56:04.802 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 2 on host 'smithi076' 2024-02-14T14:56:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8800a20c0 msgr2=0x7fa8800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8800a20c0 0x7fa8800a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa884000c00 tx=0x7fa884005fb0 comp rx=0 tx=0).stop 2024-02-14T14:56:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8941038f0 msgr2=0x7fa89419f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8941038f0 0x7fa89419f2a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00ada0 tx=0x7fa88c00c330 comp rx=0 tx=0).stop 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 shutdown_connections 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8800a20c0 0x7fa8800a4580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8941038f0 0x7fa89419f2a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894101d50 0x7fa894197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 --2- 172.21.15.76:0/59050917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa894102f50 0x7fa894198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 >> 172.21.15.76:0/59050917 conn(0x7fa8940fd330 msgr2=0x7fa894106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 shutdown_connections 2024-02-14T14:56:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:56:04.803+0000 7fa89ba6a700 1 -- 172.21.15.76:0/59050917 wait complete. 2024-02-14T14:56:05.417 DEBUG:teuthology.orchestra.run.smithi076:osd.2> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.2.service 2024-02-14T14:56:05.419 INFO:tasks.cephadm:Deploying osd.3 on smithi017 with /dev/vg_nvme/lv_3... 2024-02-14T14:56:05.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-14T14:56:05.670 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:56:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:05 smithi076 ceph-mon[37057]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:05 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:05 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:05 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:05 smithi076 ceph-mon[37057]: from='osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T14:56:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:05 smithi076 ceph-mon[37057]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T14:56:06.026 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:05 smithi017 ceph-mon[32549]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:06.026 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:05 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:06.026 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:05 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:06.026 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:05 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:06.026 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:05 smithi017 ceph-mon[32549]: from='osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T14:56:06.026 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:05 smithi017 ceph-mon[32549]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T14:56:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:05 smithi050 ceph-mon[36512]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:05 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:05 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:05 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:05 smithi050 ceph-mon[36512]: from='osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T14:56:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:05 smithi050 ceph-mon[36512]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T14:56:06.959 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:06 smithi076 ceph-mon[37057]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T14:56:06.959 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:06 smithi076 ceph-mon[37057]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T14:56:06.959 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:06 smithi076 ceph-mon[37057]: from='osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:56:06.959 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:06 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:06.959 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:06 smithi076 ceph-mon[37057]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:56:07.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:06 smithi017 ceph-mon[32549]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T14:56:07.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:06 smithi017 ceph-mon[32549]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T14:56:07.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:06 smithi017 ceph-mon[32549]: from='osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:56:07.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:06 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:07.105 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:06 smithi017 ceph-mon[32549]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:56:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:06 smithi050 ceph-mon[36512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T14:56:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:06 smithi050 ceph-mon[36512]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T14:56:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:06 smithi050 ceph-mon[36512]: from='osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:56:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:06 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:06 smithi050 ceph-mon[36512]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:56:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-14T14:56:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-14T14:56:07.699 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-02-14T14:56:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-02-14T14:56:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00813806 s, 1.3 GB/s 2024-02-14T14:56:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-02-14T14:56:07.805 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-mon[37057]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:07.805 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-mon[37057]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:56:07.805 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-mon[37057]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T14:56:07.805 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:07.805 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:07.805 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-mon[37057]: from='osd.2 ' entity='osd.2' 2024-02-14T14:56:07.805 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2[42220]: 2024-02-14T14:56:07.539+0000 7f1ae684a700 -1 osd.2 0 waiting for initial osdmap 2024-02-14T14:56:07.805 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2[42220]: 2024-02-14T14:56:07.543+0000 7f1ade63c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:56:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:07 smithi050 ceph-mon[36512]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:07 smithi050 ceph-mon[36512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:56:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:07 smithi050 ceph-mon[36512]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T14:56:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:07 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:08.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:07 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:08.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:07 smithi050 ceph-mon[36512]: from='osd.2 ' entity='osd.2' 2024-02-14T14:56:08.233 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:07 smithi017 ceph-mon[32549]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:08.233 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:07 smithi017 ceph-mon[32549]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:56:08.233 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:07 smithi017 ceph-mon[32549]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T14:56:08.233 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:07 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:08.233 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:07 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:08.233 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:07 smithi017 ceph-mon[32549]: from='osd.2 ' entity='osd.2' 2024-02-14T14:56:08.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_3 2024-02-14T14:56:08.506 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:56:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:56:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:56:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] boot 2024-02-14T14:56:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T14:56:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:09.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:09.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:09.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:09.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:56:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] boot 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:09.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: osd.2 [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] boot 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:09.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:09.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:09.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/3508188301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c54100c50 msgr2=0x7f6c54101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/3508188301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c54100c50 0x7f6c54101070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c500099c0 tx=0x7f6c50009cd0 comp rx=0 tx=0).stop 2024-02-14T14:56:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/3508188301 shutdown_connections 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/3508188301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c541027f0 0x7f6c541090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/3508188301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c54100c50 0x7f6c54101070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/3508188301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c54101e50 0x7f6c541022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/3508188301 >> 172.21.15.17:0/3508188301 conn(0x7f6c540fc1f0 msgr2=0x7f6c540fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/3508188301 shutdown_connections 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.114+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/3508188301 wait complete. 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 Processor -- start 2024-02-14T14:56:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 -- start start 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c54100c50 0x7f6c54197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c54101e50 0x7f6c54198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c541027f0 0x7f6c5419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419f8c0 con 0x7f6c541027f0 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419fa00 con 0x7f6c54100c50 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5b7f3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419fb90 con 0x7f6c54101e50 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c541027f0 0x7f6c5419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c541027f0 0x7f6c5419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39332/0 (socket says 172.21.15.17:39332) 2024-02-14T14:56:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 -- 172.21.15.17:0/351911112 learned_addr learned my addr 172.21.15.17:0/351911112 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c5958f700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c54100c50 0x7f6c54197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c58d8e700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c54101e50 0x7f6c54198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 -- 172.21.15.17:0/351911112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c54101e50 msgr2=0x7f6c54198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c54101e50 0x7f6c54198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 -- 172.21.15.17:0/351911112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c54100c50 msgr2=0x7f6c54197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c54100c50 0x7f6c54197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.115+0000 7f6c59d90700 1 -- 172.21.15.17:0/351911112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c500095f0 con 0x7f6c541027f0 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.116+0000 7f6c59d90700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c541027f0 0x7f6c5419f2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007f10 tx=0x7f6c4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.116+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c017070 con 0x7f6c541027f0 2024-02-14T14:56:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.116+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c4c00f040 con 0x7f6c541027f0 2024-02-14T14:56:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.116+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c005760 con 0x7f6c541027f0 2024-02-14T14:56:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.116+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5419fe70 con 0x7f6c541027f0 2024-02-14T14:56:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.116+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541a02e0 con 0x7f6c541027f0 2024-02-14T14:56:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.117+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c540689e0 con 0x7f6c541027f0 2024-02-14T14:56:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.118+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f6c4c0058c0 con 0x7f6c541027f0 2024-02-14T14:56:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.119+0000 7f6c4a7fc700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c400a21c0 0x7f6c400a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.119+0000 7f6c5958f700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c400a21c0 0x7f6c400a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:10.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.119+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f6c4c0cc630 con 0x7f6c541027f0 2024-02-14T14:56:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.119+0000 7f6c5958f700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c400a21c0 0x7f6c400a4680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c50006010 tx=0x7f6c50005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:56:10.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.122+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c4c096a20 con 0x7f6c541027f0 2024-02-14T14:56:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:09 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi076 2024-02-14T14:56:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:09 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi076 to 17080M 2024-02-14T14:56:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:09 smithi076 ceph-mon[37057]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T14:56:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T14:56:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:09 smithi076 ceph-mon[37057]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T14:56:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T14:56:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:09 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi076 2024-02-14T14:56:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:09 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi076 to 17080M 2024-02-14T14:56:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:09 smithi017 ceph-mon[32549]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T14:56:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T14:56:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:09 smithi017 ceph-mon[32549]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T14:56:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T14:56:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:10.334+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c5402d920 con 0x7f6c400a21c0 2024-02-14T14:56:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:09 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi076 2024-02-14T14:56:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:09 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi076 to 17080M 2024-02-14T14:56:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:09 smithi050 ceph-mon[36512]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T14:56:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T14:56:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T14:56:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:09 smithi050 ceph-mon[36512]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T14:56:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T14:56:11.203 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:10 smithi017 ceph-mon[32549]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:56:11.204 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:56:11.204 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:56:11.204 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:11.204 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:10 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T14:56:11.204 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:10 smithi017 ceph-mon[32549]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T14:56:11.204 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:10 smithi017 sudo[52435]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T14:56:11.204 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:10 smithi017 sudo[52435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:11.204 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:10 smithi017 sudo[52435]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:11.204 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:10 smithi017 sudo[52435]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:11.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:10 smithi076 ceph-mon[37057]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:56:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:10 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:56:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:10 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:56:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:10 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:10 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T14:56:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:10 smithi076 ceph-mon[37057]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T14:56:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:10 smithi050 ceph-mon[36512]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:56:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:10 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:56:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:10 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:56:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:10 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:10 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T14:56:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:10 smithi050 ceph-mon[36512]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T14:56:11.516 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:11 smithi017 sudo[52442]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T14:56:11.516 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:11 smithi017 sudo[52442]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:11.516 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:11 smithi017 sudo[52442]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:11.516 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 14:56:11 smithi017 sudo[52442]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:12.130 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:11 smithi050 sudo[45172]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T14:56:12.130 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:11 smithi050 sudo[45172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:12.130 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:11 smithi050 sudo[45172]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:12.130 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:11 smithi050 sudo[45172]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:12.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:12.243+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6c4c096df0 con 0x7f6c541027f0 2024-02-14T14:56:12.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 ceph-mon[32549]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:12.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:12.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 ceph-mon[32549]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T14:56:12.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:12.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:12 smithi050 ceph-mon[36512]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:12.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:12.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:12 smithi050 ceph-mon[36512]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T14:56:12.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:12.392 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:12 smithi050 sudo[45175]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T14:56:12.392 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:12 smithi050 sudo[45175]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:12.393 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:12 smithi050 sudo[45175]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:12.393 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 14:56:12 smithi050 sudo[45175]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:12.651 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46025]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T14:56:12.651 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:12.651 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46025]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:12.651 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46025]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:12.652 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:12 smithi076 ceph-mon[37057]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:12.652 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:12.652 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:12 smithi076 ceph-mon[37057]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T14:56:12.652 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:12.984 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46028]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T14:56:12.984 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46028]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:12.984 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46028]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:12.984 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 14:56:12 smithi076 sudo[46028]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:13.190 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 sudo[52530]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T14:56:13.191 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 sudo[52530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:13.191 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 sudo[52530]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:13.191 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:12 smithi017 sudo[52530]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:13.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:13 smithi017 sudo[52533]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T14:56:13.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:13 smithi017 sudo[52533]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:13.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:13 smithi017 sudo[52533]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:13.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:13 smithi017 sudo[52533]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:13.686 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46031]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T14:56:13.686 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:13.686 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46031]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:13.686 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46031]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:13.686 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46034]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T14:56:13.687 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46034]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:13.687 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46034]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:13.687 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:13 smithi076 sudo[46034]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:14.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45178]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T14:56:14.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:14.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45178]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:14.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45178]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45181]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T14:56:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T14:56:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45181]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T14:56:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 sudo[45181]: pam_unix(sudo:session): session closed for user root 2024-02-14T14:56:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:14.966 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-02-14T14:56:14.966 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:14.966 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:14.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:14.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:14.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:14.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:14.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:14.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:14.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-02-14T14:56:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:14.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:14.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:14.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:14.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:14.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1971784806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1e9407-028a-4d4b-af15-2f0d9b05669b"}]: dispatch 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1971784806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c1e9407-028a-4d4b-af15-2f0d9b05669b"}]': finished 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: osdmap e22: 4 total, 3 up, 4 in 2024-02-14T14:56:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:15.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:15.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1498506356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1971784806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1e9407-028a-4d4b-af15-2f0d9b05669b"}]: dispatch 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1971784806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c1e9407-028a-4d4b-af15-2f0d9b05669b"}]': finished 2024-02-14T14:56:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: osdmap e22: 4 total, 3 up, 4 in 2024-02-14T14:56:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1498506356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:56:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1971784806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1e9407-028a-4d4b-af15-2f0d9b05669b"}]: dispatch 2024-02-14T14:56:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1971784806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c1e9407-028a-4d4b-af15-2f0d9b05669b"}]': finished 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: osdmap e22: 4 total, 3 up, 4 in 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1498506356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:56:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:17 smithi076 ceph-mon[37057]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:17 smithi017 ceph-mon[32549]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:17 smithi050 ceph-mon[36512]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:19.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:19 smithi017 ceph-mon[32549]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:19 smithi076 ceph-mon[37057]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:20.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:19 smithi050 ceph-mon[36512]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:22.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:22 smithi076 ceph-mon[37057]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:22.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:22 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T14:56:22.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:22 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:22.751 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:22 smithi017 ceph-mon[32549]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:22.751 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T14:56:22.751 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:22.846 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:22 smithi050 ceph-mon[36512]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:22.846 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:22 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T14:56:22.846 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:22 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:23.571 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:23 smithi017 ceph-mon[32549]: Deploying daemon osd.3 on smithi017 2024-02-14T14:56:23.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:23 smithi050 ceph-mon[36512]: Deploying daemon osd.3 on smithi017 2024-02-14T14:56:23.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:23 smithi076 ceph-mon[37057]: Deploying daemon osd.3 on smithi017 2024-02-14T14:56:24.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:24 smithi017 ceph-mon[32549]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:24.863 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:24 smithi076 ceph-mon[37057]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:24.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:24 smithi050 ceph-mon[36512]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:26.619 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:26 smithi017 ceph-mon[32549]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:26.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:26 smithi050 ceph-mon[36512]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:26 smithi076 ceph-mon[37057]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:28.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:28 smithi076 ceph-mon[37057]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:28.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:28 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:28.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:28 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:28.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:28 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:28.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:28 smithi017 ceph-mon[32549]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:28.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:28 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:28.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:28 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:28.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:28 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:28.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:28 smithi050 ceph-mon[36512]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:28.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:28 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:28.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:28 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:28.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:28 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:30.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:30 smithi076 ceph-mon[37057]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:30.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:30.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:30.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:30 smithi017 ceph-mon[32549]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:30.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:30.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:30.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:30 smithi050 ceph-mon[36512]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:30.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:30.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:30.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:30.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.906+0000 7f6c4a7fc700 1 -- 172.21.15.17:0/351911112 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c5402d920 con 0x7f6c400a21c0 2024-02-14T14:56:30.908 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 3 on host 'smithi017' 2024-02-14T14:56:30.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c400a21c0 msgr2=0x7f6c400a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:30.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c400a21c0 0x7f6c400a4680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c50006010 tx=0x7f6c50005ec0 comp rx=0 tx=0).stop 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c541027f0 msgr2=0x7f6c5419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c541027f0 0x7f6c5419f2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007f10 tx=0x7f6c4c00d3b0 comp rx=0 tx=0).stop 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 shutdown_connections 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c400a21c0 0x7f6c400a4680 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c541027f0 0x7f6c5419f2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c54100c50 0x7f6c54197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 --2- 172.21.15.17:0/351911112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c54101e50 0x7f6c54198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 >> 172.21.15.17:0/351911112 conn(0x7f6c540fc1f0 msgr2=0x7f6c541051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.908+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 shutdown_connections 2024-02-14T14:56:30.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:56:30.909+0000 7f6c5b7f3700 1 -- 172.21.15.17:0/351911112 wait complete. 2024-02-14T14:56:31.627 DEBUG:teuthology.orchestra.run.smithi017:osd.3> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.3.service 2024-02-14T14:56:31.630 INFO:tasks.cephadm:Deploying osd.4 on smithi050 with /dev/vg_nvme/lv_3... 2024-02-14T14:56:31.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-14T14:56:31.891 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:56:32.014 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:31 smithi050 ceph-mon[36512]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:32.014 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:31 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:32.014 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:31 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:32.014 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:31 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:32.014 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:31 smithi050 ceph-mon[36512]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T14:56:32.137 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:31 smithi017 ceph-mon[32549]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:32.137 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:32.137 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:32.137 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:32.138 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:31 smithi017 ceph-mon[32549]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T14:56:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:31 smithi076 ceph-mon[37057]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:31 smithi076 ceph-mon[37057]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T14:56:33.039 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:32 smithi050 ceph-mon[36512]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T14:56:33.039 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:32 smithi050 ceph-mon[36512]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T14:56:33.039 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:32 smithi050 ceph-mon[36512]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:56:33.039 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:33.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:32 smithi017 ceph-mon[32549]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T14:56:33.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:32 smithi017 ceph-mon[32549]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T14:56:33.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:32 smithi017 ceph-mon[32549]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:56:33.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:33.121 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 14:56:32 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T14:56:32.876+0000 7f8c9f8af700 -1 osd.3 0 waiting for initial osdmap 2024-02-14T14:56:33.121 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 14:56:32 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T14:56:32.883+0000 7f8c9b6a9700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:56:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:32 smithi076 ceph-mon[37057]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T14:56:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:32 smithi076 ceph-mon[37057]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T14:56:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:32 smithi076 ceph-mon[37057]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:56:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:33.927 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-14T14:56:33.927 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-14T14:56:33.927 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-14T14:56:33.927 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-14T14:56:33.927 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00831849 s, 1.3 GB/s 2024-02-14T14:56:33.927 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 2024-02-14T14:56:33.928 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-14T14:56:34.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:34.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:56:34.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T14:56:34.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] boot 2024-02-14T14:56:34.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: osdmap e25: 4 total, 4 up, 4 in 2024-02-14T14:56:34.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:33 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] boot 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: osdmap e25: 4 total, 4 up, 4 in 2024-02-14T14:56:34.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:33 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: from='osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: osd.3 [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] boot 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: osdmap e25: 4 total, 4 up, 4 in 2024-02-14T14:56:34.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:33 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T14:56:34.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-02-14T14:56:34.810 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:56:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:34 smithi050 ceph-mon[36512]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:35.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:34 smithi076 ceph-mon[37057]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T14:56:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:56:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:56:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:35.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:35.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:35.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:34 smithi017 ceph-mon[32549]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T14:56:36.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 -- 172.21.15.50:0/3234228221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 msgr2=0x7f97d8102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:36.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/3234228221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d8102320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0099c0 tx=0x7f97cc009cd0 comp rx=0 tx=0).stop 2024-02-14T14:56:36.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 -- 172.21.15.50:0/3234228221 shutdown_connections 2024-02-14T14:56:36.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/3234228221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97d8102860 0x7f97d8109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:36.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/3234228221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97d8100cc0 0x7f97d81010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:36.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/3234228221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d8102320 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:36.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 -- 172.21.15.50:0/3234228221 >> 172.21.15.50:0/3234228221 conn(0x7f97d80fc280 msgr2=0x7f97d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:56:36.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 -- 172.21.15.50:0/3234228221 shutdown_connections 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.447+0000 7f97dcfd3700 1 -- 172.21.15.50:0/3234228221 wait complete. 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 Processor -- start 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 -- start start 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97d8100cc0 0x7f97d8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97d8102860 0x7f97d819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d819f740 con 0x7f97d8102860 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d819f880 con 0x7f97d8101ec0 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.448+0000 7f97dcfd3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d819fa10 con 0x7f97d8100cc0 2024-02-14T14:56:36.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54458/0 (socket says 172.21.15.50:54458) 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 -- 172.21.15.50:0/4061512147 learned_addr learned my addr 172.21.15.50:0/4061512147 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d659c700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97d8100cc0 0x7f97d8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d6d9d700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97d8102860 0x7f97d819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 -- 172.21.15.50:0/4061512147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97d8100cc0 msgr2=0x7f97d8197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97d8100cc0 0x7f97d8197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 -- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97d8102860 msgr2=0x7f97d819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97d8102860 0x7f97d819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:36.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 -- 172.21.15.50:0/4061512147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc0095f0 con 0x7f97d8101ec0 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d6d9d700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97d8102860 0x7f97d819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d659c700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97d8100cc0 0x7f97d8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97d5d9b700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d81980f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f97cc003790 tx=0x7f97cc003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97c77fe700 1 -- 172.21.15.50:0/4061512147 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc01d070 con 0x7f97d8101ec0 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d819fc90 con 0x7f97d8101ec0 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97c77fe700 1 -- 172.21.15.50:0/4061512147 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97cc003f00 con 0x7f97d8101ec0 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97c77fe700 1 -- 172.21.15.50:0/4061512147 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc00f460 con 0x7f97d8101ec0 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.449+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d81a0180 con 0x7f97d8101ec0 2024-02-14T14:56:36.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.450+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d80689e0 con 0x7f97d8101ec0 2024-02-14T14:56:36.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.451+0000 7f97c77fe700 1 -- 172.21.15.50:0/4061512147 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f97cc004070 con 0x7f97d8101ec0 2024-02-14T14:56:36.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.452+0000 7f97c77fe700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97c00a2150 0x7f97c00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:56:36.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.452+0000 7f97c77fe700 1 -- 172.21.15.50:0/4061512147 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f97cc0cde40 con 0x7f97d8101ec0 2024-02-14T14:56:36.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.453+0000 7f97d659c700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97c00a2150 0x7f97c00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:56:36.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.453+0000 7f97d659c700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97c00a2150 0x7f97c00a4610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f97d8101d20 tx=0x7f97c800a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:56:36.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.457+0000 7f97c77fe700 1 -- 172.21.15.50:0/4061512147 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97cc05dd20 con 0x7f97d8101ec0 2024-02-14T14:56:36.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:36 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi017 2024-02-14T14:56:36.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:36 smithi050 ceph-mon[36512]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:36.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:36 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi017 to 8546M 2024-02-14T14:56:36.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:36 smithi050 ceph-mon[36512]: osdmap e27: 4 total, 4 up, 4 in 2024-02-14T14:56:36.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:36.668+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f97d80fe340 con 0x7f97c00a2150 2024-02-14T14:56:36.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:36 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi017 2024-02-14T14:56:36.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:36 smithi076 ceph-mon[37057]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:36.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:36 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi017 to 8546M 2024-02-14T14:56:36.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:36 smithi076 ceph-mon[37057]: osdmap e27: 4 total, 4 up, 4 in 2024-02-14T14:56:36.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:36 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi017 2024-02-14T14:56:36.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:36 smithi017 ceph-mon[32549]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:36.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:36 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi017 to 8546M 2024-02-14T14:56:36.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:36 smithi017 ceph-mon[32549]: osdmap e27: 4 total, 4 up, 4 in 2024-02-14T14:56:37.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:37 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:56:37.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:37 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:56:37.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:37 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:37.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:37 smithi050 ceph-mon[36512]: osdmap e28: 4 total, 4 up, 4 in 2024-02-14T14:56:37.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:37 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:56:37.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:37 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:56:37.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:37 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:37.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:37 smithi017 ceph-mon[32549]: osdmap e28: 4 total, 4 up, 4 in 2024-02-14T14:56:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:37 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:56:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:37 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:56:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:37 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:37 smithi076 ceph-mon[37057]: osdmap e28: 4 total, 4 up, 4 in 2024-02-14T14:56:38.663 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:38 smithi050 ceph-mon[36512]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:38.663 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:38 smithi050 ceph-mon[36512]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:56:38.664 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:38 smithi050 ceph-mon[36512]: osdmap e29: 4 total, 4 up, 4 in 2024-02-14T14:56:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:38 smithi017 ceph-mon[32549]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:38 smithi017 ceph-mon[32549]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:56:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:38 smithi017 ceph-mon[32549]: osdmap e29: 4 total, 4 up, 4 in 2024-02-14T14:56:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:38 smithi076 ceph-mon[37057]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:38 smithi076 ceph-mon[37057]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:56:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:38 smithi076 ceph-mon[37057]: osdmap e29: 4 total, 4 up, 4 in 2024-02-14T14:56:39.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:39 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/1734696126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]: dispatch 2024-02-14T14:56:39.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:39 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]: dispatch 2024-02-14T14:56:39.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:39 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]': finished 2024-02-14T14:56:39.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:39 smithi050 ceph-mon[36512]: osdmap e30: 5 total, 4 up, 5 in 2024-02-14T14:56:39.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:39 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:39 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/1734696126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]: dispatch 2024-02-14T14:56:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:39 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]: dispatch 2024-02-14T14:56:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:39 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]': finished 2024-02-14T14:56:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:39 smithi076 ceph-mon[37057]: osdmap e30: 5 total, 4 up, 5 in 2024-02-14T14:56:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:39 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:40.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:39 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/1734696126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]: dispatch 2024-02-14T14:56:40.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:39 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]: dispatch 2024-02-14T14:56:40.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:39 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "445cd928-d6fe-4815-8094-e407331ec2db"}]': finished 2024-02-14T14:56:40.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:39 smithi017 ceph-mon[32549]: osdmap e30: 5 total, 4 up, 5 in 2024-02-14T14:56:40.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:39 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:40 smithi050 ceph-mon[36512]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:40 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/1324715432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:56:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:40 smithi076 ceph-mon[37057]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:40 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/1324715432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:56:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:40 smithi017 ceph-mon[32549]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:40 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/1324715432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:56:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:42 smithi050 ceph-mon[36512]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-14T14:56:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:42 smithi076 ceph-mon[37057]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-14T14:56:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:42 smithi017 ceph-mon[32549]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-14T14:56:43.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:43 smithi050 ceph-mon[36512]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:56:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:43 smithi076 ceph-mon[37057]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:56:44.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:43 smithi017 ceph-mon[32549]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:56:46.739 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:46 smithi050 ceph-mon[36512]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-14T14:56:46.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:46 smithi017 ceph-mon[32549]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-14T14:56:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:46 smithi076 ceph-mon[37057]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-14T14:56:47.733 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T14:56:47.733 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:47.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T14:56:47.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T14:56:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:48.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:48 smithi050 ceph-mon[36512]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-02-14T14:56:48.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:48 smithi050 ceph-mon[36512]: Deploying daemon osd.4 on smithi050 2024-02-14T14:56:48.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:48 smithi017 ceph-mon[32549]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-02-14T14:56:48.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:48 smithi017 ceph-mon[32549]: Deploying daemon osd.4 on smithi050 2024-02-14T14:56:48.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:48 smithi076 ceph-mon[37057]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-02-14T14:56:48.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:48 smithi076 ceph-mon[37057]: Deploying daemon osd.4 on smithi050 2024-02-14T14:56:50.665 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:50 smithi050 ceph-mon[36512]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:56:50.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:50 smithi017 ceph-mon[32549]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:56:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:50 smithi076 ceph-mon[37057]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:56:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:52 smithi017 ceph-mon[32549]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-14T14:56:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:52.786 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:52 smithi076 ceph-mon[37057]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-14T14:56:52.786 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:52.786 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:52.786 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:52 smithi050 ceph-mon[36512]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-02-14T14:56:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:54.627 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:54 smithi050 ceph-mon[36512]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:54 smithi017 ceph-mon[32549]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:54 smithi076 ceph-mon[37057]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:55.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.785+0000 7f97c77fe700 1 -- 172.21.15.50:0/4061512147 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f97d80fe340 con 0x7f97c00a2150 2024-02-14T14:56:55.787 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 4 on host 'smithi050' 2024-02-14T14:56:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97c00a2150 msgr2=0x7f97c00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97c00a2150 0x7f97c00a4610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f97d8101d20 tx=0x7f97c800a680 comp rx=0 tx=0).stop 2024-02-14T14:56:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 msgr2=0x7f97d81980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:56:55.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d81980f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f97cc003790 tx=0x7f97cc003a10 comp rx=0 tx=0).stop 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 shutdown_connections 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97c00a2150 0x7f97c00a4610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97d8102860 0x7f97d819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97d8100cc0 0x7f97d8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 --2- 172.21.15.50:0/4061512147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d8101ec0 0x7f97d81980f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 >> 172.21.15.50:0/4061512147 conn(0x7f97d80fc280 msgr2=0x7f97d80fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 shutdown_connections 2024-02-14T14:56:55.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:56:55.787+0000 7f97dcfd3700 1 -- 172.21.15.50:0/4061512147 wait complete. 2024-02-14T14:56:56.093 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:56.093 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.093 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.093 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:56.093 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:56.094 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.094 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.094 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.094 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:55 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:55 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:56.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:56.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:56.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:56.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:56.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:55 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:56:56.410 DEBUG:teuthology.orchestra.run.smithi050:osd.4> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.4.service 2024-02-14T14:56:56.412 INFO:tasks.cephadm:Deploying osd.5 on smithi076 with /dev/vg_nvme/lv_3... 2024-02-14T14:56:56.413 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-14T14:56:56.651 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:56:57.086 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:56 smithi076 ceph-mon[37057]: from='osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T14:56:57.086 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:56 smithi076 ceph-mon[37057]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T14:56:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:56 smithi050 ceph-mon[36512]: from='osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T14:56:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:56 smithi050 ceph-mon[36512]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T14:56:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:56 smithi017 ceph-mon[32549]: from='osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T14:56:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:56 smithi017 ceph-mon[32549]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T14:56:57.971 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:57 smithi050 ceph-mon[36512]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:57.971 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:57 smithi050 ceph-mon[36512]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T14:56:57.971 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:57 smithi050 ceph-mon[36512]: from='osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:56:57.972 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:57 smithi050 ceph-mon[36512]: osdmap e31: 5 total, 4 up, 5 in 2024-02-14T14:56:57.972 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:57.972 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:57 smithi050 ceph-mon[36512]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:56:58.082 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:57 smithi076 ceph-mon[37057]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:58.082 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:57 smithi076 ceph-mon[37057]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T14:56:58.082 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:57 smithi076 ceph-mon[37057]: from='osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:56:58.082 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:57 smithi076 ceph-mon[37057]: osdmap e31: 5 total, 4 up, 5 in 2024-02-14T14:56:58.083 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:58.083 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:57 smithi076 ceph-mon[37057]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:56:58.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:57 smithi017 ceph-mon[32549]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:58.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:57 smithi017 ceph-mon[32549]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T14:56:58.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:57 smithi017 ceph-mon[32549]: from='osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:56:58.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:57 smithi017 ceph-mon[32549]: osdmap e31: 5 total, 4 up, 5 in 2024-02-14T14:56:58.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:58.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:57 smithi017 ceph-mon[32549]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:56:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-14T14:56:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-14T14:56:58.638 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-02-14T14:56:58.639 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-02-14T14:56:58.639 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125177 s, 838 MB/s 2024-02-14T14:56:58.639 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-02-14T14:56:58.881 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 14:56:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4[48009]: 2024-02-14T14:56:58.491+0000 7f149c778700 -1 osd.4 0 waiting for initial osdmap 2024-02-14T14:56:58.882 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 14:56:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4[48009]: 2024-02-14T14:56:58.499+0000 7f149456a700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:56:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:58 smithi050 ceph-mon[36512]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:56:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:58 smithi050 ceph-mon[36512]: osdmap e32: 5 total, 4 up, 5 in 2024-02-14T14:56:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:58 smithi050 ceph-mon[36512]: from='osd.4 ' entity='osd.4' 2024-02-14T14:56:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:58 smithi076 ceph-mon[37057]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:56:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:58 smithi076 ceph-mon[37057]: osdmap e32: 5 total, 4 up, 5 in 2024-02-14T14:56:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:58 smithi076 ceph-mon[37057]: from='osd.4 ' entity='osd.4' 2024-02-14T14:56:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:58 smithi017 ceph-mon[32549]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:56:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:58 smithi017 ceph-mon[32549]: osdmap e32: 5 total, 4 up, 5 in 2024-02-14T14:56:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:58 smithi017 ceph-mon[32549]: from='osd.4 ' entity='osd.4' 2024-02-14T14:56:59.353 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_3 2024-02-14T14:56:59.613 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:56:59.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:56:59.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:56:59.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:56:59.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:56:59.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi050 2024-02-14T14:56:59.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:59.953 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi050 to 8542M 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] boot 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T14:56:59.954 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:56:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:57:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:57:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:57:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:57:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:57:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi050 2024-02-14T14:57:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi050 to 8542M 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] boot 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T14:57:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:56:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:57:00.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:57:00.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:57:00.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi050 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi050 to 8542M 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:00.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:00.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: osd.4 [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] boot 2024-02-14T14:57:00.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T14:57:00.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:56:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T14:57:01.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 -- 172.21.15.76:0/2916446953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c102f70 msgr2=0x7fbc0c1033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 --2- 172.21.15.76:0/2916446953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c102f70 0x7fbc0c1033d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009990 tx=0x7fbc00009ca0 comp rx=0 tx=0).stop 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 -- 172.21.15.76:0/2916446953 shutdown_connections 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 --2- 172.21.15.76:0/2916446953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0c103910 0x7fbc0c10a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 --2- 172.21.15.76:0/2916446953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc0c101d70 0x7fbc0c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 --2- 172.21.15.76:0/2916446953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c102f70 0x7fbc0c1033d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 -- 172.21.15.76:0/2916446953 >> 172.21.15.76:0/2916446953 conn(0x7fbc0c0fd330 msgr2=0x7fbc0c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 -- 172.21.15.76:0/2916446953 shutdown_connections 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 -- 172.21.15.76:0/2916446953 wait complete. 2024-02-14T14:57:01.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 Processor -- start 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.229+0000 7fbc11af1700 1 -- start start 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc11af1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c101d70 0x7fbc0c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc11af1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0c102f70 0x7fbc0c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc11af1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc0c103910 0x7fbc0c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc11af1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c19f7c0 con 0x7fbc0c103910 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc11af1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c19f900 con 0x7fbc0c102f70 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc11af1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c19fac0 con 0x7fbc0c101d70 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c101d70 0x7fbc0c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c101d70 0x7fbc0c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38802/0 (socket says 172.21.15.76:38802) 2024-02-14T14:57:01.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 -- 172.21.15.76:0/686060822 learned_addr learned my addr 172.21.15.76:0/686060822 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0affd700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0c102f70 0x7fbc0c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0bfff700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc0c103910 0x7fbc0c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 -- 172.21.15.76:0/686060822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0c102f70 msgr2=0x7fbc0c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0c102f70 0x7fbc0c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 -- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc0c103910 msgr2=0x7fbc0c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc0c103910 0x7fbc0c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 -- 172.21.15.76:0/686060822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc000095f0 con 0x7fbc0c101d70 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0bfff700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc0c103910 0x7fbc0c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc0b7fe700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c101d70 0x7fbc0c197c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00c970 tx=0x7fbbfc00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:57:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc08ff9700 1 -- 172.21.15.76:0/686060822 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc004220 con 0x7fbc0c101d70 2024-02-14T14:57:01.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c19fda0 con 0x7fbc0c101d70 2024-02-14T14:57:01.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.230+0000 7fbc08ff9700 1 -- 172.21.15.76:0/686060822 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbfc007d80 con 0x7fbc0c101d70 2024-02-14T14:57:01.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.231+0000 7fbc08ff9700 1 -- 172.21.15.76:0/686060822 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc003d00 con 0x7fbc0c101d70 2024-02-14T14:57:01.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.231+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c1a0200 con 0x7fbc0c101d70 2024-02-14T14:57:01.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.231+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0c1090d0 con 0x7fbc0c101d70 2024-02-14T14:57:01.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.232+0000 7fbc08ff9700 1 -- 172.21.15.76:0/686060822 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbbfc0078a0 con 0x7fbc0c101d70 2024-02-14T14:57:01.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.233+0000 7fbc08ff9700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbf40a2150 0x7fbbf40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:01.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.233+0000 7fbc0affd700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbf40a2150 0x7fbbf40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:01.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.233+0000 7fbc08ff9700 1 -- 172.21.15.76:0/686060822 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fbbfc0ccc60 con 0x7fbc0c101d70 2024-02-14T14:57:01.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.234+0000 7fbc0affd700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbf40a2150 0x7fbbf40a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000b5c0 tx=0x7fbc000038d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:57:01.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.237+0000 7fbc08ff9700 1 -- 172.21.15.76:0/686060822 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbfc096c00 con 0x7fbc0c101d70 2024-02-14T14:57:01.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:01.440+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc0c02d960 con 0x7fbbf40a2150 2024-02-14T14:57:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:01 smithi050 ceph-mon[36512]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:01 smithi050 ceph-mon[36512]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T14:57:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:01 smithi050 ceph-mon[36512]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:01 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:01 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:02.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:01 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:02.215 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:01 smithi076 ceph-mon[37057]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:02.215 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:01 smithi076 ceph-mon[37057]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T14:57:02.215 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:01 smithi076 ceph-mon[37057]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:02.215 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:01 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:02.215 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:01 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:02.215 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:01 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:02.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:01 smithi017 ceph-mon[32549]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:02.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:01 smithi017 ceph-mon[32549]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T14:57:02.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:01 smithi017 ceph-mon[32549]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:02.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:01 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:02.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:01 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:02.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:01 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:04.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:03 smithi050 ceph-mon[36512]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:03 smithi076 ceph-mon[37057]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:03 smithi017 ceph-mon[32549]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/2055152698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]: dispatch 2024-02-14T14:57:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:04 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]: dispatch 2024-02-14T14:57:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:04 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]': finished 2024-02-14T14:57:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:04 smithi050 ceph-mon[36512]: osdmap e35: 6 total, 5 up, 6 in 2024-02-14T14:57:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/974387641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/2055152698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]: dispatch 2024-02-14T14:57:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:04 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]: dispatch 2024-02-14T14:57:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:04 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]': finished 2024-02-14T14:57:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:04 smithi076 ceph-mon[37057]: osdmap e35: 6 total, 5 up, 6 in 2024-02-14T14:57:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/974387641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/2055152698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]: dispatch 2024-02-14T14:57:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:04 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]: dispatch 2024-02-14T14:57:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:04 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7c1a10c-7507-426e-b91a-9f788f8edd9a"}]': finished 2024-02-14T14:57:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:04 smithi017 ceph-mon[32549]: osdmap e35: 6 total, 5 up, 6 in 2024-02-14T14:57:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/974387641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:06.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:06 smithi017 ceph-mon[32549]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:06.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:06 smithi050 ceph-mon[36512]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:06.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:06 smithi076 ceph-mon[37057]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:08.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:08 smithi076 ceph-mon[37057]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:08.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:08 smithi017 ceph-mon[32549]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:08.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:08 smithi050 ceph-mon[36512]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:10.648 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:10 smithi076 ceph-mon[37057]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:10.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:10 smithi017 ceph-mon[32549]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:10.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:10 smithi050 ceph-mon[36512]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:11.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:11 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T14:57:11.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:11 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:11.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:11 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T14:57:11.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:11 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:11.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:11 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T14:57:11.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:11 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:12.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:12 smithi076 ceph-mon[37057]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:12.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:12 smithi076 ceph-mon[37057]: Deploying daemon osd.5 on smithi076 2024-02-14T14:57:12.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:12 smithi017 ceph-mon[32549]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:12.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:12 smithi017 ceph-mon[32549]: Deploying daemon osd.5 on smithi076 2024-02-14T14:57:12.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:12 smithi050 ceph-mon[36512]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:12.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:12 smithi050 ceph-mon[36512]: Deploying daemon osd.5 on smithi076 2024-02-14T14:57:14.602 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:14 smithi076 ceph-mon[37057]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:14.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:14 smithi017 ceph-mon[32549]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:14 smithi050 ceph-mon[36512]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:16 smithi076 ceph-mon[37057]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:16.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:16 smithi017 ceph-mon[32549]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:16 smithi050 ceph-mon[36512]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:17 smithi076 ceph-mon[37057]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:17 smithi017 ceph-mon[32549]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:18.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:17 smithi050 ceph-mon[36512]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:18.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:18.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:18.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:19 smithi076 ceph-mon[37057]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:20.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:19 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:19 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:19 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:20.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:19 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:20.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:19 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:19 smithi017 ceph-mon[32549]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:20.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:19 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:19 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:19 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:20.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:19 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:20.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:19 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:19 smithi050 ceph-mon[36512]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:20.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:19 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:19 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:19 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:20.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:19 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:20.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:19 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:20.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.756+0000 7fbc08ff9700 1 -- 172.21.15.76:0/686060822 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbc0c02d960 con 0x7fbbf40a2150 2024-02-14T14:57:20.758 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 5 on host 'smithi076' 2024-02-14T14:57:20.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbf40a2150 msgr2=0x7fbbf40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:20.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbf40a2150 0x7fbbf40a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000b5c0 tx=0x7fbc000038d0 comp rx=0 tx=0).stop 2024-02-14T14:57:20.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c101d70 msgr2=0x7fbc0c197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c101d70 0x7fbc0c197c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00c970 tx=0x7fbbfc00cc80 comp rx=0 tx=0).stop 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 shutdown_connections 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbf40a2150 0x7fbbf40a4610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc0c103910 0x7fbc0c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc0c101d70 0x7fbc0c197c70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 --2- 172.21.15.76:0/686060822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0c102f70 0x7fbc0c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 >> 172.21.15.76:0/686060822 conn(0x7fbc0c0fd330 msgr2=0x7fbc0c106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:57:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 shutdown_connections 2024-02-14T14:57:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:57:20.759+0000 7fbc11af1700 1 -- 172.21.15.76:0/686060822 wait complete. 2024-02-14T14:57:21.380 DEBUG:teuthology.orchestra.run.smithi076:osd.5> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.5.service 2024-02-14T14:57:21.384 INFO:tasks.cephadm:Deploying osd.6 on smithi017 with /dev/vg_nvme/lv_2... 2024-02-14T14:57:21.384 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-14T14:57:21.639 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:57:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:21 smithi076 ceph-mon[37057]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:21 smithi076 ceph-mon[37057]: from='osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T14:57:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:21 smithi076 ceph-mon[37057]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T14:57:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:21 smithi017 ceph-mon[32549]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:21 smithi017 ceph-mon[32549]: from='osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T14:57:22.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:21 smithi017 ceph-mon[32549]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T14:57:22.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:21 smithi050 ceph-mon[36512]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:22.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:22.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:21 smithi050 ceph-mon[36512]: from='osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T14:57:22.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:21 smithi050 ceph-mon[36512]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T14:57:22.892 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:22 smithi076 ceph-mon[37057]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T14:57:22.892 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:22 smithi076 ceph-mon[37057]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T14:57:22.892 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:22 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:22.893 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:22 smithi076 ceph-mon[37057]: from='osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:57:22.893 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:22 smithi076 ceph-mon[37057]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:57:22.987 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:22 smithi017 ceph-mon[32549]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T14:57:22.987 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:22 smithi017 ceph-mon[32549]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T14:57:22.987 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:22 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:22.987 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:22 smithi017 ceph-mon[32549]: from='osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:57:22.988 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:22 smithi017 ceph-mon[32549]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:57:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:22 smithi050 ceph-mon[36512]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T14:57:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:22 smithi050 ceph-mon[36512]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T14:57:23.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:22 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:23.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:22 smithi050 ceph-mon[36512]: from='osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:57:23.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:22 smithi050 ceph-mon[36512]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:57:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-14T14:57:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-14T14:57:23.659 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-02-14T14:57:23.660 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-02-14T14:57:23.660 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852534 s, 1.2 GB/s 2024-02-14T14:57:23.660 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-02-14T14:57:23.734 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T14:57:23.436+0000 7f43af302700 -1 osd.5 0 waiting for initial osdmap 2024-02-14T14:57:23.734 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T14:57:23.443+0000 7f43a68f3700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:57:24.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:23 smithi017 ceph-mon[32549]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:24.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:23 smithi017 ceph-mon[32549]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:57:24.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:23 smithi017 ceph-mon[32549]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T14:57:24.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:23 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:23 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:23 smithi017 ceph-mon[32549]: from='osd.5 ' entity='osd.5' 2024-02-14T14:57:24.046 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-mon[37057]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:24.046 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-mon[37057]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:57:24.046 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-mon[37057]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T14:57:24.046 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.046 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.046 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:23 smithi076 ceph-mon[37057]: from='osd.5 ' entity='osd.5' 2024-02-14T14:57:24.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:23 smithi050 ceph-mon[36512]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:24.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:23 smithi050 ceph-mon[36512]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:57:24.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:23 smithi050 ceph-mon[36512]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:23 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:23 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:23 smithi050 ceph-mon[36512]: from='osd.5 ' entity='osd.5' 2024-02-14T14:57:24.378 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_2 2024-02-14T14:57:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:57:24.939 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:57:24.939 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] boot 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: osdmap e38: 6 total, 6 up, 6 in 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi076 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:24.940 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:24.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:24.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi076 to 8540M 2024-02-14T14:57:24.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:24.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:24.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:24.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:24.994 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:57:24.994 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:57:24.994 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.994 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] boot 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: osdmap e38: 6 total, 6 up, 6 in 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi076 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi076 to 8540M 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:24.995 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:24.996 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:24.996 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: osd.5 [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] boot 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: osdmap e38: 6 total, 6 up, 6 in 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi076 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi076 to 8540M 2024-02-14T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:25.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:25 smithi017 ceph-mon[32549]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:25.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:25 smithi017 ceph-mon[32549]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T14:57:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:25 smithi050 ceph-mon[36512]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:25 smithi050 ceph-mon[36512]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T14:57:26.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:25 smithi076 ceph-mon[37057]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T14:57:26.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:25 smithi076 ceph-mon[37057]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T14:57:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 -- 172.21.15.17:0/2523684652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc105110 msgr2=0x7f05bc107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 --2- 172.21.15.17:0/2523684652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc105110 0x7f05bc107500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b0009ca0 comp rx=0 tx=0).stop 2024-02-14T14:57:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 -- 172.21.15.17:0/2523684652 shutdown_connections 2024-02-14T14:57:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 --2- 172.21.15.17:0/2523684652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05bc107a40 0x7f05bc109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 --2- 172.21.15.17:0/2523684652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc1027e0 0x7f05bc104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 --2- 172.21.15.17:0/2523684652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc105110 0x7f05bc107500 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 -- 172.21.15.17:0/2523684652 >> 172.21.15.17:0/2523684652 conn(0x7f05bc0fc1f0 msgr2=0x7f05bc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:57:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 -- 172.21.15.17:0/2523684652 shutdown_connections 2024-02-14T14:57:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.434+0000 7f05c3450700 1 -- 172.21.15.17:0/2523684652 wait complete. 2024-02-14T14:57:26.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 Processor -- start 2024-02-14T14:57:26.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 -- start start 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc1027e0 0x7f05bc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05bc105110 0x7f05bc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc107a40 0x7f05bc19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc19f8e0 con 0x7f05bc1027e0 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc19fa20 con 0x7f05bc107a40 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c3450700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc19fbb0 con 0x7f05bc105110 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc1027e0 0x7f05bc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc1027e0 0x7f05bc197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38996/0 (socket says 172.21.15.17:38996) 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 -- 172.21.15.17:0/556285189 learned_addr learned my addr 172.21.15.17:0/556285189 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c19ed700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc107a40 0x7f05bc19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:26.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c09eb700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05bc105110 0x7f05bc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 -- 172.21.15.17:0/556285189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05bc105110 msgr2=0x7f05bc198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05bc105110 0x7f05bc198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 -- 172.21.15.17:0/556285189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc107a40 msgr2=0x7f05bc19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc107a40 0x7f05bc19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.435+0000 7f05c11ec700 1 -- 172.21.15.17:0/556285189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00095f0 con 0x7f05bc1027e0 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.436+0000 7f05c09eb700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05bc105110 0x7f05bc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.436+0000 7f05c11ec700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc1027e0 0x7f05bc197d50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05b400add0 tx=0x7f05b400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.436+0000 7f05ba7fc700 1 -- 172.21.15.17:0/556285189 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400f070 con 0x7f05bc1027e0 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.436+0000 7f05ba7fc700 1 -- 172.21.15.17:0/556285189 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b4007bf0 con 0x7f05bc1027e0 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.436+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc19fe90 con 0x7f05bc1027e0 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.436+0000 7f05ba7fc700 1 -- 172.21.15.17:0/556285189 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4005420 con 0x7f05bc1027e0 2024-02-14T14:57:26.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.436+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc1a02a0 con 0x7f05bc1027e0 2024-02-14T14:57:26.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.437+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc0689e0 con 0x7f05bc1027e0 2024-02-14T14:57:26.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.438+0000 7f05ba7fc700 1 -- 172.21.15.17:0/556285189 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f05b4004ad0 con 0x7f05bc1027e0 2024-02-14T14:57:26.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.439+0000 7f05ba7fc700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05a40a2150 0x7f05a40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:26.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.439+0000 7f05c09eb700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05a40a2150 0x7f05a40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:26.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.439+0000 7f05ba7fc700 1 -- 172.21.15.17:0/556285189 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f05b40cc550 con 0x7f05bc1027e0 2024-02-14T14:57:26.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.439+0000 7f05c09eb700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05a40a2150 0x7f05a40a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05b00038c0 tx=0x7f05b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:57:26.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.443+0000 7f05ba7fc700 1 -- 172.21.15.17:0/556285189 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05b40963b0 con 0x7f05bc1027e0 2024-02-14T14:57:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:26.654+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f05bc02d960 con 0x7f05a40a2150 2024-02-14T14:57:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:26 smithi017 ceph-mon[32549]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T14:57:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:26 smithi050 ceph-mon[36512]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T14:57:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:27.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:26 smithi076 ceph-mon[37057]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T14:57:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:27 smithi017 ceph-mon[32549]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:27 smithi017 ceph-mon[32549]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:27 smithi017 ceph-mon[32549]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T14:57:28.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:27 smithi050 ceph-mon[36512]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:28.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:27 smithi050 ceph-mon[36512]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:28.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:27 smithi050 ceph-mon[36512]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T14:57:28.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:27 smithi076 ceph-mon[37057]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:28.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:27 smithi076 ceph-mon[37057]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:28.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:27 smithi076 ceph-mon[37057]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T14:57:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:28 smithi017 ceph-mon[32549]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T14:57:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:28 smithi050 ceph-mon[36512]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T14:57:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:28 smithi076 ceph-mon[37057]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T14:57:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:29 smithi017 ceph-mon[32549]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:29 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2657101113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837621c6-44f2-4ede-83d8-031db03b8ad2"}]: dispatch 2024-02-14T14:57:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:29 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2657101113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "837621c6-44f2-4ede-83d8-031db03b8ad2"}]': finished 2024-02-14T14:57:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:29 smithi017 ceph-mon[32549]: osdmap e43: 7 total, 6 up, 7 in 2024-02-14T14:57:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:29 smithi050 ceph-mon[36512]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:29 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2657101113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837621c6-44f2-4ede-83d8-031db03b8ad2"}]: dispatch 2024-02-14T14:57:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:29 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2657101113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "837621c6-44f2-4ede-83d8-031db03b8ad2"}]': finished 2024-02-14T14:57:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:29 smithi050 ceph-mon[36512]: osdmap e43: 7 total, 6 up, 7 in 2024-02-14T14:57:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:29 smithi076 ceph-mon[37057]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:29 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2657101113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837621c6-44f2-4ede-83d8-031db03b8ad2"}]: dispatch 2024-02-14T14:57:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:29 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2657101113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "837621c6-44f2-4ede-83d8-031db03b8ad2"}]': finished 2024-02-14T14:57:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:29 smithi076 ceph-mon[37057]: osdmap e43: 7 total, 6 up, 7 in 2024-02-14T14:57:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:30 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/661821933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:30 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/661821933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:30 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/661821933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:32.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:32 smithi017 ceph-mon[32549]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-14T14:57:32.888 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:32 smithi050 ceph-mon[36512]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-14T14:57:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:32 smithi076 ceph-mon[37057]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-14T14:57:34.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:34 smithi076 ceph-mon[37057]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:57:34.739 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:34 smithi017 ceph-mon[32549]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:57:34.794 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:34 smithi050 ceph-mon[36512]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:57:36.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:36 smithi017 ceph-mon[32549]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-14T14:57:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:36 smithi050 ceph-mon[36512]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-14T14:57:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:36 smithi076 ceph-mon[37057]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-14T14:57:37.761 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:37 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T14:57:37.761 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:37 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:37.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:37 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T14:57:37.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:37 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:37 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T14:57:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:37 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:38 smithi017 ceph-mon[32549]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-14T14:57:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:38 smithi017 ceph-mon[32549]: Deploying daemon osd.6 on smithi017 2024-02-14T14:57:38.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:38 smithi050 ceph-mon[36512]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-14T14:57:38.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:38 smithi050 ceph-mon[36512]: Deploying daemon osd.6 on smithi017 2024-02-14T14:57:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:38 smithi076 ceph-mon[37057]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-14T14:57:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:38 smithi076 ceph-mon[37057]: Deploying daemon osd.6 on smithi017 2024-02-14T14:57:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:40 smithi017 ceph-mon[32549]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:57:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:40 smithi050 ceph-mon[36512]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:57:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:40 smithi076 ceph-mon[37057]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:57:42.691 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:42 smithi017 ceph-mon[32549]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-14T14:57:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:42 smithi050 ceph-mon[36512]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-14T14:57:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:42 smithi076 ceph-mon[37057]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-02-14T14:57:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:43 smithi076 ceph-mon[37057]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:44.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:44.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:43 smithi017 ceph-mon[32549]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:44.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:44.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:44.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:43 smithi050 ceph-mon[36512]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:44.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:44.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:46.741 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:46 smithi017 ceph-mon[32549]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:46.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:46.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:46.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:46 smithi050 ceph-mon[36512]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:46.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:46.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:46 smithi076 ceph-mon[37057]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:47.182 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 6 on host 'smithi017' 2024-02-14T14:57:47.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.180+0000 7f05ba7fc700 1 -- 172.21.15.17:0/556285189 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f05bc02d960 con 0x7f05a40a2150 2024-02-14T14:57:47.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05a40a2150 msgr2=0x7f05a40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:47.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05a40a2150 0x7f05a40a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05b00038c0 tx=0x7f05b0005fb0 comp rx=0 tx=0).stop 2024-02-14T14:57:47.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc1027e0 msgr2=0x7f05bc197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc1027e0 0x7f05bc197d50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05b400add0 tx=0x7f05b400c330 comp rx=0 tx=0).stop 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 shutdown_connections 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05a40a2150 0x7f05a40a4610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc107a40 0x7f05bc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05bc1027e0 0x7f05bc197d50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 --2- 172.21.15.17:0/556285189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05bc105110 0x7f05bc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 >> 172.21.15.17:0/556285189 conn(0x7f05bc0fc1f0 msgr2=0x7f05bc103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 shutdown_connections 2024-02-14T14:57:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:57:47.182+0000 7f05c3450700 1 -- 172.21.15.17:0/556285189 wait complete. 2024-02-14T14:57:47.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:47 smithi017 ceph-mon[32549]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T14:57:47.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:47.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:47.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:47.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:47 smithi050 ceph-mon[36512]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T14:57:47.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:47.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:47.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:47.958 DEBUG:teuthology.orchestra.run.smithi017:osd.6> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.6.service 2024-02-14T14:57:47.960 INFO:tasks.cephadm:Deploying osd.7 on smithi050 with /dev/vg_nvme/lv_2... 2024-02-14T14:57:47.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-14T14:57:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:47 smithi076 ceph-mon[37057]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T14:57:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:57:48.220 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:57:48.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:48 smithi050 ceph-mon[36512]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:48.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:48 smithi050 ceph-mon[36512]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T14:57:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:48 smithi050 ceph-mon[36512]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T14:57:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:48 smithi050 ceph-mon[36512]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:57:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:48.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:48 smithi017 ceph-mon[32549]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:48.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:48 smithi017 ceph-mon[32549]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T14:57:48.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:48 smithi017 ceph-mon[32549]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T14:57:48.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:48 smithi017 ceph-mon[32549]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:57:48.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:48.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:48 smithi076 ceph-mon[37057]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:48.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:48 smithi076 ceph-mon[37057]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T14:57:48.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:48 smithi076 ceph-mon[37057]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T14:57:48.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:48 smithi076 ceph-mon[37057]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:57:48.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.414 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T14:57:49.032+0000 7f3e49f83700 -1 osd.6 0 waiting for initial osdmap 2024-02-14T14:57:49.415 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T14:57:49.040+0000 7f3e41d75700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:57:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:49 smithi050 ceph-mon[36512]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:57:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:49 smithi050 ceph-mon[36512]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T14:57:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:49 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:49 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:49 smithi050 ceph-mon[36512]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' 2024-02-14T14:57:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:49 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.724 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-mon[32549]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:57:49.724 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-mon[32549]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T14:57:49.724 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.724 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.724 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-mon[32549]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' 2024-02-14T14:57:49.724 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:49 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:49 smithi076 ceph-mon[37057]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:57:49.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:49 smithi076 ceph-mon[37057]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T14:57:49.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:49 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:49 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:49.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:49 smithi076 ceph-mon[37057]: from='osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579]' entity='osd.6' 2024-02-14T14:57:49.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:49 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:50.305 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-14T14:57:50.306 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-14T14:57:50.306 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-14T14:57:50.306 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-14T14:57:50.306 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832679 s, 1.3 GB/s 2024-02-14T14:57:50.306 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-14T14:57:50.613 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:50 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:57:50.613 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:50 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:57:50.613 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:50 smithi050 ceph-mon[36512]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:50.613 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:50 smithi050 ceph-mon[36512]: osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] boot 2024-02-14T14:57:50.613 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:50 smithi050 ceph-mon[36512]: osdmap e46: 7 total, 7 up, 7 in 2024-02-14T14:57:50.613 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:50.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:50 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:57:50.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:50 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:57:50.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:50 smithi017 ceph-mon[32549]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:50.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:50 smithi017 ceph-mon[32549]: osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] boot 2024-02-14T14:57:50.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:50 smithi017 ceph-mon[32549]: osdmap e46: 7 total, 7 up, 7 in 2024-02-14T14:57:50.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:50.967 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-02-14T14:57:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:50 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:57:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:50 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:57:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:50 smithi076 ceph-mon[37057]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-02-14T14:57:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:50 smithi076 ceph-mon[37057]: osd.6 [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] boot 2024-02-14T14:57:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:50 smithi076 ceph-mon[37057]: osdmap e46: 7 total, 7 up, 7 in 2024-02-14T14:57:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T14:57:51.233 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi017 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.617 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi017 to 5697M 2024-02-14T14:57:52.618 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.618 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:52.618 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:52.618 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.618 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:52 smithi050 ceph-mon[36512]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T14:57:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:52.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi017 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi017 to 5697M 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:52.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:52.736 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.736 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:52 smithi076 ceph-mon[37057]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T14:57:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:52.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi017 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi017 to 5697M 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:57:52.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:57:52.768 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:52 smithi017 ceph-mon[32549]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T14:57:52.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/1825890435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8101e80 msgr2=0x7f5cd81022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:52.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/1825890435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8101e80 0x7f5cd81022e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40099c0 tx=0x7f5cd4009cd0 comp rx=0 tx=0).stop 2024-02-14T14:57:52.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/1825890435 shutdown_connections 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/1825890435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd8102820 0x7f5cd81090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/1825890435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cd8100c80 0x7f5cd81010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/1825890435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8101e80 0x7f5cd81022e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/1825890435 >> 172.21.15.50:0/1825890435 conn(0x7f5cd80fc240 msgr2=0x7f5cd80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/1825890435 shutdown_connections 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.860+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/1825890435 wait complete. 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 Processor -- start 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 -- start start 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8100c80 0x7f5cd8196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cd8101e80 0x7f5cd8197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd8102820 0x7f5cd819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd819e770 con 0x7f5cd8102820 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd819e8b0 con 0x7f5cd8100c80 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdf2fb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd819ea40 con 0x7f5cd8101e80 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdd097700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8100c80 0x7f5cd8196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdd097700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8100c80 0x7f5cd8196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36020/0 (socket says 172.21.15.50:36020) 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdc896700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cd8101e80 0x7f5cd8197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.861+0000 7f5cdd097700 1 -- 172.21.15.50:0/3549742729 learned_addr learned my addr 172.21.15.50:0/3549742729 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdd898700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd8102820 0x7f5cd819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdd097700 1 -- 172.21.15.50:0/3549742729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cd8101e80 msgr2=0x7f5cd8197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdd097700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cd8101e80 0x7f5cd8197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdd097700 1 -- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd8102820 msgr2=0x7f5cd819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdd097700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd8102820 0x7f5cd819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdd097700 1 -- 172.21.15.50:0/3549742729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40095f0 con 0x7f5cd8100c80 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdd097700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8100c80 0x7f5cd8196be0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800d980 tx=0x7f5cc800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cce7fc700 1 -- 172.21.15.50:0/3549742729 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8004020 con 0x7f5cd8100c80 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cce7fc700 1 -- 172.21.15.50:0/3549742729 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cc800f690 con 0x7f5cd8100c80 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd819ed20 con 0x7f5cd8100c80 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cce7fc700 1 -- 172.21.15.50:0/3549742729 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8003b40 con 0x7f5cd8100c80 2024-02-14T14:57:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.862+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd819f170 con 0x7f5cd8100c80 2024-02-14T14:57:52.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.863+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd8108040 con 0x7f5cd8100c80 2024-02-14T14:57:52.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.864+0000 7f5cce7fc700 1 -- 172.21.15.50:0/3549742729 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5cc800f800 con 0x7f5cd8100c80 2024-02-14T14:57:52.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.865+0000 7f5cce7fc700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5cc40a2150 0x7f5cc40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:57:52.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.865+0000 7f5cce7fc700 1 -- 172.21.15.50:0/3549742729 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f5cc80ccad0 con 0x7f5cd8100c80 2024-02-14T14:57:52.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.865+0000 7f5cdc896700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5cc40a2150 0x7f5cc40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:57:52.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.866+0000 7f5cdc896700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5cc40a2150 0x7f5cc40a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009990 tx=0x7f5cd4005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:57:52.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:52.870+0000 7f5cce7fc700 1 -- 172.21.15.50:0/3549742729 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cc80967b0 con 0x7f5cd8100c80 2024-02-14T14:57:53.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:57:53.082+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cd802da10 con 0x7f5cc40a2150 2024-02-14T14:57:53.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:53 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:53.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:53 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:53.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:53 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:53.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:53 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:53.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:53 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:53.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:53 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:53.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:53 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:57:53.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:53 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:57:53.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:53 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:57:54.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:54 smithi050 ceph-mon[36512]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:54.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:54 smithi050 ceph-mon[36512]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:54.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:54 smithi076 ceph-mon[37057]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:54.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:54 smithi076 ceph-mon[37057]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:54 smithi017 ceph-mon[32549]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:54 smithi017 ceph-mon[32549]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:57:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:56 smithi017 ceph-mon[32549]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:56 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/1099203844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]: dispatch 2024-02-14T14:57:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:56 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]: dispatch 2024-02-14T14:57:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:56 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]': finished 2024-02-14T14:57:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:56 smithi017 ceph-mon[32549]: osdmap e48: 8 total, 7 up, 8 in 2024-02-14T14:57:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:57:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:56 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/4230658039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:56 smithi050 ceph-mon[36512]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:56 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/1099203844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]: dispatch 2024-02-14T14:57:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:56 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]: dispatch 2024-02-14T14:57:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:56 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]': finished 2024-02-14T14:57:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:56 smithi050 ceph-mon[36512]: osdmap e48: 8 total, 7 up, 8 in 2024-02-14T14:57:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:57:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:56 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/4230658039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:56 smithi076 ceph-mon[37057]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:56 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/1099203844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]: dispatch 2024-02-14T14:57:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:56 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]: dispatch 2024-02-14T14:57:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:56 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ea6e493-7d1b-4570-a7d0-2f50d2adf530"}]': finished 2024-02-14T14:57:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:56 smithi076 ceph-mon[37057]: osdmap e48: 8 total, 7 up, 8 in 2024-02-14T14:57:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:57:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:56 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/4230658039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:57:58.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:57:58 smithi017 ceph-mon[32549]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:57:58 smithi050 ceph-mon[36512]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:57:58.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:57:58 smithi076 ceph-mon[37057]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:00.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:00 smithi017 ceph-mon[32549]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:00.785 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:00 smithi050 ceph-mon[36512]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:00 smithi076 ceph-mon[37057]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:02 smithi017 ceph-mon[32549]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:02.847 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:02 smithi050 ceph-mon[36512]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:02 smithi076 ceph-mon[37057]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:03.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:03 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T14:58:03.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:03 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:03.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:03 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T14:58:03.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:03 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:03.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:03 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T14:58:03.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:03 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:04.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:04 smithi017 ceph-mon[32549]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:04.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:04 smithi017 ceph-mon[32549]: Deploying daemon osd.7 on smithi050 2024-02-14T14:58:04.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:04 smithi050 ceph-mon[36512]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:04.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:04 smithi050 ceph-mon[36512]: Deploying daemon osd.7 on smithi050 2024-02-14T14:58:04.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:04 smithi076 ceph-mon[37057]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:04 smithi076 ceph-mon[37057]: Deploying daemon osd.7 on smithi050 2024-02-14T14:58:06.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:06 smithi050 ceph-mon[36512]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:06.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:06 smithi076 ceph-mon[37057]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:07.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:06 smithi017 ceph-mon[32549]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:08.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:08 smithi050 ceph-mon[36512]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:08.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:08 smithi076 ceph-mon[37057]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:08 smithi017 ceph-mon[32549]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:09.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:09 smithi076 ceph-mon[37057]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:09.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:09.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:09.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:10.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:09 smithi017 ceph-mon[32549]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:10.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:10.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:10.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:10.092 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:09 smithi050 ceph-mon[36512]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:10.092 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:10.092 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:10.092 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:12.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.248+0000 7f5cce7fc700 1 -- 172.21.15.50:0/3549742729 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5cd802da10 con 0x7f5cc40a2150 2024-02-14T14:58:12.250 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 7 on host 'smithi050' 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5cc40a2150 msgr2=0x7f5cc40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5cc40a2150 0x7f5cc40a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009990 tx=0x7f5cd4005e80 comp rx=0 tx=0).stop 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8100c80 msgr2=0x7f5cd8196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8100c80 0x7f5cd8196be0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800d980 tx=0x7f5cc800dc90 comp rx=0 tx=0).stop 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 shutdown_connections 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5cc40a2150 0x7f5cc40a4610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd8102820 0x7f5cd819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:12.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cd8100c80 0x7f5cd8196be0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:12.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 --2- 172.21.15.50:0/3549742729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cd8101e80 0x7f5cd8197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:12.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 >> 172.21.15.50:0/3549742729 conn(0x7f5cd80fc240 msgr2=0x7f5cd8105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:58:12.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 shutdown_connections 2024-02-14T14:58:12.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:58:12.251+0000 7f5cdf2fb700 1 -- 172.21.15.50:0/3549742729 wait complete. 2024-02-14T14:58:12.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:12 smithi017 ceph-mon[32549]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:12.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:12.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:12.867 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:12 smithi050 ceph-mon[36512]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:12.867 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:12.867 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:12.896 DEBUG:teuthology.orchestra.run.smithi050:osd.7> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.7.service 2024-02-14T14:58:12.899 INFO:tasks.cephadm:Deploying osd.8 on smithi076 with /dev/vg_nvme/lv_2... 2024-02-14T14:58:12.899 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-14T14:58:12.929 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:12 smithi076 ceph-mon[37057]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:12.929 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:12.930 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:13.143 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:58:13.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:13 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:13.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:13 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:13.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:13 smithi017 ceph-mon[32549]: from='osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T14:58:13.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:13 smithi017 ceph-mon[32549]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T14:58:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:13 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:13 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:13.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:13 smithi050 ceph-mon[36512]: from='osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T14:58:13.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:13 smithi050 ceph-mon[36512]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T14:58:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:13 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:13 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:13 smithi076 ceph-mon[37057]: from='osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T14:58:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:13 smithi076 ceph-mon[37057]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T14:58:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-mon[36512]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-mon[36512]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T14:58:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-mon[36512]: from='osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:58:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-mon[36512]: osdmap e49: 8 total, 7 up, 8 in 2024-02-14T14:58:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-mon[36512]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:58:14.893 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T14:58:14.552+0000 7f5197bd5700 -1 osd.7 0 waiting for initial osdmap 2024-02-14T14:58:14.893 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 14:58:14 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T14:58:14.565+0000 7f51939cf700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:58:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:14 smithi076 ceph-mon[37057]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:14 smithi076 ceph-mon[37057]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T14:58:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:14 smithi076 ceph-mon[37057]: from='osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:58:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:14 smithi076 ceph-mon[37057]: osdmap e49: 8 total, 7 up, 8 in 2024-02-14T14:58:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:14 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:14.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:14 smithi076 ceph-mon[37057]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:58:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:14 smithi017 ceph-mon[32549]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:14 smithi017 ceph-mon[32549]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T14:58:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:14 smithi017 ceph-mon[32549]: from='osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:58:15.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:14 smithi017 ceph-mon[32549]: osdmap e49: 8 total, 7 up, 8 in 2024-02-14T14:58:15.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:14 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:15.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:14 smithi017 ceph-mon[32549]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:58:15.068 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-14T14:58:15.068 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-14T14:58:15.068 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-02-14T14:58:15.068 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-02-14T14:58:15.068 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113598 s, 923 MB/s 2024-02-14T14:58:15.069 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-02-14T14:58:15.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:15 smithi050 ceph-mon[36512]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:58:15.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:15 smithi050 ceph-mon[36512]: osdmap e50: 8 total, 7 up, 8 in 2024-02-14T14:58:15.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:15.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:15.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:15 smithi050 ceph-mon[36512]: osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] boot 2024-02-14T14:58:15.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:15 smithi050 ceph-mon[36512]: osdmap e51: 8 total, 8 up, 8 in 2024-02-14T14:58:15.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:15.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:15 smithi076 ceph-mon[37057]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:58:15.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:15 smithi076 ceph-mon[37057]: osdmap e50: 8 total, 7 up, 8 in 2024-02-14T14:58:15.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:15.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:15.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:15 smithi076 ceph-mon[37057]: osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] boot 2024-02-14T14:58:15.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:15 smithi076 ceph-mon[37057]: osdmap e51: 8 total, 8 up, 8 in 2024-02-14T14:58:15.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:15.789 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_2 2024-02-14T14:58:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:15 smithi017 ceph-mon[32549]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:58:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:15 smithi017 ceph-mon[32549]: osdmap e50: 8 total, 7 up, 8 in 2024-02-14T14:58:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:15 smithi017 ceph-mon[32549]: osd.7 [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] boot 2024-02-14T14:58:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:15 smithi017 ceph-mon[32549]: osdmap e51: 8 total, 8 up, 8 in 2024-02-14T14:58:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T14:58:16.076 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:58:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:58:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:58:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi050 2024-02-14T14:58:16.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi050 to 5694M 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:16.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:16.736 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.736 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:16 smithi076 ceph-mon[37057]: osdmap e52: 8 total, 8 up, 8 in 2024-02-14T14:58:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:58:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:58:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi050 2024-02-14T14:58:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi050 to 5694M 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:16.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:16.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:16.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:16 smithi050 ceph-mon[36512]: osdmap e52: 8 total, 8 up, 8 in 2024-02-14T14:58:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi050 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi050 to 5694M 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:17.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:17.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:17.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:17.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:17.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:17.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:17.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:16 smithi017 ceph-mon[32549]: osdmap e52: 8 total, 8 up, 8 in 2024-02-14T14:58:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 -- 172.21.15.76:0/3881648637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 msgr2=0x7f56581022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 --2- 172.21.15.76:0/3881648637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f56581022e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f564c009990 tx=0x7f564c009ca0 comp rx=0 tx=0).stop 2024-02-14T14:58:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 -- 172.21.15.76:0/3881648637 shutdown_connections 2024-02-14T14:58:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 --2- 172.21.15.76:0/3881648637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5658102820 0x7f56581090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 --2- 172.21.15.76:0/3881648637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5658100c80 0x7f56581010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 --2- 172.21.15.76:0/3881648637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f56581022e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 -- 172.21.15.76:0/3881648637 >> 172.21.15.76:0/3881648637 conn(0x7f56580fc220 msgr2=0x7f56580fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 -- 172.21.15.76:0/3881648637 shutdown_connections 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.848+0000 7f565df83700 1 -- 172.21.15.76:0/3881648637 wait complete. 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 Processor -- start 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 -- start start 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5658100c80 0x7f5658104c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f5658103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5658102820 0x7f56581037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5658103f20 con 0x7f5658102820 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5658104090 con 0x7f5658100c80 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f565df83700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581a1d70 con 0x7f5658101e80 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f5658103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f5658103260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44122/0 (socket says 172.21.15.76:44122) 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 -- 172.21.15.76:0/4216485362 learned_addr learned my addr 172.21.15.76:0/4216485362 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f56577fe700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5658100c80 0x7f5658104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5657fff700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5658102820 0x7f56581037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 -- 172.21.15.76:0/4216485362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5658100c80 msgr2=0x7f5658104c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5658100c80 0x7f5658104c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 -- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5658102820 msgr2=0x7f56581037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5658102820 0x7f56581037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5656ffd700 1 -- 172.21.15.76:0/4216485362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c0095f0 con 0x7f5658101e80 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.849+0000 7f5657fff700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5658102820 0x7f56581037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.850+0000 7f5656ffd700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f5658103260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f564c0095c0 tx=0x7f564c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.850+0000 7f5654ff9700 1 -- 172.21.15.76:0/4216485362 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c01d070 con 0x7f5658101e80 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.850+0000 7f5654ff9700 1 -- 172.21.15.76:0/4216485362 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f564c00fb40 con 0x7f5658101e80 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.850+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56581a1ff0 con 0x7f5658101e80 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.850+0000 7f5654ff9700 1 -- 172.21.15.76:0/4216485362 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c0173f0 con 0x7f5658101e80 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.850+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56581a23a0 con 0x7f5658101e80 2024-02-14T14:58:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.851+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5658068a10 con 0x7f5658101e80 2024-02-14T14:58:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.852+0000 7f5654ff9700 1 -- 172.21.15.76:0/4216485362 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f564c00fcb0 con 0x7f5658101e80 2024-02-14T14:58:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.852+0000 7f5654ff9700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56400a2150 0x7f56400a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.853+0000 7f5654ff9700 1 -- 172.21.15.76:0/4216485362 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f564c0ce500 con 0x7f5658101e80 2024-02-14T14:58:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.853+0000 7f56577fe700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56400a2150 0x7f56400a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.853+0000 7f56577fe700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56400a2150 0x7f56400a4610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5658101ce0 tx=0x7f564800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:58:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:17.856+0000 7f5654ff9700 1 -- 172.21.15.76:0/4216485362 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f564c0980a0 con 0x7f5658101e80 2024-02-14T14:58:18.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:18.053+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f56580fe3a0 con 0x7f56400a2150 2024-02-14T14:58:18.765 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:18 smithi076 ceph-mon[37057]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:18.766 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:18 smithi076 ceph-mon[37057]: osdmap e53: 8 total, 8 up, 8 in 2024-02-14T14:58:18.766 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:18 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:58:18.766 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:18 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:58:18.766 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:18 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:18.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:18 smithi017 ceph-mon[32549]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:18.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:18 smithi017 ceph-mon[32549]: osdmap e53: 8 total, 8 up, 8 in 2024-02-14T14:58:18.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:18 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:58:18.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:18 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:58:18.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:18 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:18.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:18 smithi050 ceph-mon[36512]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:18 smithi050 ceph-mon[36512]: osdmap e53: 8 total, 8 up, 8 in 2024-02-14T14:58:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:18 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:58:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:18 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:58:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:18 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:19.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:19 smithi017 ceph-mon[32549]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:58:19.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:19 smithi017 ceph-mon[32549]: osdmap e54: 8 total, 8 up, 8 in 2024-02-14T14:58:19.783 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:19 smithi076 ceph-mon[37057]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:58:19.784 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:19 smithi076 ceph-mon[37057]: osdmap e54: 8 total, 8 up, 8 in 2024-02-14T14:58:19.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:19 smithi050 ceph-mon[36512]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:58:19.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:19 smithi050 ceph-mon[36512]: osdmap e54: 8 total, 8 up, 8 in 2024-02-14T14:58:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:20 smithi017 ceph-mon[32549]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:20.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:20 smithi017 ceph-mon[32549]: osdmap e55: 8 total, 8 up, 8 in 2024-02-14T14:58:20.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:20 smithi050 ceph-mon[36512]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:20.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:20 smithi050 ceph-mon[36512]: osdmap e55: 8 total, 8 up, 8 in 2024-02-14T14:58:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:20 smithi076 ceph-mon[37057]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:20 smithi076 ceph-mon[37057]: osdmap e55: 8 total, 8 up, 8 in 2024-02-14T14:58:21.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:21 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/3684190134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]: dispatch 2024-02-14T14:58:21.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:21 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]: dispatch 2024-02-14T14:58:21.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:21 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]': finished 2024-02-14T14:58:21.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:21 smithi050 ceph-mon[36512]: osdmap e56: 9 total, 8 up, 9 in 2024-02-14T14:58:21.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:21.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:21 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/988720555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:58:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:21 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/3684190134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]: dispatch 2024-02-14T14:58:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:21 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]: dispatch 2024-02-14T14:58:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:21 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]': finished 2024-02-14T14:58:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:21 smithi076 ceph-mon[37057]: osdmap e56: 9 total, 8 up, 9 in 2024-02-14T14:58:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:21 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/988720555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:58:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:21 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/3684190134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]: dispatch 2024-02-14T14:58:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:21 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]: dispatch 2024-02-14T14:58:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:21 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4cf495a-56b2-45b3-9527-0ee4f829e144"}]': finished 2024-02-14T14:58:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:21 smithi017 ceph-mon[32549]: osdmap e56: 9 total, 8 up, 9 in 2024-02-14T14:58:22.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:22.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:21 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/988720555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:58:22.794 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:22 smithi050 ceph-mon[36512]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:22.886 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:22 smithi076 ceph-mon[37057]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:22 smithi017 ceph-mon[32549]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:24.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:24 smithi076 ceph-mon[37057]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:24.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:24 smithi050 ceph-mon[36512]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:24 smithi017 ceph-mon[32549]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:26.834 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:26 smithi076 ceph-mon[37057]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:26.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:26 smithi050 ceph-mon[36512]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:26 smithi017 ceph-mon[32549]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:28.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:28 smithi076 ceph-mon[37057]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:28.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:28 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-14T14:58:28.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:28 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:28.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:28 smithi050 ceph-mon[36512]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:28.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:28 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-14T14:58:28.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:28 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:28 smithi017 ceph-mon[32549]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:28 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-14T14:58:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:28 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:29.785 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:29 smithi076 ceph-mon[37057]: Deploying daemon osd.8 on smithi076 2024-02-14T14:58:29.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:29 smithi050 ceph-mon[36512]: Deploying daemon osd.8 on smithi076 2024-02-14T14:58:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:29 smithi017 ceph-mon[32549]: Deploying daemon osd.8 on smithi076 2024-02-14T14:58:30.798 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:30 smithi076 ceph-mon[37057]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:30 smithi050 ceph-mon[36512]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:30 smithi017 ceph-mon[32549]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:32 smithi076 ceph-mon[37057]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:32.819 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:32 smithi017 ceph-mon[32549]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:32 smithi050 ceph-mon[36512]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:34.672 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:34 smithi076 ceph-mon[37057]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:34.672 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:34.672 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:34.672 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:34.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:34 smithi050 ceph-mon[36512]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:34.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:34.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:34.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:34.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:34 smithi017 ceph-mon[32549]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:34.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:34.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:34.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:36 smithi050 ceph-mon[36512]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:36.897 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:36 smithi076 ceph-mon[37057]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:36 smithi017 ceph-mon[32549]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:37.272 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 8 on host 'smithi076' 2024-02-14T14:58:37.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.271+0000 7f5654ff9700 1 -- 172.21.15.76:0/4216485362 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f56580fe3a0 con 0x7f56400a2150 2024-02-14T14:58:37.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56400a2150 msgr2=0x7f56400a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56400a2150 0x7f56400a4610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5658101ce0 tx=0x7f564800b410 comp rx=0 tx=0).stop 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 msgr2=0x7f5658103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f5658103260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f564c0095c0 tx=0x7f564c00f690 comp rx=0 tx=0).stop 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 shutdown_connections 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56400a2150 0x7f56400a4610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5658102820 0x7f56581037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5658100c80 0x7f5658104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 --2- 172.21.15.76:0/4216485362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5658101e80 0x7f5658103260 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.273+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 >> 172.21.15.76:0/4216485362 conn(0x7f56580fc220 msgr2=0x7f56580fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:58:37.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.274+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 shutdown_connections 2024-02-14T14:58:37.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:58:37.274+0000 7f565df83700 1 -- 172.21.15.76:0/4216485362 wait complete. 2024-02-14T14:58:37.954 DEBUG:teuthology.orchestra.run.smithi076:osd.8> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.8.service 2024-02-14T14:58:37.957 INFO:tasks.cephadm:Deploying osd.9 on smithi017 with /dev/vg_nvme/lv_1... 2024-02-14T14:58:37.957 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-14T14:58:38.213 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:58:38.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:38 smithi017 ceph-mon[32549]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:38.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:38 smithi017 ceph-mon[32549]: from='osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-14T14:58:38.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:38 smithi017 ceph-mon[32549]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-14T14:58:38.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:38 smithi076 ceph-mon[37057]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:38.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:38 smithi076 ceph-mon[37057]: from='osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-14T14:58:38.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:38 smithi076 ceph-mon[37057]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-14T14:58:38.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:38 smithi050 ceph-mon[36512]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:38.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:38 smithi050 ceph-mon[36512]: from='osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-14T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:38 smithi050 ceph-mon[36512]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-14T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:38.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:39.685 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:39 smithi017 ceph-mon[32549]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-14T14:58:39.686 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:39 smithi017 ceph-mon[32549]: osdmap e57: 9 total, 8 up, 9 in 2024-02-14T14:58:39.686 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:39 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:39.686 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:39 smithi017 ceph-mon[32549]: from='osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:58:39.686 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:39 smithi017 ceph-mon[32549]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:58:39.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:39 smithi076 ceph-mon[37057]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-14T14:58:39.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:39 smithi076 ceph-mon[37057]: osdmap e57: 9 total, 8 up, 9 in 2024-02-14T14:58:39.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:39 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:39.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:39 smithi076 ceph-mon[37057]: from='osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:58:39.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:39 smithi076 ceph-mon[37057]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:58:39.858 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 14:58:39 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T14:58:39.563+0000 7f1a00946700 -1 osd.8 0 waiting for initial osdmap 2024-02-14T14:58:39.859 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 14:58:39 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T14:58:39.577+0000 7f19fc740700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:58:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:39 smithi050 ceph-mon[36512]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-14T14:58:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:39 smithi050 ceph-mon[36512]: osdmap e57: 9 total, 8 up, 9 in 2024-02-14T14:58:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:39 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:39.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:39 smithi050 ceph-mon[36512]: from='osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:58:39.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:39 smithi050 ceph-mon[36512]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:58:40.424 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-14T14:58:40.424 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-14T14:58:40.424 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2024-02-14T14:58:40.424 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2024-02-14T14:58:40.424 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852308 s, 1.2 GB/s 2024-02-14T14:58:40.424 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2024-02-14T14:58:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:58:40.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: osdmap e58: 9 total, 8 up, 9 in 2024-02-14T14:58:40.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] boot 2024-02-14T14:58:40.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: osdmap e59: 9 total, 9 up, 9 in 2024-02-14T14:58:40.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.860 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:40.860 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:58:40.861 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: osdmap e58: 9 total, 8 up, 9 in 2024-02-14T14:58:40.861 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.861 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.861 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] boot 2024-02-14T14:58:40.861 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: osdmap e59: 9 total, 9 up, 9 in 2024-02-14T14:58:40.861 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-02-14T14:58:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:58:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: osdmap e58: 9 total, 8 up, 9 in 2024-02-14T14:58:40.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:40.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: osd.8 [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] boot 2024-02-14T14:58:40.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: osdmap e59: 9 total, 9 up, 9 in 2024-02-14T14:58:40.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T14:58:41.144 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi017:vg_nvme/lv_1 2024-02-14T14:58:41.391 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:58:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:41 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:58:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:41 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:58:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:41 smithi017 ceph-mon[32549]: osdmap e60: 9 total, 9 up, 9 in 2024-02-14T14:58:41.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:41 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:58:41.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:41 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:58:41.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:41 smithi050 ceph-mon[36512]: osdmap e60: 9 total, 9 up, 9 in 2024-02-14T14:58:41.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:41 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:58:41.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:41 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:58:41.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:41 smithi076 ceph-mon[37057]: osdmap e60: 9 total, 9 up, 9 in 2024-02-14T14:58:42.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:42.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:42 smithi017 ceph-mon[32549]: osdmap e61: 9 total, 9 up, 9 in 2024-02-14T14:58:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:42.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:42.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:42.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:42.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:42.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:42 smithi050 ceph-mon[36512]: osdmap e61: 9 total, 9 up, 9 in 2024-02-14T14:58:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:58:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:58:42.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:42 smithi076 ceph-mon[37057]: osdmap e61: 9 total, 9 up, 9 in 2024-02-14T14:58:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 -- 172.21.15.17:0/3664338033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 msgr2=0x7fdbb41090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/3664338033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb41090a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00b0d0 tx=0x7fdbac00b3e0 comp rx=0 tx=0).stop 2024-02-14T14:58:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 -- 172.21.15.17:0/3664338033 shutdown_connections 2024-02-14T14:58:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/3664338033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb41090a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/3664338033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbb4100c50 0x7fdbb4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/3664338033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbb4101e50 0x7fdbb41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 -- 172.21.15.17:0/3664338033 >> 172.21.15.17:0/3664338033 conn(0x7fdbb40fc1f0 msgr2=0x7fdbb40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 -- 172.21.15.17:0/3664338033 shutdown_connections 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.091+0000 7fdbb9fda700 1 -- 172.21.15.17:0/3664338033 wait complete. 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 Processor -- start 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 -- start start 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbb4100c50 0x7fdbb4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbb4101e50 0x7fdbb4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb419f9e0 con 0x7fdbb41027f0 2024-02-14T14:58:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb419fb20 con 0x7fdbb4100c50 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb9fda700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb419fcb0 con 0x7fdbb4101e50 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39430/0 (socket says 172.21.15.17:39430) 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 -- 172.21.15.17:0/1026507471 learned_addr learned my addr 172.21.15.17:0/1026507471 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbaaffd700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbb4101e50 0x7fdbb4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb37fe700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbb4100c50 0x7fdbb4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 -- 172.21.15.17:0/1026507471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbb4101e50 msgr2=0x7fdbb4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbb4101e50 0x7fdbb4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 -- 172.21.15.17:0/1026507471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbb4100c50 msgr2=0x7fdbb4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:58:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbb4100c50 0x7fdbb4197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.092+0000 7fdbb3fff700 1 -- 172.21.15.17:0/1026507471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac009d00 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.093+0000 7fdbb3fff700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb419f410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0078d0 tx=0x7fdbac0079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.093+0000 7fdbb17fa700 1 -- 172.21.15.17:0/1026507471 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00e040 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.093+0000 7fdbb17fa700 1 -- 172.21.15.17:0/1026507471 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbac006040 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.093+0000 7fdbb17fa700 1 -- 172.21.15.17:0/1026507471 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac01ba60 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.093+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb419ff30 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.093+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb41a03c0 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.094+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb40689e0 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.095+0000 7fdbb17fa700 1 -- 172.21.15.17:0/1026507471 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdbac01bbc0 con 0x7fdbb41027f0 2024-02-14T14:58:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.096+0000 7fdbb17fa700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdba00a21d0 0x7fdba00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:58:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.096+0000 7fdbb37fe700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdba00a21d0 0x7fdba00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:58:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.096+0000 7fdbb17fa700 1 -- 172.21.15.17:0/1026507471 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fdbac0cefe0 con 0x7fdbb41027f0 2024-02-14T14:58:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.096+0000 7fdbb37fe700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdba00a21d0 0x7fdba00a4690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdba4005950 tx=0x7fdba40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:58:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.100+0000 7fdbb17fa700 1 -- 172.21.15.17:0/1026507471 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdbac098950 con 0x7fdbb41027f0 2024-02-14T14:58:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:58:43.314+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbb402d920 con 0x7fdba00a21d0 2024-02-14T14:58:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:43 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi076 2024-02-14T14:58:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:43 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi076 to 5693M 2024-02-14T14:58:43.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:43 smithi050 ceph-mon[36512]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:43.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:43 smithi050 ceph-mon[36512]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:58:43.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:58:43.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:58:43.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:43.933 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:43 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi076 2024-02-14T14:58:43.934 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:43 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi076 to 5693M 2024-02-14T14:58:43.934 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:43 smithi017 ceph-mon[32549]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:43.934 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:43 smithi017 ceph-mon[32549]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:58:43.934 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:58:43.934 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:58:43.934 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:43 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi076 2024-02-14T14:58:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:43 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi076 to 5693M 2024-02-14T14:58:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:43 smithi076 ceph-mon[37057]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:43 smithi076 ceph-mon[37057]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:58:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:58:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:58:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:44.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:44 smithi017 ceph-mon[32549]: osdmap e62: 9 total, 9 up, 9 in 2024-02-14T14:58:44.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:44 smithi050 ceph-mon[36512]: osdmap e62: 9 total, 9 up, 9 in 2024-02-14T14:58:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:44 smithi076 ceph-mon[37057]: osdmap e62: 9 total, 9 up, 9 in 2024-02-14T14:58:45.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:45 smithi017 ceph-mon[32549]: osdmap e63: 9 total, 9 up, 9 in 2024-02-14T14:58:45.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:45 smithi017 ceph-mon[32549]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:45.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:45 smithi050 ceph-mon[36512]: osdmap e63: 9 total, 9 up, 9 in 2024-02-14T14:58:45.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:45 smithi050 ceph-mon[36512]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:45 smithi076 ceph-mon[37057]: osdmap e63: 9 total, 9 up, 9 in 2024-02-14T14:58:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:45 smithi076 ceph-mon[37057]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/407721570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b22fd6-b721-41e5-b910-db93fa49a826"}]: dispatch 2024-02-14T14:58:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/407721570' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b22fd6-b721-41e5-b910-db93fa49a826"}]': finished 2024-02-14T14:58:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:46 smithi050 ceph-mon[36512]: osdmap e64: 10 total, 9 up, 10 in 2024-02-14T14:58:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:58:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/407721570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b22fd6-b721-41e5-b910-db93fa49a826"}]: dispatch 2024-02-14T14:58:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/407721570' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b22fd6-b721-41e5-b910-db93fa49a826"}]': finished 2024-02-14T14:58:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:46 smithi076 ceph-mon[37057]: osdmap e64: 10 total, 9 up, 10 in 2024-02-14T14:58:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:58:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/407721570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b22fd6-b721-41e5-b910-db93fa49a826"}]: dispatch 2024-02-14T14:58:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/407721570' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b22fd6-b721-41e5-b910-db93fa49a826"}]': finished 2024-02-14T14:58:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:46 smithi017 ceph-mon[32549]: osdmap e64: 10 total, 9 up, 10 in 2024-02-14T14:58:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:58:47.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:47 smithi050 ceph-mon[36512]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-02-14T14:58:47.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:47 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/690462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:58:47.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:47 smithi076 ceph-mon[37057]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-02-14T14:58:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:47 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/690462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:58:48.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:47 smithi017 ceph-mon[32549]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-02-14T14:58:48.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:47 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/690462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:58:49.936 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:49 smithi017 ceph-mon[32549]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:58:49.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:49 smithi076 ceph-mon[37057]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:58:50.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:49 smithi050 ceph-mon[36512]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:58:52.640 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:52 smithi017 ceph-mon[32549]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-02-14T14:58:52.845 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:52 smithi050 ceph-mon[36512]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-02-14T14:58:52.963 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:52 smithi076 ceph-mon[37057]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-02-14T14:58:54.794 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:54 smithi017 ceph-mon[32549]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T14:58:54.795 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:54 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-14T14:58:54.795 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:54 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:54.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:54 smithi050 ceph-mon[36512]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T14:58:54.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:54 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-14T14:58:54.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:54 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:54 smithi076 ceph-mon[37057]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T14:58:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:54 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-14T14:58:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:54 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:58:55.873 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:55 smithi017 ceph-mon[32549]: Deploying daemon osd.9 on smithi017 2024-02-14T14:58:55.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:55 smithi050 ceph-mon[36512]: Deploying daemon osd.9 on smithi017 2024-02-14T14:58:55.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:55 smithi076 ceph-mon[37057]: Deploying daemon osd.9 on smithi017 2024-02-14T14:58:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:56 smithi017 ceph-mon[32549]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:58:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:56 smithi050 ceph-mon[36512]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:58:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:56 smithi076 ceph-mon[37057]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T14:58:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:58:58 smithi050 ceph-mon[36512]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:58.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:58:58 smithi076 ceph-mon[37057]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:58:58.999 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:58:58 smithi017 ceph-mon[32549]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:00.802 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:00 smithi017 ceph-mon[32549]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:00.802 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:00.802 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:00.802 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:00 smithi050 ceph-mon[36512]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:00 smithi076 ceph-mon[37057]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:02 smithi017 ceph-mon[32549]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:02 smithi050 ceph-mon[36512]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:02.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:02 smithi076 ceph-mon[37057]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:04.073 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 9 on host 'smithi017' 2024-02-14T14:59:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.072+0000 7fdbb17fa700 1 -- 172.21.15.17:0/1026507471 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdbb402d920 con 0x7fdba00a21d0 2024-02-14T14:59:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.075+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdba00a21d0 msgr2=0x7fdba00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.075+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdba00a21d0 0x7fdba00a4690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdba4005950 tx=0x7fdba40058e0 comp rx=0 tx=0).stop 2024-02-14T14:59:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.075+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 msgr2=0x7fdbb419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.075+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb419f410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0078d0 tx=0x7fdbac0079b0 comp rx=0 tx=0).stop 2024-02-14T14:59:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 shutdown_connections 2024-02-14T14:59:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdba00a21d0 0x7fdba00a4690 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbb41027f0 0x7fdbb419f410 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbb4100c50 0x7fdbb4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 --2- 172.21.15.17:0/1026507471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbb4101e50 0x7fdbb4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 >> 172.21.15.17:0/1026507471 conn(0x7fdbb40fc1f0 msgr2=0x7fdbb41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:59:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 shutdown_connections 2024-02-14T14:59:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:04.076+0000 7fdbb9fda700 1 -- 172.21.15.17:0/1026507471 wait complete. 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:04.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-14T14:59:04.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:04 smithi076 ceph-mon[37057]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:04.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-14T14:59:04.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:04 smithi017 ceph-mon[32549]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-14T14:59:04.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:04.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-14T14:59:04.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:04 smithi050 ceph-mon[36512]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-14T14:59:04.821 DEBUG:teuthology.orchestra.run.smithi017:osd.9> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.9.service 2024-02-14T14:59:04.823 INFO:tasks.cephadm:Deploying osd.10 on smithi050 with /dev/vg_nvme/lv_1... 2024-02-14T14:59:04.824 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-14T14:59:05.086 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:59:06.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:06 smithi050 ceph-mon[36512]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:06.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:06 smithi050 ceph-mon[36512]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-14T14:59:06.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:06 smithi050 ceph-mon[36512]: osdmap e65: 10 total, 9 up, 10 in 2024-02-14T14:59:06.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:06 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:06.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:06 smithi050 ceph-mon[36512]: from='osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:59:06.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:06 smithi050 ceph-mon[36512]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:59:06.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-mon[32549]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:06.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-mon[32549]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-14T14:59:06.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-mon[32549]: osdmap e65: 10 total, 9 up, 10 in 2024-02-14T14:59:06.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:06.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-mon[32549]: from='osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:59:06.386 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-mon[32549]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:59:06.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:06 smithi076 ceph-mon[37057]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:06.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:06 smithi076 ceph-mon[37057]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-14T14:59:06.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:06 smithi076 ceph-mon[37057]: osdmap e65: 10 total, 9 up, 10 in 2024-02-14T14:59:06.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:06 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:06.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:06 smithi076 ceph-mon[37057]: from='osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:59:06.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:06 smithi076 ceph-mon[37057]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2024-02-14T14:59:06.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T14:59:06.612+0000 7f0ca5bc2700 -1 osd.9 0 waiting for initial osdmap 2024-02-14T14:59:06.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 14:59:06 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T14:59:06.623+0000 7f0c9d9b4700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:59:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-14T14:59:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-14T14:59:07.205 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-14T14:59:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-14T14:59:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852922 s, 1.2 GB/s 2024-02-14T14:59:07.205 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-14T14:59:07.205 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:07 smithi050 ceph-mon[36512]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:59:07.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:07 smithi050 ceph-mon[36512]: osdmap e66: 10 total, 9 up, 10 in 2024-02-14T14:59:07.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:07 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:07.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:07 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:07.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:07 smithi050 ceph-mon[36512]: from='osd.9 ' entity='osd.9' 2024-02-14T14:59:07.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:07 smithi076 ceph-mon[37057]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:59:07.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:07 smithi076 ceph-mon[37057]: osdmap e66: 10 total, 9 up, 10 in 2024-02-14T14:59:07.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:07 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:07.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:07 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:07.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:07 smithi076 ceph-mon[37057]: from='osd.9 ' entity='osd.9' 2024-02-14T14:59:07.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:07 smithi017 ceph-mon[32549]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2024-02-14T14:59:07.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:07 smithi017 ceph-mon[32549]: osdmap e66: 10 total, 9 up, 10 in 2024-02-14T14:59:07.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:07 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:07.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:07 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:07.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:07 smithi017 ceph-mon[32549]: from='osd.9 ' entity='osd.9' 2024-02-14T14:59:07.943 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-02-14T14:59:08.198 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T14:59:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:08 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:59:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:08 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:59:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:08 smithi050 ceph-mon[36512]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:08 smithi050 ceph-mon[36512]: osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] boot 2024-02-14T14:59:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:08 smithi050 ceph-mon[36512]: osdmap e67: 10 total, 10 up, 10 in 2024-02-14T14:59:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:08 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:08.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:08 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:59:08.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:08 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:59:08.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:08 smithi076 ceph-mon[37057]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:08.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:08.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:08 smithi076 ceph-mon[37057]: osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] boot 2024-02-14T14:59:08.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:08 smithi076 ceph-mon[37057]: osdmap e67: 10 total, 10 up, 10 in 2024-02-14T14:59:08.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:08 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:08.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:08 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:59:08.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:08 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:59:08.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:08 smithi017 ceph-mon[32549]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-02-14T14:59:08.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:08.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:08 smithi017 ceph-mon[32549]: osd.9 [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] boot 2024-02-14T14:59:08.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:08 smithi017 ceph-mon[32549]: osdmap e67: 10 total, 10 up, 10 in 2024-02-14T14:59:08.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:08 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T14:59:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi017 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi017 to 4273M 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:09 smithi050 ceph-mon[36512]: osdmap e68: 10 total, 10 up, 10 in 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi017 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi017 to 4273M 2024-02-14T14:59:09.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:09.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:09.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:09 smithi076 ceph-mon[37057]: osdmap e68: 10 total, 10 up, 10 in 2024-02-14T14:59:09.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi017 2024-02-14T14:59:09.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi017 to 4273M 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:09.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:09 smithi017 ceph-mon[32549]: osdmap e68: 10 total, 10 up, 10 in 2024-02-14T14:59:10.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 -- 172.21.15.50:0/1577966493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 msgr2=0x7f2100109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:10.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 --2- 172.21.15.50:0/1577966493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f2100109f50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20f800b0a0 tx=0x7f20f800b3b0 comp rx=0 tx=0).stop 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 -- 172.21.15.50:0/1577966493 shutdown_connections 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 --2- 172.21.15.50:0/1577966493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f2100109f50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 --2- 172.21.15.50:0/1577966493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21001028e0 0x7f2100104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 --2- 172.21.15.50:0/1577966493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2100105210 0x7f2100107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 -- 172.21.15.50:0/1577966493 >> 172.21.15.50:0/1577966493 conn(0x7f21000fc2b0 msgr2=0x7f21000fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 -- 172.21.15.50:0/1577966493 shutdown_connections 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.002+0000 7f2106d0a700 1 -- 172.21.15.50:0/1577966493 wait complete. 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 Processor -- start 2024-02-14T14:59:10.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 -- start start 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21001028e0 0x7f2100197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2100105210 0x7f2100198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f210019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210019f980 con 0x7f2100105210 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210019fac0 con 0x7f2100107b40 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2106d0a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210019fc50 con 0x7f21001028e0 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f21052a7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f210019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f21052a7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f210019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52288/0 (socket says 172.21.15.50:52288) 2024-02-14T14:59:10.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f21052a7700 1 -- 172.21.15.50:0/3865853730 learned_addr learned my addr 172.21.15.50:0/3865853730 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.003+0000 7f2104aa6700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21001028e0 0x7f2100197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f20fffff700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2100105210 0x7f2100198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f21052a7700 1 -- 172.21.15.50:0/3865853730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21001028e0 msgr2=0x7f2100197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f21052a7700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21001028e0 0x7f2100197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f21052a7700 1 -- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2100105210 msgr2=0x7f2100198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f21052a7700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2100105210 0x7f2100198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f21052a7700 1 -- 172.21.15.50:0/3865853730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8009d00 con 0x7f2100107b40 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f20fffff700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2100105210 0x7f2100198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f21052a7700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f210019f3b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f20f80093f0 tx=0x7f20f8009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:59:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f20fdffb700 1 -- 172.21.15.50:0/3865853730 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800e040 con 0x7f2100107b40 2024-02-14T14:59:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210019fed0 con 0x7f2100107b40 2024-02-14T14:59:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f20fdffb700 1 -- 172.21.15.50:0/3865853730 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20f8006040 con 0x7f2100107b40 2024-02-14T14:59:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f20fdffb700 1 -- 172.21.15.50:0/3865853730 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f801b990 con 0x7f2100107b40 2024-02-14T14:59:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.004+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21001a02f0 con 0x7f2100107b40 2024-02-14T14:59:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.005+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21000fde30 con 0x7f2100107b40 2024-02-14T14:59:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.006+0000 7f20fdffb700 1 -- 172.21.15.50:0/3865853730 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f20f801bb10 con 0x7f2100107b40 2024-02-14T14:59:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.007+0000 7f20fdffb700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f20f00a2150 0x7f20f00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.007+0000 7f20fdffb700 1 -- 172.21.15.50:0/3865853730 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f20f80cf0e0 con 0x7f2100107b40 2024-02-14T14:59:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.007+0000 7f2104aa6700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f20f00a2150 0x7f20f00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.008+0000 7f2104aa6700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f20f00a2150 0x7f20f00a4610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20f4005950 tx=0x7f20f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:59:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.011+0000 7f20fdffb700 1 -- 172.21.15.50:0/3865853730 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20f80988a0 con 0x7f2100107b40 2024-02-14T14:59:10.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:10.223+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f210002d920 con 0x7f20f00a2150 2024-02-14T14:59:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:10 smithi050 ceph-mon[36512]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:10.483 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:10 smithi076 ceph-mon[37057]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:10.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:10 smithi017 ceph-mon[32549]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:11 smithi050 ceph-mon[36512]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:59:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:11 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:59:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:11 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:59:11.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:11 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:11.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:11 smithi076 ceph-mon[37057]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:59:11.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:11 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:59:11.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:11 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:59:11.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:11 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:11.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:11 smithi017 ceph-mon[32549]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:59:11.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:11 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:59:11.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:11 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:59:11.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:11 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:12.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:12 smithi050 ceph-mon[36512]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:12 smithi076 ceph-mon[37057]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:12 smithi017 ceph-mon[32549]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:13.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/720047794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]: dispatch 2024-02-14T14:59:13.892 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:13 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]: dispatch 2024-02-14T14:59:13.892 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:13 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]': finished 2024-02-14T14:59:13.892 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:13 smithi017 ceph-mon[32549]: osdmap e69: 11 total, 10 up, 11 in 2024-02-14T14:59:13.892 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:13 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/720047794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]: dispatch 2024-02-14T14:59:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:13 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]: dispatch 2024-02-14T14:59:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:13 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]': finished 2024-02-14T14:59:13.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:13 smithi050 ceph-mon[36512]: osdmap e69: 11 total, 10 up, 11 in 2024-02-14T14:59:13.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:13 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/720047794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]: dispatch 2024-02-14T14:59:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:13 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]: dispatch 2024-02-14T14:59:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:13 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4f225cb-2077-48f1-ab76-3e94e0566653"}]': finished 2024-02-14T14:59:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:13 smithi076 ceph-mon[37057]: osdmap e69: 11 total, 10 up, 11 in 2024-02-14T14:59:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:13 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:14.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:14 smithi017 ceph-mon[32549]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:14.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:14 smithi017 ceph-mon[32549]: from='client.? 172.21.15.50:0/539098570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:59:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:14 smithi050 ceph-mon[36512]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:14 smithi050 ceph-mon[36512]: from='client.? 172.21.15.50:0/539098570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:59:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:14 smithi076 ceph-mon[37057]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:14 smithi076 ceph-mon[37057]: from='client.? 172.21.15.50:0/539098570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:59:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:16 smithi050 ceph-mon[36512]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:16.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:16 smithi076 ceph-mon[37057]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:16 smithi017 ceph-mon[32549]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:18.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:18 smithi050 ceph-mon[36512]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:18 smithi076 ceph-mon[37057]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:18 smithi017 ceph-mon[32549]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:20.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:20 smithi050 ceph-mon[36512]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:20.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-14T14:59:20.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:20 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:20 smithi076 ceph-mon[37057]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-14T14:59:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:20 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:20 smithi017 ceph-mon[32549]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-14T14:59:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:20 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:21.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:21 smithi050 ceph-mon[36512]: Deploying daemon osd.10 on smithi050 2024-02-14T14:59:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:21 smithi076 ceph-mon[37057]: Deploying daemon osd.10 on smithi050 2024-02-14T14:59:22.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:21 smithi017 ceph-mon[32549]: Deploying daemon osd.10 on smithi050 2024-02-14T14:59:22.885 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:22 smithi076 ceph-mon[37057]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:22.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:22 smithi050 ceph-mon[36512]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:22 smithi017 ceph-mon[32549]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:24.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:24 smithi050 ceph-mon[36512]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:24 smithi076 ceph-mon[37057]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:24 smithi017 ceph-mon[32549]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:26.796 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:26 smithi050 ceph-mon[36512]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:26.796 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:26.796 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:26.796 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:26 smithi076 ceph-mon[37057]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:26 smithi017 ceph-mon[32549]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:28.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:28 smithi050 ceph-mon[36512]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:28 smithi076 ceph-mon[37057]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:28 smithi017 ceph-mon[32549]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:29.642 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 10 on host 'smithi050' 2024-02-14T14:59:29.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.640+0000 7f20fdffb700 1 -- 172.21.15.50:0/3865853730 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f210002d920 con 0x7f20f00a2150 2024-02-14T14:59:29.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f20f00a2150 msgr2=0x7f20f00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:29.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f20f00a2150 0x7f20f00a4610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20f4005950 tx=0x7f20f400b410 comp rx=0 tx=0).stop 2024-02-14T14:59:29.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 msgr2=0x7f210019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:29.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f210019f3b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f20f80093f0 tx=0x7f20f8009420 comp rx=0 tx=0).stop 2024-02-14T14:59:29.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 shutdown_connections 2024-02-14T14:59:29.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f20f00a2150 0x7f20f00a4610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:29.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2100107b40 0x7f210019f3b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:29.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21001028e0 0x7f2100197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:29.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 --2- 172.21.15.50:0/3865853730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2100105210 0x7f2100198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:29.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 >> 172.21.15.50:0/3865853730 conn(0x7f21000fc2b0 msgr2=0x7f2100103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:59:29.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 shutdown_connections 2024-02-14T14:59:29.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-14T14:59:29.643+0000 7f2106d0a700 1 -- 172.21.15.50:0/3865853730 wait complete. 2024-02-14T14:59:30.272 DEBUG:teuthology.orchestra.run.smithi050:osd.10> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.10.service 2024-02-14T14:59:30.274 INFO:tasks.cephadm:Deploying osd.11 on smithi076 with /dev/vg_nvme/lv_1... 2024-02-14T14:59:30.274 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-14T14:59:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:59:30.555 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:30 smithi076 ceph-mon[37057]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:30.555 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:30.555 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:30.555 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:30 smithi076 ceph-mon[37057]: from='osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-14T14:59:30.885 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:30 smithi076 ceph-mon[37057]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-14T14:59:30.886 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:30.886 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:30 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:30 smithi050 ceph-mon[36512]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:30 smithi050 ceph-mon[36512]: from='osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-14T14:59:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:30 smithi050 ceph-mon[36512]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-14T14:59:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:30 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:30 smithi017 ceph-mon[32549]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:30 smithi017 ceph-mon[32549]: from='osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-14T14:59:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:30 smithi017 ceph-mon[32549]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-14T14:59:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:30 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:31.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:31 smithi050 ceph-mon[36512]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-14T14:59:31.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:31 smithi050 ceph-mon[36512]: from='osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:59:31.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:31 smithi050 ceph-mon[36512]: osdmap e70: 11 total, 10 up, 11 in 2024-02-14T14:59:31.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:31 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:31.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:31 smithi050 ceph-mon[36512]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:59:31.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:31 smithi050 ceph-mon[36512]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:31.901 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:31 smithi076 ceph-mon[37057]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-14T14:59:31.901 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:31 smithi076 ceph-mon[37057]: from='osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:59:31.901 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:31 smithi076 ceph-mon[37057]: osdmap e70: 11 total, 10 up, 11 in 2024-02-14T14:59:31.901 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:31 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:31.901 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:31 smithi076 ceph-mon[37057]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:59:31.901 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:31 smithi076 ceph-mon[37057]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:31 smithi017 ceph-mon[32549]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-14T14:59:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:31 smithi017 ceph-mon[32549]: from='osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:59:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:31 smithi017 ceph-mon[32549]: osdmap e70: 11 total, 10 up, 11 in 2024-02-14T14:59:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:31 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:31 smithi017 ceph-mon[32549]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-14T14:59:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:31 smithi017 ceph-mon[32549]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:32.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 14:59:32 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T14:59:32.142+0000 7f28e0c31700 -1 osd.10 0 waiting for initial osdmap 2024-02-14T14:59:32.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 14:59:32 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T14:59:32.155+0000 7f28d8a23700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:59:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-14T14:59:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-14T14:59:32.651 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-02-14T14:59:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-02-14T14:59:32.651 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112173 s, 935 MB/s 2024-02-14T14:59:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-02-14T14:59:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:32 smithi076 ceph-mon[37057]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:59:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:32 smithi076 ceph-mon[37057]: osdmap e71: 11 total, 10 up, 11 in 2024-02-14T14:59:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:32.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:32 smithi076 ceph-mon[37057]: from='osd.10 ' entity='osd.10' 2024-02-14T14:59:32.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:32 smithi050 ceph-mon[36512]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:59:32.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:32 smithi050 ceph-mon[36512]: osdmap e71: 11 total, 10 up, 11 in 2024-02-14T14:59:32.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:32.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:32.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:32 smithi050 ceph-mon[36512]: from='osd.10 ' entity='osd.10' 2024-02-14T14:59:32.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:32 smithi017 ceph-mon[32549]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-14T14:59:32.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:32 smithi017 ceph-mon[32549]: osdmap e71: 11 total, 10 up, 11 in 2024-02-14T14:59:32.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:32.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:32.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:32 smithi017 ceph-mon[32549]: from='osd.10 ' entity='osd.10' 2024-02-14T14:59:33.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_1 2024-02-14T14:59:33.727 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T14:59:33.733 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:33 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:59:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:33 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:59:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:33 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:33 smithi076 ceph-mon[37057]: osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] boot 2024-02-14T14:59:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:33 smithi076 ceph-mon[37057]: osdmap e72: 11 total, 11 up, 11 in 2024-02-14T14:59:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:33 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:33 smithi076 ceph-mon[37057]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:33.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:33 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:59:33.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:33 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:59:33.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:33 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:33.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:33 smithi050 ceph-mon[36512]: osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] boot 2024-02-14T14:59:33.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:33 smithi050 ceph-mon[36512]: osdmap e72: 11 total, 11 up, 11 in 2024-02-14T14:59:33.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:33 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:33.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:33 smithi050 ceph-mon[36512]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:33 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:59:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:33 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:59:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:33 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:33 smithi017 ceph-mon[32549]: osd.10 [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] boot 2024-02-14T14:59:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:33 smithi017 ceph-mon[32549]: osdmap e72: 11 total, 11 up, 11 in 2024-02-14T14:59:34.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:33 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T14:59:34.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:33 smithi017 ceph-mon[32549]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: osdmap e73: 11 total, 11 up, 11 in 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi050 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi050 to 4271M 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:34.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:34.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:34.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:34.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:34.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:34 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.903 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: osdmap e73: 11 total, 11 up, 11 in 2024-02-14T14:59:34.903 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi050 2024-02-14T14:59:34.903 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi050 to 4271M 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:34.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:34 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: osdmap e73: 11 total, 11 up, 11 in 2024-02-14T14:59:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi050 2024-02-14T14:59:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi050 to 4271M 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:35.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:34 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 -- 172.21.15.76:0/967550422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 msgr2=0x7f02681090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 --2- 172.21.15.76:0/967550422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f02681090d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f026000b0a0 tx=0x7f026000b3b0 comp rx=0 tx=0).stop 2024-02-14T14:59:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 -- 172.21.15.76:0/967550422 shutdown_connections 2024-02-14T14:59:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 --2- 172.21.15.76:0/967550422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f02681090d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 --2- 172.21.15.76:0/967550422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0268100c80 0x7f02681010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 --2- 172.21.15.76:0/967550422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0268101e80 0x7f02681022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 -- 172.21.15.76:0/967550422 >> 172.21.15.76:0/967550422 conn(0x7f02680fc240 msgr2=0x7f02680fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 -- 172.21.15.76:0/967550422 shutdown_connections 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 -- 172.21.15.76:0/967550422 wait complete. 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 Processor -- start 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.435+0000 7f026d5d1700 1 -- start start 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f026d5d1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0268100c80 0x7f026810b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f026d5d1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0268101e80 0x7f0268109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f026d5d1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f026810a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f026d5d1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026810a8a0 con 0x7f0268101e80 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f026d5d1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026810aa10 con 0x7f0268100c80 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f026d5d1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02681a1da0 con 0x7f0268102820 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f026810a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f026810a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35634/0 (socket says 172.21.15.76:35634) 2024-02-14T14:59:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 -- 172.21.15.76:0/4271101023 learned_addr learned my addr 172.21.15.76:0/4271101023 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02667fc700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0268101e80 0x7f0268109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f0266ffd700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0268100c80 0x7f026810b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 -- 172.21.15.76:0/4271101023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0268100c80 msgr2=0x7f026810b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0268100c80 0x7f026810b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 -- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0268101e80 msgr2=0x7f0268109be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0268101e80 0x7f0268109be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 -- 172.21.15.76:0/4271101023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260009d00 con 0x7f0268102820 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f02677fe700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f026810a120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0260009850 tx=0x7f02600094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f024ffff700 1 -- 172.21.15.76:0/4271101023 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026000e040 con 0x7f0268102820 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.436+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02681a2020 con 0x7f0268102820 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.437+0000 7f024ffff700 1 -- 172.21.15.76:0/4271101023 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f026001d960 con 0x7f0268102820 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.437+0000 7f024ffff700 1 -- 172.21.15.76:0/4271101023 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026001b9d0 con 0x7f0268102820 2024-02-14T14:59:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.437+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02681a2440 con 0x7f0268102820 2024-02-14T14:59:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.437+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0268107f70 con 0x7f0268102820 2024-02-14T14:59:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.438+0000 7f024ffff700 1 -- 172.21.15.76:0/4271101023 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f026001dad0 con 0x7f0268102820 2024-02-14T14:59:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.439+0000 7f024ffff700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02500a2150 0x7f02500a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.439+0000 7f024ffff700 1 -- 172.21.15.76:0/4271101023 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f02600cf390 con 0x7f0268102820 2024-02-14T14:59:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.440+0000 7f0266ffd700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02500a2150 0x7f02500a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.440+0000 7f0266ffd700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02500a2150 0x7f02500a4610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0268101ce0 tx=0x7f025800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:59:35.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.444+0000 7f024ffff700 1 -- 172.21.15.76:0/4271101023 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0260098a20 con 0x7f0268102820 2024-02-14T14:59:35.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:35.646+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f026802d7e0 con 0x7f02500a2150 2024-02-14T14:59:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:35 smithi050 ceph-mon[36512]: osdmap e74: 11 total, 11 up, 11 in 2024-02-14T14:59:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:35 smithi050 ceph-mon[36512]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:35.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:35 smithi050 ceph-mon[36512]: osdmap e75: 11 total, 11 up, 11 in 2024-02-14T14:59:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:35 smithi076 ceph-mon[37057]: osdmap e74: 11 total, 11 up, 11 in 2024-02-14T14:59:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:35 smithi076 ceph-mon[37057]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:35 smithi076 ceph-mon[37057]: osdmap e75: 11 total, 11 up, 11 in 2024-02-14T14:59:36.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:35 smithi017 ceph-mon[32549]: osdmap e74: 11 total, 11 up, 11 in 2024-02-14T14:59:36.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:35 smithi017 ceph-mon[32549]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:36.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:35 smithi017 ceph-mon[32549]: osdmap e75: 11 total, 11 up, 11 in 2024-02-14T14:59:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:36 smithi050 ceph-mon[36512]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:59:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:59:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:59:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:36 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:36.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:36 smithi050 ceph-mon[36512]: osdmap e76: 11 total, 11 up, 11 in 2024-02-14T14:59:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:36 smithi076 ceph-mon[37057]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:59:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:36 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:59:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:36 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:59:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:36 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:36 smithi076 ceph-mon[37057]: osdmap e76: 11 total, 11 up, 11 in 2024-02-14T14:59:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:36 smithi017 ceph-mon[32549]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T14:59:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:36 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T14:59:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:36 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T14:59:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:36 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:36 smithi017 ceph-mon[32549]: osdmap e76: 11 total, 11 up, 11 in 2024-02-14T14:59:37.839 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:37 smithi076 ceph-mon[37057]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-14T14:59:37.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:37 smithi050 ceph-mon[36512]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-14T14:59:38.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:37 smithi017 ceph-mon[32549]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-14T14:59:38.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:38 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/3434429129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]: dispatch 2024-02-14T14:59:38.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:38 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]: dispatch 2024-02-14T14:59:38.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:38 smithi050 ceph-mon[36512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]': finished 2024-02-14T14:59:38.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:38 smithi050 ceph-mon[36512]: osdmap e77: 12 total, 11 up, 12 in 2024-02-14T14:59:38.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:38.917 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:38 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/3434429129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]: dispatch 2024-02-14T14:59:38.917 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:38 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]: dispatch 2024-02-14T14:59:38.917 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:38 smithi076 ceph-mon[37057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]': finished 2024-02-14T14:59:38.917 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:38 smithi076 ceph-mon[37057]: osdmap e77: 12 total, 11 up, 12 in 2024-02-14T14:59:38.917 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:38 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/3434429129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]: dispatch 2024-02-14T14:59:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:38 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]: dispatch 2024-02-14T14:59:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:38 smithi017 ceph-mon[32549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982cfd49-b3f7-4e98-a92a-7927c0aee762"}]': finished 2024-02-14T14:59:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:38 smithi017 ceph-mon[32549]: osdmap e77: 12 total, 11 up, 12 in 2024-02-14T14:59:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:39 smithi050 ceph-mon[36512]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-02-14T14:59:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:39 smithi050 ceph-mon[36512]: from='client.? 172.21.15.76:0/961506865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:59:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:39 smithi076 ceph-mon[37057]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-02-14T14:59:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:39 smithi076 ceph-mon[37057]: from='client.? 172.21.15.76:0/961506865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:59:40.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:39 smithi017 ceph-mon[32549]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-02-14T14:59:40.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:39 smithi017 ceph-mon[32549]: from='client.? 172.21.15.76:0/961506865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T14:59:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:42 smithi050 ceph-mon[36512]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:59:42.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:42 smithi076 ceph-mon[37057]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:59:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:42 smithi017 ceph-mon[32549]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T14:59:44.820 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:44 smithi076 ceph-mon[37057]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-14T14:59:44.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:44 smithi050 ceph-mon[36512]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-14T14:59:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:44 smithi017 ceph-mon[32549]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-14T14:59:46.851 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:46 smithi076 ceph-mon[37057]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:46.852 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-14T14:59:46.852 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:46 smithi050 ceph-mon[36512]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-14T14:59:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:46 smithi017 ceph-mon[32549]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-14T14:59:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:47.826 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:47 smithi076 ceph-mon[37057]: Deploying daemon osd.11 on smithi076 2024-02-14T14:59:47.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:47 smithi050 ceph-mon[36512]: Deploying daemon osd.11 on smithi076 2024-02-14T14:59:48.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:47 smithi017 ceph-mon[32549]: Deploying daemon osd.11 on smithi076 2024-02-14T14:59:48.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:48 smithi076 ceph-mon[37057]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:48.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:48 smithi050 ceph-mon[36512]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:49.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:48 smithi017 ceph-mon[32549]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:50 smithi076 ceph-mon[37057]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:50.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:50 smithi050 ceph-mon[36512]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:50 smithi017 ceph-mon[32549]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:52.882 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:52 smithi076 ceph-mon[37057]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:52.883 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:52.883 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:52.883 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:52 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:52 smithi050 ceph-mon[36512]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:52 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:52 smithi017 ceph-mon[32549]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:52 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:54.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:54 smithi050 ceph-mon[36512]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:54.936 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:54 smithi076 ceph-mon[37057]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:55.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:54 smithi017 ceph-mon[32549]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:56.369 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 11 on host 'smithi076' 2024-02-14T14:59:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.368+0000 7f024ffff700 1 -- 172.21.15.76:0/4271101023 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f026802d7e0 con 0x7f02500a2150 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02500a2150 msgr2=0x7f02500a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02500a2150 0x7f02500a4610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0268101ce0 tx=0x7f025800b410 comp rx=0 tx=0).stop 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 msgr2=0x7f026810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f026810a120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0260009850 tx=0x7f02600094c0 comp rx=0 tx=0).stop 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 shutdown_connections 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02500a2150 0x7f02500a4610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0268102820 0x7f026810a120 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0268100c80 0x7f026810b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 --2- 172.21.15.76:0/4271101023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0268101e80 0x7f0268109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 >> 172.21.15.76:0/4271101023 conn(0x7f02680fc240 msgr2=0x7f02681050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:59:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 shutdown_connections 2024-02-14T14:59:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T14:59:56.371+0000 7f026d5d1700 1 -- 172.21.15.76:0/4271101023 wait complete. 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:56 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:56 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T14:59:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T14:59:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-14T14:59:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-14T14:59:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T14:59:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:56 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T14:59:57.086 DEBUG:teuthology.orchestra.run.smithi076:osd.11> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.11.service 2024-02-14T14:59:57.088 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-14T14:59:57.088 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-14T14:59:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T14:59:57.718 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:57 smithi017 ceph-mon[32549]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-14T14:59:57.718 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:57 smithi017 ceph-mon[32549]: osdmap e78: 12 total, 11 up, 12 in 2024-02-14T14:59:57.718 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:57 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:57.718 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:57 smithi017 ceph-mon[32549]: from='osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:59:57.718 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:57 smithi017 ceph-mon[32549]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:59:57.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:57 smithi050 ceph-mon[36512]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-14T14:59:57.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:57 smithi050 ceph-mon[36512]: osdmap e78: 12 total, 11 up, 12 in 2024-02-14T14:59:57.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:57 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:57.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:57 smithi050 ceph-mon[36512]: from='osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:59:57.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:57 smithi050 ceph-mon[36512]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:59:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:57 smithi076 ceph-mon[37057]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-14T14:59:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:57 smithi076 ceph-mon[37057]: osdmap e78: 12 total, 11 up, 12 in 2024-02-14T14:59:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:57 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:57 smithi076 ceph-mon[37057]: from='osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:59:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:57 smithi076 ceph-mon[37057]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-02-14T14:59:57.984 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 14:59:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T14:59:57.564+0000 7f25ef422700 -1 osd.11 0 waiting for initial osdmap 2024-02-14T14:59:57.984 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 14:59:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T14:59:57.576+0000 7f25eb21c700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T14:59:58.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:58 smithi017 ceph-mon[32549]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:58.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:58 smithi017 ceph-mon[32549]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:59:58.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:58 smithi017 ceph-mon[32549]: osdmap e79: 12 total, 11 up, 12 in 2024-02-14T14:59:58.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:58.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:58 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:58.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:58 smithi076 ceph-mon[37057]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:58.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:58 smithi076 ceph-mon[37057]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:59:58.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:58 smithi076 ceph-mon[37057]: osdmap e79: 12 total, 11 up, 12 in 2024-02-14T14:59:58.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:58.858 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:58 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:58 smithi050 ceph-mon[36512]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T14:59:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:58 smithi050 ceph-mon[36512]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-02-14T14:59:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:58 smithi050 ceph-mon[36512]: osdmap e79: 12 total, 11 up, 12 in 2024-02-14T14:59:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:58 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:59.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.154+0000 7f27db500700 1 -- 172.21.15.17:0/448652838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4101d80 msgr2=0x7f27d41021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.154+0000 7f27db500700 1 --2- 172.21.15.17:0/448652838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4101d80 0x7f27d41021a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f27d00099c0 tx=0x7f27d0009cd0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- 172.21.15.17:0/448652838 shutdown_connections 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 --2- 172.21.15.17:0/448652838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27d4103920 0x7f27d410a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 --2- 172.21.15.17:0/448652838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4101d80 0x7f27d41021a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 --2- 172.21.15.17:0/448652838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27d4102f80 0x7f27d41033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- 172.21.15.17:0/448652838 >> 172.21.15.17:0/448652838 conn(0x7f27d40fd320 msgr2=0x7f27d40ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- 172.21.15.17:0/448652838 shutdown_connections 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- 172.21.15.17:0/448652838 wait complete. 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 Processor -- start 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- start start 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27d4101d80 0x7f27d4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4102f80 0x7f27d4074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27d4103920 0x7f27d4074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d4074dd0 con 0x7f27d4102f80 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d4074f40 con 0x7f27d4103920 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.155+0000 7f27db500700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d41a1d90 con 0x7f27d4101d80 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4102f80 0x7f27d4074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4102f80 0x7f27d4074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50518/0 (socket says 172.21.15.17:50518) 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 -- 172.21.15.17:0/1930689816 learned_addr learned my addr 172.21.15.17:0/1930689816 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d929c700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27d4101d80 0x7f27d4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d9a9d700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27d4103920 0x7f27d4074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 -- 172.21.15.17:0/1930689816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27d4101d80 msgr2=0x7f27d4075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27d4101d80 0x7f27d4075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 -- 172.21.15.17:0/1930689816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27d4103920 msgr2=0x7f27d4074650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27d4103920 0x7f27d4074650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 -- 172.21.15.17:0/1930689816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00095f0 con 0x7f27d4102f80 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d9a9d700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27d4103920 0x7f27d4074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T14:59:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27d8a9b700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4102f80 0x7f27d4074110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27c400d950 tx=0x7f27c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:59:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27ca7fc700 1 -- 172.21.15.17:0/1930689816 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4009940 con 0x7f27d4102f80 2024-02-14T14:59:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d41a2070 con 0x7f27d4102f80 2024-02-14T14:59:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27ca7fc700 1 -- 172.21.15.17:0/1930689816 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27c4010460 con 0x7f27d4102f80 2024-02-14T14:59:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27ca7fc700 1 -- 172.21.15.17:0/1930689816 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c400f620 con 0x7f27d4102f80 2024-02-14T14:59:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.156+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d41a25c0 con 0x7f27d4102f80 2024-02-14T14:59:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.158+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d4109140 con 0x7f27d4102f80 2024-02-14T14:59:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.158+0000 7f27ca7fc700 1 -- 172.21.15.17:0/1930689816 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f27c40105d0 con 0x7f27d4102f80 2024-02-14T14:59:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.159+0000 7f27ca7fc700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f27c00a2150 0x7f27c00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T14:59:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.159+0000 7f27d929c700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f27c00a2150 0x7f27c00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T14:59:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.159+0000 7f27ca7fc700 1 -- 172.21.15.17:0/1930689816 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27c40cd0e0 con 0x7f27d4102f80 2024-02-14T14:59:59.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.160+0000 7f27d929c700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f27c00a2150 0x7f27c00a4610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27d00038c0 tx=0x7f27d000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T14:59:59.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.164+0000 7f27ca7fc700 1 -- 172.21.15.17:0/1930689816 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27c409a080 con 0x7f27d4102f80 2024-02-14T14:59:59.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.369+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f27d410d740 con 0x7f27d4102f80 2024-02-14T14:59:59.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.370+0000 7f27ca7fc700 1 -- 172.21.15.17:0/1930689816 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f27c4099c10 con 0x7f27d4102f80 2024-02-14T14:59:59.370 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T14:59:59.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f27c00a2150 msgr2=0x7f27c00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f27c00a2150 0x7f27c00a4610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27d00038c0 tx=0x7f27d000b540 comp rx=0 tx=0).stop 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4102f80 msgr2=0x7f27d4074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4102f80 0x7f27d4074110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27c400d950 tx=0x7f27c400dc60 comp rx=0 tx=0).stop 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 shutdown_connections 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f27c00a2150 0x7f27c00a4610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27d4103920 0x7f27d4074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27d4101d80 0x7f27d4075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 --2- 172.21.15.17:0/1930689816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27d4102f80 0x7f27d4074110 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 >> 172.21.15.17:0/1930689816 conn(0x7f27d40fd320 msgr2=0x7f27d4106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 shutdown_connections 2024-02-14T14:59:59.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T14:59:59.373+0000 7f27db500700 1 -- 172.21.15.17:0/1930689816 wait complete. 2024-02-14T14:59:59.688 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:59 smithi017 ceph-mon[32549]: purged_snaps scrub starts 2024-02-14T14:59:59.688 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:59 smithi017 ceph-mon[32549]: purged_snaps scrub ok 2024-02-14T14:59:59.688 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:59 smithi017 ceph-mon[32549]: osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] boot 2024-02-14T14:59:59.688 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:59 smithi017 ceph-mon[32549]: osdmap e80: 12 total, 12 up, 12 in 2024-02-14T14:59:59.688 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:59 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:59.688 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 14:59:59 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1930689816' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T14:59:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:59 smithi076 ceph-mon[37057]: purged_snaps scrub starts 2024-02-14T14:59:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:59 smithi076 ceph-mon[37057]: purged_snaps scrub ok 2024-02-14T14:59:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:59 smithi076 ceph-mon[37057]: osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] boot 2024-02-14T14:59:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:59 smithi076 ceph-mon[37057]: osdmap e80: 12 total, 12 up, 12 in 2024-02-14T14:59:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:59 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:59.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 14:59:59 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1930689816' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T14:59:59.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:59 smithi050 ceph-mon[36512]: purged_snaps scrub starts 2024-02-14T14:59:59.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:59 smithi050 ceph-mon[36512]: purged_snaps scrub ok 2024-02-14T14:59:59.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:59 smithi050 ceph-mon[36512]: osd.11 [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] boot 2024-02-14T14:59:59.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:59 smithi050 ceph-mon[36512]: osdmap e80: 12 total, 12 up, 12 in 2024-02-14T14:59:59.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:59 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T14:59:59.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 14:59:59 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1930689816' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T15:00:00.034 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1707922798,"num_in_osds":12,"osd_in_since":1707922778,"num_remapped_pgs":0} 2024-02-14T15:00:00.035 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T15:00:00.282 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:00:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T15:00:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: osdmap e81: 12 total, 12 up, 12 in 2024-02-14T15:00:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: overall HEALTH_OK 2024-02-14T15:00:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:00.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:00 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: osdmap e81: 12 total, 12 up, 12 in 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: overall HEALTH_OK 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:00.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:00.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:00.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:00.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:00 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-02-14T15:00:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: osdmap e81: 12 total, 12 up, 12 in 2024-02-14T15:00:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: overall HEALTH_OK 2024-02-14T15:00:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:01.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:00 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:01.739 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:01 smithi017 ceph-mon[32549]: Detected new or changed devices on smithi076 2024-02-14T15:00:01.739 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:01 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi076 to 4270M 2024-02-14T15:00:01.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:01 smithi050 ceph-mon[36512]: Detected new or changed devices on smithi076 2024-02-14T15:00:01.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:01 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi076 to 4270M 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 -- 172.21.15.17:0/1983869090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948102730 msgr2=0x7f994810eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 --2- 172.21.15.17:0/1983869090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948102730 0x7f994810eb70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f993800b0d0 tx=0x7f993800b3e0 comp rx=0 tx=0).stop 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 -- 172.21.15.17:0/1983869090 shutdown_connections 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 --2- 172.21.15.17:0/1983869090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948102730 0x7f994810eb70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 --2- 172.21.15.17:0/1983869090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9948107d50 0x7f9948108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 --2- 172.21.15.17:0/1983869090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9948101d70 0x7f99481021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 -- 172.21.15.17:0/1983869090 >> 172.21.15.17:0/1983869090 conn(0x7f9948078e50 msgr2=0x7f9948079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 -- 172.21.15.17:0/1983869090 shutdown_connections 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.947+0000 7f994fe8f700 1 -- 172.21.15.17:0/1983869090 wait complete. 2024-02-14T15:00:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 Processor -- start 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 -- start start 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948101d70 0x7f9948198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9948102730 0x7f9948198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9948107d50 0x7f994819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819d2d0 con 0x7f9948101d70 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819d440 con 0x7f9948107d50 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948101d70 0x7f9948198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994fe8f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819d600 con 0x7f9948102730 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948101d70 0x7f9948198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44616/0 (socket says 172.21.15.17:44616) 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 -- 172.21.15.17:0/3024914200 learned_addr learned my addr 172.21.15.17:0/3024914200 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994e42c700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9948107d50 0x7f994819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994d42a700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9948102730 0x7f9948198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:01.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 -- 172.21.15.17:0/3024914200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9948102730 msgr2=0x7f9948198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9948102730 0x7f9948198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 -- 172.21.15.17:0/3024914200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9948107d50 msgr2=0x7f994819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9948107d50 0x7f994819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994dc2b700 1 -- 172.21.15.17:0/3024914200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938009d00 con 0x7f9948101d70 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.948+0000 7f994d42a700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9948102730 0x7f9948198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.949+0000 7f994dc2b700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948101d70 0x7f9948198440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f994000b530 tx=0x7f994000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.949+0000 7f9946ffd700 1 -- 172.21.15.17:0/3024914200 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940011830 con 0x7f9948101d70 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.949+0000 7f9946ffd700 1 -- 172.21.15.17:0/3024914200 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9940011e70 con 0x7f9948101d70 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.949+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994819d8e0 con 0x7f9948101d70 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.949+0000 7f9946ffd700 1 -- 172.21.15.17:0/3024914200 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994000f2e0 con 0x7f9948101d70 2024-02-14T15:00:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.949+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99481a5140 con 0x7f9948101d70 2024-02-14T15:00:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.950+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994810a230 con 0x7f9948101d70 2024-02-14T15:00:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.951+0000 7f9946ffd700 1 -- 172.21.15.17:0/3024914200 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f99400105f0 con 0x7f9948101d70 2024-02-14T15:00:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.951+0000 7f9946ffd700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f99300a2150 0x7f99300a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.951+0000 7f994d42a700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f99300a2150 0x7f99300a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:01.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.951+0000 7f9946ffd700 1 -- 172.21.15.17:0/3024914200 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f99400cd7f0 con 0x7f9948101d70 2024-02-14T15:00:01.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.952+0000 7f994d42a700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f99300a2150 0x7f99300a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9948079530 tx=0x7f993c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:01.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:01.955+0000 7f9946ffd700 1 -- 172.21.15.17:0/3024914200 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9940096bc0 con 0x7f9948101d70 2024-02-14T15:00:01.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:01 smithi076 ceph-mon[37057]: Detected new or changed devices on smithi076 2024-02-14T15:00:01.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:01 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi076 to 4270M 2024-02-14T15:00:02.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.167+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9948111e50 con 0x7f9948101d70 2024-02-14T15:00:02.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.168+0000 7f9946ffd700 1 -- 172.21.15.17:0/3024914200 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f9940010380 con 0x7f9948101d70 2024-02-14T15:00:02.169 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:02.170 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":81,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T14:59:59.565307+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:00:02.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f99300a2150 msgr2=0x7f99300a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:02.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f99300a2150 0x7f99300a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9948079530 tx=0x7f993c008040 comp rx=0 tx=0).stop 2024-02-14T15:00:02.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948101d70 msgr2=0x7f9948198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:02.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948101d70 0x7f9948198440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f994000b530 tx=0x7f994000b8f0 comp rx=0 tx=0).stop 2024-02-14T15:00:02.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 shutdown_connections 2024-02-14T15:00:02.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f99300a2150 0x7f99300a4610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:02.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9948107d50 0x7f994819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:02.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9948101d70 0x7f9948198440 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:02.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 --2- 172.21.15.17:0/3024914200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9948102730 0x7f9948198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:02.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 >> 172.21.15.17:0/3024914200 conn(0x7f9948078e50 msgr2=0x7f9948100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:02.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 shutdown_connections 2024-02-14T15:00:02.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:02.170+0000 7f994fe8f700 1 -- 172.21.15.17:0/3024914200 wait complete. 2024-02-14T15:00:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:02 smithi017 ceph-mon[32549]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:02.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3024914200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:02.847 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-14T15:00:02.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-14T15:00:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:02 smithi050 ceph-mon[36512]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3024914200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:02 smithi076 ceph-mon[37057]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3024914200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:03.093 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:00:03.841 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:03 smithi076 ceph-mon[37057]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:03.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:03 smithi050 ceph-mon[36512]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:04.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:03 smithi017 ceph-mon[32549]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:04.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 -- 172.21.15.17:0/3324446809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c105f00 msgr2=0x7f378c10dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 --2- 172.21.15.17:0/3324446809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c105f00 0x7f378c10dad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f378400b0a0 tx=0x7f378400b3b0 comp rx=0 tx=0).stop 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 -- 172.21.15.17:0/3324446809 shutdown_connections 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 --2- 172.21.15.17:0/3324446809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c105f00 0x7f378c10dad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 --2- 172.21.15.17:0/3324446809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f378c100eb0 0x7f378c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 --2- 172.21.15.17:0/3324446809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f378c101860 0x7f378c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 -- 172.21.15.17:0/3324446809 >> 172.21.15.17:0/3324446809 conn(0x7f378c076c70 msgr2=0x7f378c077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 -- 172.21.15.17:0/3324446809 shutdown_connections 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.917+0000 7f3790ba2700 1 -- 172.21.15.17:0/3324446809 wait complete. 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 Processor -- start 2024-02-14T15:00:04.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 -- start start 2024-02-14T15:00:04.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f378c100eb0 0x7f378c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:04.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c101860 0x7f378c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:04.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f378c105f00 0x7f378c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c19d240 con 0x7f378c101860 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c19d3b0 con 0x7f378c100eb0 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3790ba2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c19d570 con 0x7f378c105f00 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3789d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c101860 0x7f378c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3789d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c101860 0x7f378c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44632/0 (socket says 172.21.15.17:44632) 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f3789d9b700 1 -- 172.21.15.17:0/311406473 learned_addr learned my addr 172.21.15.17:0/311406473 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f378a59c700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f378c100eb0 0x7f378c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.918+0000 7f378ad9d700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f378c105f00 0x7f378c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3789d9b700 1 -- 172.21.15.17:0/311406473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f378c105f00 msgr2=0x7f378c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3789d9b700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f378c105f00 0x7f378c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3789d9b700 1 -- 172.21.15.17:0/311406473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f378c100eb0 msgr2=0x7f378c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3789d9b700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f378c100eb0 0x7f378c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3789d9b700 1 -- 172.21.15.17:0/311406473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009d00 con 0x7f378c101860 2024-02-14T15:00:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f378a59c700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f378c100eb0 0x7f378c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:00:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3789d9b700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c101860 0x7f378c1988f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f378000d980 tx=0x7f378000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f377b7fe700 1 -- 172.21.15.17:0/311406473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378000f890 con 0x7f378c101860 2024-02-14T15:00:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f377b7fe700 1 -- 172.21.15.17:0/311406473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3780010460 con 0x7f378c101860 2024-02-14T15:00:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c19d850 con 0x7f378c101860 2024-02-14T15:00:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f377b7fe700 1 -- 172.21.15.17:0/311406473 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378000e610 con 0x7f378c101860 2024-02-14T15:00:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.919+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c1029c0 con 0x7f378c101860 2024-02-14T15:00:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.920+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c10a230 con 0x7f378c101860 2024-02-14T15:00:04.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.922+0000 7f377b7fe700 1 -- 172.21.15.17:0/311406473 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3780010940 con 0x7f378c101860 2024-02-14T15:00:04.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.922+0000 7f377b7fe700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37740a2150 0x7f37740a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:04.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.923+0000 7f378a59c700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37740a2150 0x7f37740a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:04.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.923+0000 7f377b7fe700 1 -- 172.21.15.17:0/311406473 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f37800cd300 con 0x7f378c101860 2024-02-14T15:00:04.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.923+0000 7f378a59c700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37740a2150 0x7f37740a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f377c009de0 tx=0x7f377c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:04.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:04.926+0000 7f377b7fe700 1 -- 172.21.15.17:0/311406473 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37800966d0 con 0x7f378c101860 2024-02-14T15:00:05.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.125+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f378c1996c0 con 0x7f378c101860 2024-02-14T15:00:05.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.126+0000 7f377b7fe700 1 -- 172.21.15.17:0/311406473 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37800105d0 con 0x7f378c101860 2024-02-14T15:00:05.126 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37740a2150 msgr2=0x7f37740a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37740a2150 0x7f37740a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f377c009de0 tx=0x7f377c009190 comp rx=0 tx=0).stop 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c101860 msgr2=0x7f378c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c101860 0x7f378c1988f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f378000d980 tx=0x7f378000dc90 comp rx=0 tx=0).stop 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 shutdown_connections 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37740a2150 0x7f37740a4610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f378c105f00 0x7f378c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f378c100eb0 0x7f378c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 --2- 172.21.15.17:0/311406473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c101860 0x7f378c1988f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:05.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 >> 172.21.15.17:0/311406473 conn(0x7f378c076c70 msgr2=0x7f378c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:05.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 shutdown_connections 2024-02-14T15:00:05.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:05.128+0000 7f3790ba2700 1 -- 172.21.15.17:0/311406473 wait complete. 2024-02-14T15:00:05.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:05 smithi017 ceph-mon[32549]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:05.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:05 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/311406473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.a on smithi017 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.d on smithi017 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.g on smithi017 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.j on smithi017 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.b on smithi050 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.e on smithi050 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.h on smithi050 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.k on smithi050 2024-02-14T15:00:05.911 INFO:tasks.cephadm:Adding mds.c on smithi076 2024-02-14T15:00:05.912 INFO:tasks.cephadm:Adding mds.f on smithi076 2024-02-14T15:00:05.912 INFO:tasks.cephadm:Adding mds.i on smithi076 2024-02-14T15:00:05.912 INFO:tasks.cephadm:Adding mds.l on smithi076 2024-02-14T15:00:05.912 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph orch apply mds all '12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l' 2024-02-14T15:00:05.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:05 smithi076 ceph-mon[37057]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:05 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/311406473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:05 smithi050 ceph-mon[36512]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:05 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/311406473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:06.161 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T15:00:07.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 -- 172.21.15.76:0/2409824242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98100010 msgr2=0x7fcd98100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:07.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 --2- 172.21.15.76:0/2409824242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98100010 0x7fcd98100430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009990 tx=0x7fcd94009ca0 comp rx=0 tx=0).stop 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 -- 172.21.15.76:0/2409824242 shutdown_connections 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 --2- 172.21.15.76:0/2409824242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd98103990 0x7fcd9810a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 --2- 172.21.15.76:0/2409824242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98100010 0x7fcd98100430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 --2- 172.21.15.76:0/2409824242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd98102ff0 0x7fcd98103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 -- 172.21.15.76:0/2409824242 >> 172.21.15.76:0/2409824242 conn(0x7fcd980fb990 msgr2=0x7fcd980fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 -- 172.21.15.76:0/2409824242 shutdown_connections 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.883+0000 7fcd9f712700 1 -- 172.21.15.76:0/2409824242 wait complete. 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 Processor -- start 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 -- start start 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd98100010 0x7fcd98102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98102ff0 0x7fcd98100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd98103990 0x7fcd981014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd98101c60 con 0x7fcd98100010 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd98101dd0 con 0x7fcd98103990 2024-02-14T15:00:07.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9f712700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd981a1d70 con 0x7fcd98102ff0 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98102ff0 0x7fcd98100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98102ff0 0x7fcd98100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46162/0 (socket says 172.21.15.76:46162) 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 -- 172.21.15.76:0/3838391736 learned_addr learned my addr 172.21.15.76:0/3838391736 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9dcaf700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd98103990 0x7fcd981014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9d4ae700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd98100010 0x7fcd98102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 -- 172.21.15.76:0/3838391736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd98103990 msgr2=0x7fcd981014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd98103990 0x7fcd981014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 -- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd98100010 msgr2=0x7fcd98102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd98100010 0x7fcd98102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 -- 172.21.15.76:0/3838391736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940095f0 con 0x7fcd98102ff0 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9d4ae700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd98100010 0x7fcd98102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.884+0000 7fcd9ccad700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98102ff0 0x7fcd98100fa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d950 tx=0x7fcd8800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.885+0000 7fcd8e7fc700 1 -- 172.21.15.76:0/3838391736 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88009940 con 0x7fcd98102ff0 2024-02-14T15:00:07.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.885+0000 7fcd8e7fc700 1 -- 172.21.15.76:0/3838391736 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd88010460 con 0x7fcd98102ff0 2024-02-14T15:00:07.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.885+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd981a1ff0 con 0x7fcd98102ff0 2024-02-14T15:00:07.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.885+0000 7fcd8e7fc700 1 -- 172.21.15.76:0/3838391736 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800f620 con 0x7fcd98102ff0 2024-02-14T15:00:07.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.885+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd981a24b0 con 0x7fcd98102ff0 2024-02-14T15:00:07.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.886+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd980689e0 con 0x7fcd98102ff0 2024-02-14T15:00:07.888 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:07 smithi076 ceph-mon[37057]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:07.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.887+0000 7fcd8e7fc700 1 -- 172.21.15.76:0/3838391736 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcd880105d0 con 0x7fcd98102ff0 2024-02-14T15:00:07.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.887+0000 7fcd8e7fc700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd840a2150 0x7fcd840a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:07.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.887+0000 7fcd8e7fc700 1 -- 172.21.15.76:0/3838391736 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcd880cd6a0 con 0x7fcd98102ff0 2024-02-14T15:00:07.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.888+0000 7fcd9d4ae700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd840a2150 0x7fcd840a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:07.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.888+0000 7fcd9d4ae700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd840a2150 0x7fcd840a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd940038c0 tx=0x7fcd94005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:07.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:07.892+0000 7fcd8e7fc700 1 -- 172.21.15.76:0/3838391736 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd8809a640 con 0x7fcd98102ff0 2024-02-14T15:00:08.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:07 smithi017 ceph-mon[32549]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.095+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd9802da20 con 0x7fcd840a2150 2024-02-14T15:00:08.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.103+0000 7fcd8e7fc700 1 -- 172.21.15.76:0/3838391736 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fcd9802da20 con 0x7fcd840a2150 2024-02-14T15:00:08.103 INFO:teuthology.orchestra.run.smithi076.stdout:Scheduled mds.all update... 2024-02-14T15:00:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd840a2150 msgr2=0x7fcd840a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd840a2150 0x7fcd840a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd940038c0 tx=0x7fcd94005fb0 comp rx=0 tx=0).stop 2024-02-14T15:00:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98102ff0 msgr2=0x7fcd98100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98102ff0 0x7fcd98100fa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d950 tx=0x7fcd8800dc60 comp rx=0 tx=0).stop 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 shutdown_connections 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd840a2150 0x7fcd840a4610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd98103990 0x7fcd981014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd98100010 0x7fcd98102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 --2- 172.21.15.76:0/3838391736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd98102ff0 0x7fcd98100fa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 >> 172.21.15.76:0/3838391736 conn(0x7fcd980fb990 msgr2=0x7fcd98106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 shutdown_connections 2024-02-14T15:00:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-02-14T15:00:08.105+0000 7fcd9f712700 1 -- 172.21.15.76:0/3838391736 wait complete. 2024-02-14T15:00:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:07 smithi050 ceph-mon[36512]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:08.835 DEBUG:teuthology.orchestra.run.smithi017:mds.a> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.a.service 2024-02-14T15:00:08.837 DEBUG:teuthology.orchestra.run.smithi017:mds.d> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.d.service 2024-02-14T15:00:08.881 DEBUG:teuthology.orchestra.run.smithi017:mds.g> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.g.service 2024-02-14T15:00:08.923 DEBUG:teuthology.orchestra.run.smithi017:mds.j> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.j.service 2024-02-14T15:00:08.965 DEBUG:teuthology.orchestra.run.smithi050:mds.b> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.b.service 2024-02-14T15:00:08.968 DEBUG:teuthology.orchestra.run.smithi050:mds.e> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.e.service 2024-02-14T15:00:09.011 DEBUG:teuthology.orchestra.run.smithi050:mds.h> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service 2024-02-14T15:00:09.054 DEBUG:teuthology.orchestra.run.smithi050:mds.k> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.k.service 2024-02-14T15:00:09.097 DEBUG:teuthology.orchestra.run.smithi076:mds.c> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.c.service 2024-02-14T15:00:09.100 DEBUG:teuthology.orchestra.run.smithi076:mds.f> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.f.service 2024-02-14T15:00:09.142 DEBUG:teuthology.orchestra.run.smithi076:mds.i> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.i.service 2024-02-14T15:00:09.185 DEBUG:teuthology.orchestra.run.smithi076:mds.l> sudo journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l.service 2024-02-14T15:00:09.227 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-14T15:00:09.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l;count:12 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:09.253 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:09 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:09.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 -- 172.21.15.17:0/3517143788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f01012c0 msgr2=0x7f87f01016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 --2- 172.21.15.17:0/3517143788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f01012c0 0x7f87f01016e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009900 tx=0x7f87e002bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:09.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 -- 172.21.15.17:0/3517143788 shutdown_connections 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 --2- 172.21.15.17:0/3517143788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f0102e70 0x7f87f0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 --2- 172.21.15.17:0/3517143788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87f01024b0 0x7f87f0102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 --2- 172.21.15.17:0/3517143788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f01012c0 0x7f87f01016e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 -- 172.21.15.17:0/3517143788 >> 172.21.15.17:0/3517143788 conn(0x7f87f00fc850 msgr2=0x7f87f00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 -- 172.21.15.17:0/3517143788 shutdown_connections 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.338+0000 7f87f52df640 1 -- 172.21.15.17:0/3517143788 wait complete. 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 Processor -- start 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 -- start start 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87f01012c0 0x7f87f01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f01024b0 0x7f87f01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f0102e70 0x7f87f019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f019eda0 con 0x7f87f0102e70 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f019ef10 con 0x7f87f01012c0 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87f52df640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f019f0d0 con 0x7f87f01024b0 2024-02-14T15:00:09.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f0102e70 0x7f87f019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f0102e70 0x7f87f019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44662/0 (socket says 172.21.15.17:44662) 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87ef7fe640 1 -- 172.21.15.17:0/3397204065 learned_addr learned my addr 172.21.15.17:0/3397204065 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87eeffd640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87f01012c0 0x7f87f01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.339+0000 7f87e7fff640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f01024b0 0x7f87f01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ef7fe640 1 -- 172.21.15.17:0/3397204065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f01024b0 msgr2=0x7f87f01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ef7fe640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f01024b0 0x7f87f01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ef7fe640 1 -- 172.21.15.17:0/3397204065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87f01012c0 msgr2=0x7f87f01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ef7fe640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87f01012c0 0x7f87f01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ef7fe640 1 -- 172.21.15.17:0/3397204065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e0009560 con 0x7f87f0102e70 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87e7fff640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f01024b0 0x7f87f01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ef7fe640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f0102e70 0x7f87f019e860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87e800eb10 tx=0x7f87e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ecff9640 1 -- 172.21.15.17:0/3397204065 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8004b20 con 0x7f87f0102e70 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ecff9640 1 -- 172.21.15.17:0/3397204065 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87e8004c80 con 0x7f87f0102e70 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f019f3b0 con 0x7f87f0102e70 2024-02-14T15:00:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87ecff9640 1 -- 172.21.15.17:0/3397204065 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8010650 con 0x7f87f0102e70 2024-02-14T15:00:09.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.340+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f019f8c0 con 0x7f87f0102e70 2024-02-14T15:00:09.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.341+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f01085e0 con 0x7f87f0102e70 2024-02-14T15:00:09.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.342+0000 7f87ecff9640 1 -- 172.21.15.17:0/3397204065 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f87e8005020 con 0x7f87f0102e70 2024-02-14T15:00:09.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.342+0000 7f87ecff9640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f87dc0a20c0 0x7f87dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.343+0000 7f87eeffd640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f87dc0a20c0 0x7f87dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.343+0000 7f87ecff9640 1 -- 172.21.15.17:0/3397204065 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87e8014070 con 0x7f87f0102e70 2024-02-14T15:00:09.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.343+0000 7f87eeffd640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f87dc0a20c0 0x7f87dc0a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87e0005ab0 tx=0x7f87e0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.347+0000 7f87ecff9640 1 -- 172.21.15.17:0/3397204065 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f87e8097240 con 0x7f87f0102e70 2024-02-14T15:00:09.372 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l;count:12 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:09.373 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:09 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l;count:12 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:09.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:09 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:09.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.531+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87f002ce50 con 0x7f87f0102e70 2024-02-14T15:00:09.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.532+0000 7f87ecff9640 1 -- 172.21.15.17:0/3397204065 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f87e809a900 con 0x7f87f0102e70 2024-02-14T15:00:09.533 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:09.534 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":81,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T14:59:59.565307+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f87dc0a20c0 msgr2=0x7f87dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f87dc0a20c0 0x7f87dc0a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87e0005ab0 tx=0x7f87e0005fd0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f0102e70 msgr2=0x7f87f019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f0102e70 0x7f87f019e860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87e800eb10 tx=0x7f87e800c2d0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 shutdown_connections 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f87dc0a20c0 0x7f87dc0a4580 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87f0102e70 0x7f87f019e860 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f01024b0 0x7f87f01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 --2- 172.21.15.17:0/3397204065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87f01012c0 0x7f87f01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 >> 172.21.15.17:0/3397204065 conn(0x7f87f00fc850 msgr2=0x7f87f01057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 shutdown_connections 2024-02-14T15:00:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.535+0000 7f87f52df640 1 -- 172.21.15.17:0/3397204065 wait complete. 2024-02-14T15:00:09.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-14T15:00:09.547 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:00:09.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.692+0000 7fcc67ad4640 1 -- 172.21.15.17:0/821692232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc601090f0 msgr2=0x7fcc6010b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.692+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/821692232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc601090f0 0x7fcc6010b580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009ca0 tx=0x7fcc5802c180 comp rx=0 tx=0).stop 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.692+0000 7fcc67ad4640 1 -- 172.21.15.17:0/821692232 shutdown_connections 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.692+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/821692232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc601090f0 0x7fcc6010b580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.692+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/821692232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc60073c10 0x7fcc60074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.692+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/821692232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc600751f0 0x7fcc60073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.692+0000 7fcc67ad4640 1 -- 172.21.15.17:0/821692232 >> 172.21.15.17:0/821692232 conn(0x7fcc600fc7b0 msgr2=0x7fcc600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 -- 172.21.15.17:0/821692232 shutdown_connections 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 -- 172.21.15.17:0/821692232 wait complete. 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 Processor -- start 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 -- start start 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc60073c10 0x7fcc60199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc600751f0 0x7fcc601999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc601090f0 0x7fcc601a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a0f50 con 0x7fcc60073c10 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a10c0 con 0x7fcc601090f0 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc67ad4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601a1280 con 0x7fcc600751f0 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc65870640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc60073c10 0x7fcc60199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc65870640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc60073c10 0x7fcc60199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44686/0 (socket says 172.21.15.17:44686) 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc65870640 1 -- 172.21.15.17:0/2137165537 learned_addr learned my addr 172.21.15.17:0/2137165537 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc66071640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc601090f0 0x7fcc601a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.693+0000 7fcc6506f640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc600751f0 0x7fcc601999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc65870640 1 -- 172.21.15.17:0/2137165537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc600751f0 msgr2=0x7fcc601999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc65870640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc600751f0 0x7fcc601999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc65870640 1 -- 172.21.15.17:0/2137165537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc601090f0 msgr2=0x7fcc601a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc65870640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc601090f0 0x7fcc601a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc65870640 1 -- 172.21.15.17:0/2137165537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58009900 con 0x7fcc60073c10 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc66071640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc601090f0 0x7fcc601a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc65870640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc60073c10 0x7fcc60199460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00d950 tx=0x7fcc5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc56ffd640 1 -- 172.21.15.17:0/2137165537 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c009c30 con 0x7fcc60073c10 2024-02-14T15:00:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc56ffd640 1 -- 172.21.15.17:0/2137165537 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc5c009d90 con 0x7fcc60073c10 2024-02-14T15:00:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc601a1560 con 0x7fcc60073c10 2024-02-14T15:00:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc56ffd640 1 -- 172.21.15.17:0/2137165537 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c00f7e0 con 0x7fcc60073c10 2024-02-14T15:00:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.694+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc601a1a70 con 0x7fcc60073c10 2024-02-14T15:00:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.695+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc60107eb0 con 0x7fcc60073c10 2024-02-14T15:00:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.696+0000 7fcc56ffd640 1 -- 172.21.15.17:0/2137165537 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcc5c010ad0 con 0x7fcc60073c10 2024-02-14T15:00:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.697+0000 7fcc56ffd640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc4c0a20c0 0x7fcc4c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.697+0000 7fcc6506f640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc4c0a20c0 0x7fcc4c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.697+0000 7fcc56ffd640 1 -- 172.21.15.17:0/2137165537 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcc5c0cd520 con 0x7fcc60073c10 2024-02-14T15:00:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.697+0000 7fcc6506f640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc4c0a20c0 0x7fcc4c0a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005fd0 tx=0x7fcc50005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.701+0000 7fcc56ffd640 1 -- 172.21.15.17:0/2137165537 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc5c096990 con 0x7fcc60073c10 2024-02-14T15:00:09.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.881+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc6002ce50 con 0x7fcc60073c10 2024-02-14T15:00:09.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.881+0000 7fcc56ffd640 1 -- 172.21.15.17:0/2137165537 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc5c0b8e60 con 0x7fcc60073c10 2024-02-14T15:00:09.882 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:00:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc4c0a20c0 msgr2=0x7fcc4c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc4c0a20c0 0x7fcc4c0a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005fd0 tx=0x7fcc50005850 comp rx=0 tx=0).stop 2024-02-14T15:00:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc60073c10 msgr2=0x7fcc60199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc60073c10 0x7fcc60199460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00d950 tx=0x7fcc5c00de10 comp rx=0 tx=0).stop 2024-02-14T15:00:09.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 shutdown_connections 2024-02-14T15:00:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc4c0a20c0 0x7fcc4c0a4580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc601090f0 0x7fcc601a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc600751f0 0x7fcc601999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 --2- 172.21.15.17:0/2137165537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc60073c10 0x7fcc60199460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 >> 172.21.15.17:0/2137165537 conn(0x7fcc600fc7b0 msgr2=0x7fcc60105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 shutdown_connections 2024-02-14T15:00:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.883+0000 7fcc67ad4640 1 -- 172.21.15.17:0/2137165537 wait complete. 2024-02-14T15:00:09.895 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:09.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 -- 172.21.15.17:0/661302687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60101510 msgr2=0x7ffb60107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:09.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 --2- 172.21.15.17:0/661302687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60101510 0x7ffb60107da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009ca0 tx=0x7ffb5802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 -- 172.21.15.17:0/661302687 shutdown_connections 2024-02-14T15:00:09.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 --2- 172.21.15.17:0/661302687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60101510 0x7ffb60107da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 --2- 172.21.15.17:0/661302687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb60100b50 0x7ffb60100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 --2- 172.21.15.17:0/661302687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb600ff960 0x7ffb600ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 -- 172.21.15.17:0/661302687 >> 172.21.15.17:0/661302687 conn(0x7ffb600faeb0 msgr2=0x7ffb600fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 -- 172.21.15.17:0/661302687 shutdown_connections 2024-02-14T15:00:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.996+0000 7ffb66615640 1 -- 172.21.15.17:0/661302687 wait complete. 2024-02-14T15:00:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 Processor -- start 2024-02-14T15:00:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 -- start start 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb600ff960 0x7ffb600fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60100b50 0x7ffb6019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb60101510 0x7ffb6019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6019ede0 con 0x7ffb60100b50 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6019ef50 con 0x7ffb60101510 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb66615640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6019f110 con 0x7ffb600ff960 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb5f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60100b50 0x7ffb6019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb5f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60100b50 0x7ffb6019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44704/0 (socket says 172.21.15.17:44704) 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb5f7fe640 1 -- 172.21.15.17:0/2391152607 learned_addr learned my addr 172.21.15.17:0/2391152607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb5ffff640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb600ff960 0x7ffb600fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.997+0000 7ffb64bb2640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb60101510 0x7ffb6019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5f7fe640 1 -- 172.21.15.17:0/2391152607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb600ff960 msgr2=0x7ffb600fcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5f7fe640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb600ff960 0x7ffb600fcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5f7fe640 1 -- 172.21.15.17:0/2391152607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb60101510 msgr2=0x7ffb6019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5f7fe640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb60101510 0x7ffb6019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5f7fe640 1 -- 172.21.15.17:0/2391152607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009900 con 0x7ffb60100b50 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5f7fe640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60100b50 0x7ffb6019bfe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c890 tx=0x7ffb5400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5d7fa640 1 -- 172.21.15.17:0/2391152607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54004220 con 0x7ffb60100b50 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6019f390 con 0x7ffb60100b50 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5d7fa640 1 -- 172.21.15.17:0/2391152607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb54007e90 con 0x7ffb60100b50 2024-02-14T15:00:10.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb5d7fa640 1 -- 172.21.15.17:0/2391152607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54004a00 con 0x7ffb60100b50 2024-02-14T15:00:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.998+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6019f8f0 con 0x7ffb60100b50 2024-02-14T15:00:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:09.999+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb60107030 con 0x7ffb60100b50 2024-02-14T15:00:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.000+0000 7ffb5d7fa640 1 -- 172.21.15.17:0/2391152607 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffb540079b0 con 0x7ffb60100b50 2024-02-14T15:00:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.000+0000 7ffb5d7fa640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb480a20c0 0x7ffb480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.001+0000 7ffb5ffff640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb480a20c0 0x7ffb480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.001+0000 7ffb5d7fa640 1 -- 172.21.15.17:0/2391152607 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffb540cd890 con 0x7ffb60100b50 2024-02-14T15:00:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.001+0000 7ffb5ffff640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb480a20c0 0x7ffb480a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb601009b0 tx=0x7ffb5000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:10.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.004+0000 7ffb5d7fa640 1 -- 172.21.15.17:0/2391152607 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb54096d10 con 0x7ffb60100b50 2024-02-14T15:00:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.225+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb6010b2d0 con 0x7ffb60100b50 2024-02-14T15:00:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.226+0000 7ffb5d7fa640 1 -- 172.21.15.17:0/2391152607 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7ffb5401d090 con 0x7ffb60100b50 2024-02-14T15:00:10.227 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:10.227 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-14T15:00:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 1 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb480a20c0 msgr2=0x7ffb480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb480a20c0 0x7ffb480a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb601009b0 tx=0x7ffb5000b390 comp rx=0 tx=0).stop 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60100b50 msgr2=0x7ffb6019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60100b50 0x7ffb6019bfe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c890 tx=0x7ffb5400cd50 comp rx=0 tx=0).stop 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 shutdown_connections 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb480a20c0 0x7ffb480a4580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb60100b50 0x7ffb6019bfe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb60101510 0x7ffb6019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 --2- 172.21.15.17:0/2391152607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb600ff960 0x7ffb600fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 >> 172.21.15.17:0/2391152607 conn(0x7ffb600faeb0 msgr2=0x7ffb60104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 shutdown_connections 2024-02-14T15:00:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:10.228+0000 7ffb66615640 1 -- 172.21.15.17:0/2391152607 wait complete. 2024-02-14T15:00:10.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:10 smithi017 ceph-mon[32549]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:10.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:10 smithi017 ceph-mon[32549]: Deploying daemon mds.l on smithi076 2024-02-14T15:00:10.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3397204065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:10.239 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2137165537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:10 smithi050 ceph-mon[36512]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:10 smithi050 ceph-mon[36512]: Deploying daemon mds.l on smithi076 2024-02-14T15:00:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:10 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3397204065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:10 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2137165537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:10.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:10 smithi076 ceph-mon[37057]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:10.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:10 smithi076 ceph-mon[37057]: Deploying daemon mds.l on smithi076 2024-02-14T15:00:10.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:10 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3397204065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:10.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:10 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2137165537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:11.108 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 systemd[1]: Starting Ceph mds.l for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:11.371 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:11 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2391152607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:11.372 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 podman[66955]: 2024-02-14 15:00:11.128570229 +0000 UTC m=+0.018991092 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:11.372 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 podman[66955]: 2024-02-14 15:00:11.247433108 +0000 UTC m=+0.137853952 container create 177ff9ea3110e493cb8d17b0a52be39c19db749a6cc45319dbc33ef42c14779f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-14T15:00:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:11 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2391152607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:11.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:11 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2391152607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:11.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 podman[66955]: 2024-02-14 15:00:11.492821218 +0000 UTC m=+0.383242072 container init 177ff9ea3110e493cb8d17b0a52be39c19db749a6cc45319dbc33ef42c14779f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-14T15:00:11.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 podman[66955]: 2024-02-14 15:00:11.495385713 +0000 UTC m=+0.385806558 container start 177ff9ea3110e493cb8d17b0a52be39c19db749a6cc45319dbc33ef42c14779f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:00:11.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 ceph-mds[66974]: -- 172.21.15.76:0/77577079 <== mon.1 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a5c802f520 con 0x55a5c8005000 2024-02-14T15:00:11.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: starting mds.l at 2024-02-14T15:00:11.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 bash[66955]: 177ff9ea3110e493cb8d17b0a52be39c19db749a6cc45319dbc33ef42c14779f 2024-02-14T15:00:11.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:00:11 smithi076 systemd[1]: Started Ceph mds.l for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:12.241 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 -- 172.21.15.17:0/3063652957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 msgr2=0x7f76700ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 --2- 172.21.15.17:0/3063652957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f76700ff9b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7668009900 tx=0x7f766802bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 -- 172.21.15.17:0/3063652957 shutdown_connections 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 --2- 172.21.15.17:0/3063652957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7670102f20 0x7f76701097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 --2- 172.21.15.17:0/3063652957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7670102560 0x7f76701029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 --2- 172.21.15.17:0/3063652957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f76700ff9b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 -- 172.21.15.17:0/3063652957 >> 172.21.15.17:0/3063652957 conn(0x7f76700faea0 msgr2=0x7f76700fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 -- 172.21.15.17:0/3063652957 shutdown_connections 2024-02-14T15:00:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.347+0000 7f7675039640 1 -- 172.21.15.17:0/3063652957 wait complete. 2024-02-14T15:00:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 Processor -- start 2024-02-14T15:00:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 -- start start 2024-02-14T15:00:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f7670101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7670102560 0x7f7670100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7670102f20 0x7f7670100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701011b0 con 0x7f76700ff590 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7670101320 con 0x7f7670102f20 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f7675039640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701a1360 con 0x7f7670102560 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f766ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f7670101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f766ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f7670101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39280/0 (socket says 172.21.15.17:39280) 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f766ed9d640 1 -- 172.21.15.17:0/716553976 learned_addr learned my addr 172.21.15.17:0/716553976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f766f59e640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7670102f20 0x7f7670100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.348+0000 7f766e59c640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7670102560 0x7f7670100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f766ed9d640 1 -- 172.21.15.17:0/716553976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7670102560 msgr2=0x7f7670100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f766ed9d640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7670102560 0x7f7670100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f766ed9d640 1 -- 172.21.15.17:0/716553976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7670102f20 msgr2=0x7f7670100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f766ed9d640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7670102f20 0x7f7670100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f766ed9d640 1 -- 172.21.15.17:0/716553976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7668009560 con 0x7f76700ff590 2024-02-14T15:00:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f766e59c640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7670102560 0x7f7670100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f766ed9d640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f7670101ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7668036040 tx=0x7f7668004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f7667fff640 1 -- 172.21.15.17:0/716553976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668004df0 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f7667fff640 1 -- 172.21.15.17:0/716553976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7668005430 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76701a15b0 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f7667fff640 1 -- 172.21.15.17:0/716553976 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766803e450 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.349+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76701a1ac0 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.350+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7670108690 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.351+0000 7f7667fff640 1 -- 172.21.15.17:0/716553976 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f76680055a0 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.352+0000 7f7667fff640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f765c0a2140 0x7f765c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.352+0000 7f766e59c640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f765c0a2140 0x7f765c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.352+0000 7f7667fff640 1 -- 172.21.15.17:0/716553976 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f76680efc70 con 0x7f76700ff590 2024-02-14T15:00:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.352+0000 7f766e59c640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f765c0a2140 0x7f765c0a4600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76580058c0 tx=0x7f7658005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.357+0000 7f7667fff640 1 -- 172.21.15.17:0/716553976 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76680b9040 con 0x7f76700ff590 2024-02-14T15:00:12.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.575+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f767002cde0 con 0x7f76700ff590 2024-02-14T15:00:12.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.575+0000 7f7667fff640 1 -- 172.21.15.17:0/716553976 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1322 (secure 0 0 0) 0x7f76680bc700 con 0x7f76700ff590 2024-02-14T15:00:12.576 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:12.576 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-14T15:00:12.577 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2024-02-14T15:00:12.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.577+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f765c0a2140 msgr2=0x7f765c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:12.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.577+0000 7f7675039640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f765c0a2140 0x7f765c0a4600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76580058c0 tx=0x7f7658005850 comp rx=0 tx=0).stop 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.577+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 msgr2=0x7f7670101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.577+0000 7f7675039640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f7670101ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7668036040 tx=0x7f7668004270 comp rx=0 tx=0).stop 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 shutdown_connections 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f765c0a2140 0x7f765c0a4600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7670102f20 0x7f7670100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7670102560 0x7f7670100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 --2- 172.21.15.17:0/716553976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76700ff590 0x7f7670101ed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 >> 172.21.15.17:0/716553976 conn(0x7f76700faea0 msgr2=0x7f7670105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 shutdown_connections 2024-02-14T15:00:12.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:12.578+0000 7f7675039640 1 -- 172.21.15.17:0/716553976 wait complete. 2024-02-14T15:00:12.596 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:12.596 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6832/580745027,v1:172.21.15.76:6833/580745027] up:boot 2024-02-14T15:00:12.596 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: fsmap 1 up:standby 2024-02-14T15:00:12.596 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:00:12.596 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:12.596 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:12.597 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:12.597 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:12.597 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:12.597 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:12 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6832/580745027,v1:172.21.15.76:6833/580745027] up:boot 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: fsmap 1 up:standby 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:12 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6832/580745027,v1:172.21.15.76:6833/580745027] up:boot 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: fsmap 1 up:standby 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:12 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:13.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:13 smithi050 ceph-mon[36512]: Deploying daemon mds.k on smithi050 2024-02-14T15:00:13.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/716553976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:13 smithi076 ceph-mon[37057]: Deploying daemon mds.k on smithi050 2024-02-14T15:00:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/716553976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:14.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:13 smithi017 ceph-mon[32549]: Deploying daemon mds.k on smithi050 2024-02-14T15:00:14.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/716553976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:14.393 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:00:14 smithi050 podman[66251]: 2024-02-14 15:00:14.346460014 +0000 UTC m=+0.124325723 container create bfcf9c1b0bed5e3d89451a09494b0890751a7fc020e3786cf44b82a70c4f8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:00:14.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3229797306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c381024b0 msgr2=0x7f0c38102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3229797306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c381024b0 0x7f0c38102930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009900 tx=0x7f0c2c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3229797306 shutdown_connections 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3229797306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c381024b0 0x7f0c38102930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3229797306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c38102e70 0x7f0c38109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3229797306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c381012c0 0x7f0c381016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3229797306 >> 172.21.15.17:0/3229797306 conn(0x7f0c380fc850 msgr2=0x7f0c380fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:14.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3229797306 shutdown_connections 2024-02-14T15:00:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.706+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3229797306 wait complete. 2024-02-14T15:00:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 Processor -- start 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 -- start start 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c381012c0 0x7f0c381972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c381024b0 0x7f0c381977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c38102e70 0x7f0c3819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3819ed90 con 0x7f0c38102e70 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3819ef00 con 0x7f0c381024b0 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c3d8a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3819f0c0 con 0x7f0c381012c0 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c38102e70 0x7f0c3819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c38102e70 0x7f0c3819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39308/0 (socket says 172.21.15.17:39308) 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 -- 172.21.15.17:0/3630751667 learned_addr learned my addr 172.21.15.17:0/3630751667 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c367fc640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c381024b0 0x7f0c381977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c36ffd640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c381012c0 0x7f0c381972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 -- 172.21.15.17:0/3630751667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c381012c0 msgr2=0x7f0c381972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c381012c0 0x7f0c381972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 -- 172.21.15.17:0/3630751667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c381024b0 msgr2=0x7f0c381977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c381024b0 0x7f0c381977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c377fe640 1 -- 172.21.15.17:0/3630751667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c009560 con 0x7f0c38102e70 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.707+0000 7f0c36ffd640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c381012c0 0x7f0c381972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.708+0000 7f0c377fe640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c38102e70 0x7f0c3819e850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009fd0 tx=0x7f0c3000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.708+0000 7f0c3c8a2640 1 -- 172.21.15.17:0/3630751667 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000bea0 con 0x7f0c38102e70 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.708+0000 7f0c3c8a2640 1 -- 172.21.15.17:0/3630751667 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c30009940 con 0x7f0c38102e70 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.708+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3819f3a0 con 0x7f0c38102e70 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.708+0000 7f0c3c8a2640 1 -- 172.21.15.17:0/3630751667 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c300106c0 con 0x7f0c38102e70 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.708+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3819f830 con 0x7f0c38102e70 2024-02-14T15:00:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.709+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c38108630 con 0x7f0c38102e70 2024-02-14T15:00:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.710+0000 7f0c3c8a2640 1 -- 172.21.15.17:0/3630751667 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0c30004240 con 0x7f0c38102e70 2024-02-14T15:00:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.710+0000 7f0c3c8a2640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0c200a20c0 0x7f0c200a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.710+0000 7f0c36ffd640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0c200a20c0 0x7f0c200a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.711+0000 7f0c3c8a2640 1 -- 172.21.15.17:0/3630751667 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0c300cd510 con 0x7f0c38102e70 2024-02-14T15:00:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.711+0000 7f0c36ffd640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0c200a20c0 0x7f0c200a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c38102310 tx=0x7f0c28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.715+0000 7f0c3c8a2640 1 -- 172.21.15.17:0/3630751667 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c30096980 con 0x7f0c38102e70 2024-02-14T15:00:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:14 smithi050 ceph-mon[36512]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:14.892 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:00:14 smithi050 podman[66251]: 2024-02-14 15:00:14.582995538 +0000 UTC m=+0.360861232 container init bfcf9c1b0bed5e3d89451a09494b0890751a7fc020e3786cf44b82a70c4f8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:00:14.892 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:00:14 smithi050 podman[66251]: 2024-02-14 15:00:14.585724533 +0000 UTC m=+0.363590221 container start bfcf9c1b0bed5e3d89451a09494b0890751a7fc020e3786cf44b82a70c4f8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0) 2024-02-14T15:00:14.892 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:00:14 smithi050 ceph-mds[66270]: -- 172.21.15.50:0/1993623248 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f931e49520 con 0x55f931e1f000 2024-02-14T15:00:14.893 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:00:14 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k[66266]: starting mds.k at 2024-02-14T15:00:14.893 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:00:14 smithi050 bash[66251]: bfcf9c1b0bed5e3d89451a09494b0890751a7fc020e3786cf44b82a70c4f8fbc 2024-02-14T15:00:14.893 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:00:14 smithi050 systemd[1]: Started Ceph mds.k for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.943+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c3810cc60 con 0x7f0c38102e70 2024-02-14T15:00:14.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.943+0000 7f0c3c8a2640 1 -- 172.21.15.17:0/3630751667 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1322 (secure 0 0 0) 0x7f0c3009a040 con 0x7f0c38102e70 2024-02-14T15:00:14.944 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:14.944 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-14T15:00:14.944 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2024-02-14T15:00:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0c200a20c0 msgr2=0x7f0c200a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0c200a20c0 0x7f0c200a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0c38102310 tx=0x7f0c28005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c38102e70 msgr2=0x7f0c3819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c38102e70 0x7f0c3819e850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009fd0 tx=0x7f0c3000b260 comp rx=0 tx=0).stop 2024-02-14T15:00:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 shutdown_connections 2024-02-14T15:00:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0c200a20c0 0x7f0c200a4580 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c38102e70 0x7f0c3819e850 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c381024b0 0x7f0c381977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 --2- 172.21.15.17:0/3630751667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c381012c0 0x7f0c381972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 >> 172.21.15.17:0/3630751667 conn(0x7f0c380fc850 msgr2=0x7f0c38105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 shutdown_connections 2024-02-14T15:00:14.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:14.945+0000 7f0c3d8a4640 1 -- 172.21.15.17:0/3630751667 wait complete. 2024-02-14T15:00:14.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:14 smithi017 ceph-mon[32549]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:14 smithi076 ceph-mon[37057]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: Deploying daemon mds.j on smithi017 2024-02-14T15:00:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3630751667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6832/277751222,v1:172.21.15.50:6833/277751222] up:boot 2024-02-14T15:00:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: fsmap 2 up:standby 2024-02-14T15:00:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:15 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:16.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: Deploying daemon mds.j on smithi017 2024-02-14T15:00:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3630751667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6832/277751222,v1:172.21.15.50:6833/277751222] up:boot 2024-02-14T15:00:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: fsmap 2 up:standby 2024-02-14T15:00:16.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:15 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-14T15:00:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: Deploying daemon mds.j on smithi017 2024-02-14T15:00:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3630751667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6832/277751222,v1:172.21.15.50:6833/277751222] up:boot 2024-02-14T15:00:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: fsmap 2 up:standby 2024-02-14T15:00:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:15 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-14T15:00:16.958 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 -- 172.21.15.17:0/276025607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708101280 msgr2=0x7f07081016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:17.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 --2- 172.21.15.17:0/276025607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708101280 0x7f07081016a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0700009900 tx=0x7f070002bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 -- 172.21.15.17:0/276025607 shutdown_connections 2024-02-14T15:00:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 --2- 172.21.15.17:0/276025607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0708102e30 0x7f07081096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 --2- 172.21.15.17:0/276025607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0708102470 0x7f07081028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 --2- 172.21.15.17:0/276025607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708101280 0x7f07081016a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 -- 172.21.15.17:0/276025607 >> 172.21.15.17:0/276025607 conn(0x7f07080fc7f0 msgr2=0x7f07080fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 -- 172.21.15.17:0/276025607 shutdown_connections 2024-02-14T15:00:17.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 -- 172.21.15.17:0/276025607 wait complete. 2024-02-14T15:00:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 Processor -- start 2024-02-14T15:00:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.069+0000 7f070cdac640 1 -- start start 2024-02-14T15:00:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0708101280 0x7f07081973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:17.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708102470 0x7f07081978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0708102e30 0x7f070819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070819ee90 con 0x7f0708102470 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070819f000 con 0x7f0708101280 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070819f1c0 con 0x7f0708102e30 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708102470 0x7f07081978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708102470 0x7f07081978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39334/0 (socket says 172.21.15.17:39334) 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 -- 172.21.15.17:0/2529054311 learned_addr learned my addr 172.21.15.17:0/2529054311 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070659c640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0708101280 0x7f07081973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0706d9d640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0708102e30 0x7f070819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 -- 172.21.15.17:0/2529054311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0708102e30 msgr2=0x7f070819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0708102e30 0x7f070819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 -- 172.21.15.17:0/2529054311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0708101280 msgr2=0x7f07081973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0708101280 0x7f07081973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 -- 172.21.15.17:0/2529054311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700009560 con 0x7f0708102470 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0706d9d640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0708102e30 0x7f070819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f0705d9b640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708102470 0x7f07081978e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d950 tx=0x7f06fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f06f77fe640 1 -- 172.21.15.17:0/2529054311 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc009c30 con 0x7f0708102470 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f06f77fe640 1 -- 172.21.15.17:0/2529054311 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06fc009d90 con 0x7f0708102470 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070819f4a0 con 0x7f0708102470 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f06f77fe640 1 -- 172.21.15.17:0/2529054311 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00f7e0 con 0x7f0708102470 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.070+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070819fa30 con 0x7f0708102470 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.071+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07081085a0 con 0x7f0708102470 2024-02-14T15:00:17.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.072+0000 7f06f77fe640 1 -- 172.21.15.17:0/2529054311 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f06fc010ad0 con 0x7f0708102470 2024-02-14T15:00:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.073+0000 7f06f77fe640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06f00a20c0 0x7f06f00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.073+0000 7f070659c640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06f00a20c0 0x7f06f00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.073+0000 7f06f77fe640 1 -- 172.21.15.17:0/2529054311 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06fc0cd400 con 0x7f0708102470 2024-02-14T15:00:17.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.073+0000 7f070659c640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06f00a20c0 0x7f06f00a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0700005ab0 tx=0x7f0700005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:17.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.077+0000 7f06f77fe640 1 -- 172.21.15.17:0/2529054311 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06fc096870 con 0x7f0708102470 2024-02-14T15:00:17.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.301+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f070810cdf0 con 0x7f0708102470 2024-02-14T15:00:17.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.302+0000 7f06f77fe640 1 -- 172.21.15.17:0/2529054311 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2110 (secure 0 0 0) 0x7f06fc020850 con 0x7f0708102470 2024-02-14T15:00:17.303 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:17.303 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-14T15:00:17.303 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 3 2024-02-14T15:00:17.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06f00a20c0 msgr2=0x7f06f00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:17.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06f00a20c0 0x7f06f00a4580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0700005ab0 tx=0x7f0700005fd0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708102470 msgr2=0x7f07081978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:17.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708102470 0x7f07081978e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d950 tx=0x7f06fc00de10 comp rx=0 tx=0).stop 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 shutdown_connections 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06f00a20c0 0x7f06f00a4580 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0708102470 0x7f07081978e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0708102e30 0x7f070819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 --2- 172.21.15.17:0/2529054311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0708101280 0x7f07081973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 >> 172.21.15.17:0/2529054311 conn(0x7f07080fc7f0 msgr2=0x7f0708105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 shutdown_connections 2024-02-14T15:00:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:17.304+0000 7f070cdac640 1 -- 172.21.15.17:0/2529054311 wait complete. 2024-02-14T15:00:17.725 INFO:journalctl@ceph.mds.j.smithi017.stdout:Feb 14 15:00:17 smithi017 systemd[1]: Started Ceph mds.j for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2529054311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:17 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2529054311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:18.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:17 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2529054311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:18.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:17 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:18 smithi076 ceph-mon[37057]: Deploying daemon mds.d on smithi017 2024-02-14T15:00:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:18 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.17:6834/763584482,v1:172.21.15.17:6835/763584482] up:boot 2024-02-14T15:00:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:18 smithi076 ceph-mon[37057]: fsmap 3 up:standby 2024-02-14T15:00:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:18 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-14T15:00:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:18 smithi017 ceph-mon[32549]: Deploying daemon mds.d on smithi017 2024-02-14T15:00:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:18 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.17:6834/763584482,v1:172.21.15.17:6835/763584482] up:boot 2024-02-14T15:00:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:18 smithi017 ceph-mon[32549]: fsmap 3 up:standby 2024-02-14T15:00:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:18 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-14T15:00:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:18 smithi050 ceph-mon[36512]: Deploying daemon mds.d on smithi017 2024-02-14T15:00:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:18 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.17:6834/763584482,v1:172.21.15.17:6835/763584482] up:boot 2024-02-14T15:00:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:18 smithi050 ceph-mon[36512]: fsmap 3 up:standby 2024-02-14T15:00:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:18 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-14T15:00:19.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 -- 172.21.15.17:0/3117173472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101d10 msgr2=0x7fdbe41085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 --2- 172.21.15.17:0/3117173472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101d10 0x7fdbe41085a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009ca0 tx=0x7fdbdc02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 -- 172.21.15.17:0/3117173472 shutdown_connections 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 --2- 172.21.15.17:0/3117173472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101d10 0x7fdbe41085a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 --2- 172.21.15.17:0/3117173472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe4101350 0x7fdbe41017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 --2- 172.21.15.17:0/3117173472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbe4100160 0x7fdbe4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 -- 172.21.15.17:0/3117173472 >> 172.21.15.17:0/3117173472 conn(0x7fdbe40fb6d0 msgr2=0x7fdbe40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 -- 172.21.15.17:0/3117173472 shutdown_connections 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 -- 172.21.15.17:0/3117173472 wait complete. 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.435+0000 7fdbea13e640 1 Processor -- start 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbea13e640 1 -- start start 2024-02-14T15:00:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbea13e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe4100160 0x7fdbe4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbea13e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101350 0x7fdbe4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbea13e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbe4101d10 0x7fdbe419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbea13e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe419ecf0 con 0x7fdbe4101350 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbea13e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe419ee60 con 0x7fdbe4100160 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbea13e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe419f020 con 0x7fdbe4101d10 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101350 0x7fdbe4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101350 0x7fdbe4197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39374/0 (socket says 172.21.15.17:39374) 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 -- 172.21.15.17:0/1199767407 learned_addr learned my addr 172.21.15.17:0/1199767407 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe3fff640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbe4101d10 0x7fdbe419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe37fe640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe4100160 0x7fdbe4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 -- 172.21.15.17:0/1199767407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbe4101d10 msgr2=0x7fdbe419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbe4101d10 0x7fdbe419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 -- 172.21.15.17:0/1199767407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe4100160 msgr2=0x7fdbe4197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe4100160 0x7fdbe4197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.436+0000 7fdbe2ffd640 1 -- 172.21.15.17:0/1199767407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc009900 con 0x7fdbe4101350 2024-02-14T15:00:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.437+0000 7fdbe2ffd640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101350 0x7fdbe4197740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400d950 tx=0x7fdbd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.437+0000 7fdbe0ff9640 1 -- 172.21.15.17:0/1199767407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4009c30 con 0x7fdbe4101350 2024-02-14T15:00:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.437+0000 7fdbe0ff9640 1 -- 172.21.15.17:0/1199767407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbd4009d90 con 0x7fdbe4101350 2024-02-14T15:00:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.437+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe419f300 con 0x7fdbe4101350 2024-02-14T15:00:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.437+0000 7fdbe0ff9640 1 -- 172.21.15.17:0/1199767407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400f7e0 con 0x7fdbe4101350 2024-02-14T15:00:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.437+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe419f860 con 0x7fdbe4101350 2024-02-14T15:00:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.438+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe4107480 con 0x7fdbe4101350 2024-02-14T15:00:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.440+0000 7fdbe0ff9640 1 -- 172.21.15.17:0/1199767407 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdbd4010ad0 con 0x7fdbe4101350 2024-02-14T15:00:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.440+0000 7fdbe0ff9640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdbd00a2140 0x7fdbd00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.440+0000 7fdbe37fe640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdbd00a2140 0x7fdbd00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.440+0000 7fdbe0ff9640 1 -- 172.21.15.17:0/1199767407 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbd40cd350 con 0x7fdbe4101350 2024-02-14T15:00:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.441+0000 7fdbe37fe640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdbd00a2140 0x7fdbd00a4600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdbe41011b0 tx=0x7fdbcc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.445+0000 7fdbe0ff9640 1 -- 172.21.15.17:0/1199767407 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdbd4096870 con 0x7fdbe4101350 2024-02-14T15:00:19.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.696+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbe402ce50 con 0x7fdbe4101350 2024-02-14T15:00:19.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.697+0000 7fdbe0ff9640 1 -- 172.21.15.17:0/1199767407 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2898 (secure 0 0 0) 0x7fdbd4099f30 con 0x7fdbe4101350 2024-02-14T15:00:19.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:19.698 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-14T15:00:19.698 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 4 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdbd00a2140 msgr2=0x7fdbd00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdbd00a2140 0x7fdbd00a4600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdbe41011b0 tx=0x7fdbcc00b390 comp rx=0 tx=0).stop 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101350 msgr2=0x7fdbe4197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101350 0x7fdbe4197740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400d950 tx=0x7fdbd400de10 comp rx=0 tx=0).stop 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 shutdown_connections 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdbd00a2140 0x7fdbd00a4600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbe4101350 0x7fdbe4197740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbe4101d10 0x7fdbe419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 --2- 172.21.15.17:0/1199767407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbe4100160 0x7fdbe4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 >> 172.21.15.17:0/1199767407 conn(0x7fdbe40fb6d0 msgr2=0x7fdbe4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:19.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 shutdown_connections 2024-02-14T15:00:19.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:19.699+0000 7fdbea13e640 1 -- 172.21.15.17:0/1199767407 wait complete. 2024-02-14T15:00:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:19 smithi076 ceph-mon[37057]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:19 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1199767407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:20.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:19 smithi017 ceph-mon[32549]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:20.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:19 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1199767407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:20.016 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:19 smithi017 systemd[1]: Starting Ceph mds.d for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:20.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:19 smithi050 ceph-mon[36512]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:20.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:19 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1199767407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:20.283 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 podman[73188]: 2024-02-14 15:00:19.928715441 +0000 UTC m=+0.017715052 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:20.284 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 podman[73188]: 2024-02-14 15:00:20.050885922 +0000 UTC m=+0.139885528 container create 0686cf100b386a546f422a715eafb9a99af52c65fb22060034d5cb9161cfd9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:00:20.284 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 podman[73188]: 2024-02-14 15:00:20.283342909 +0000 UTC m=+0.372342512 container init 0686cf100b386a546f422a715eafb9a99af52c65fb22060034d5cb9161cfd9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-14T15:00:20.766 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 podman[73188]: 2024-02-14 15:00:20.285973442 +0000 UTC m=+0.374973044 container start 0686cf100b386a546f422a715eafb9a99af52c65fb22060034d5cb9161cfd9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-14T15:00:20.767 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 ceph-mds[73207]: -- 172.21.15.17:0/2192876884 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55942d31f520 con 0x55942d2f4c00 2024-02-14T15:00:20.767 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d[73203]: starting mds.d at 2024-02-14T15:00:20.767 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 bash[73188]: 0686cf100b386a546f422a715eafb9a99af52c65fb22060034d5cb9161cfd9d1 2024-02-14T15:00:20.767 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:00:20 smithi017 systemd[1]: Started Ceph mds.d for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:21.712 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: Deploying daemon mds.e on smithi050 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.17:6836/2485072299,v1:172.21.15.17:6837/2485072299] up:boot 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: fsmap 4 up:standby 2024-02-14T15:00:21.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:21 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-14T15:00:21.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.742 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.743 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:21.743 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:21.743 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:21.743 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: Deploying daemon mds.e on smithi050 2024-02-14T15:00:21.743 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.17:6836/2485072299,v1:172.21.15.17:6837/2485072299] up:boot 2024-02-14T15:00:21.743 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: fsmap 4 up:standby 2024-02-14T15:00:21.743 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:21 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-14T15:00:21.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:21.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:21.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:21.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:21.757 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: Deploying daemon mds.e on smithi050 2024-02-14T15:00:21.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.17:6836/2485072299,v1:172.21.15.17:6837/2485072299] up:boot 2024-02-14T15:00:21.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: fsmap 4 up:standby 2024-02-14T15:00:21.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:21 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-14T15:00:21.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.829+0000 7f762474f640 1 -- 172.21.15.17:0/2617666600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 msgr2=0x7f761c109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:21.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.829+0000 7f762474f640 1 --2- 172.21.15.17:0/2617666600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c109400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7614009ca0 tx=0x7f761402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:21.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- 172.21.15.17:0/2617666600 shutdown_connections 2024-02-14T15:00:21.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 --2- 172.21.15.17:0/2617666600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c109400 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:21.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 --2- 172.21.15.17:0/2617666600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f761c104680 0x7f761c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:21.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 --2- 172.21.15.17:0/2617666600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c101d50 0x7f761c104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:21.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- 172.21.15.17:0/2617666600 >> 172.21.15.17:0/2617666600 conn(0x7f761c0fb740 msgr2=0x7f761c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- 172.21.15.17:0/2617666600 shutdown_connections 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- 172.21.15.17:0/2617666600 wait complete. 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 Processor -- start 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- start start 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f761c101d50 0x7f761c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c104680 0x7f761c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c19ed60 con 0x7f761c106fb0 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c19eed0 con 0x7f761c101d50 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.830+0000 7f762474f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c19f090 con 0x7f761c104680 2024-02-14T15:00:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41282/0 (socket says 172.21.15.17:41282) 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 -- 172.21.15.17:0/2536106305 learned_addr learned my addr 172.21.15.17:0/2536106305 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f76224eb640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f761c101d50 0x7f761c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7621cea640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c104680 0x7f761c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 -- 172.21.15.17:0/2536106305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c104680 msgr2=0x7f761c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c104680 0x7f761c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 -- 172.21.15.17:0/2536106305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f761c101d50 msgr2=0x7f761c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f761c101d50 0x7f761c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 -- 172.21.15.17:0/2536106305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614009900 con 0x7f761c106fb0 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f7622cec640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c19e820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f761402c700 tx=0x7f7614034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f76137fe640 1 -- 172.21.15.17:0/2536106305 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76140329b0 con 0x7f761c106fb0 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c19f310 con 0x7f761c106fb0 2024-02-14T15:00:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f76137fe640 1 -- 172.21.15.17:0/2536106305 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7614003e50 con 0x7f761c106fb0 2024-02-14T15:00:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f76137fe640 1 -- 172.21.15.17:0/2536106305 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614040450 con 0x7f761c106fb0 2024-02-14T15:00:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.831+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c19f810 con 0x7f761c106fb0 2024-02-14T15:00:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.832+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c0fd230 con 0x7f761c106fb0 2024-02-14T15:00:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.834+0000 7f76137fe640 1 -- 172.21.15.17:0/2536106305 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f761400a040 con 0x7f761c106fb0 2024-02-14T15:00:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.834+0000 7f76137fe640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76080a20c0 0x7f76080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.834+0000 7f76224eb640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76080a20c0 0x7f76080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.834+0000 7f76137fe640 1 -- 172.21.15.17:0/2536106305 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f76140f0a40 con 0x7f761c106fb0 2024-02-14T15:00:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.835+0000 7f76224eb640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76080a20c0 0x7f76080a4580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7618009bd0 tx=0x7f7618005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:21.838+0000 7f76137fe640 1 -- 172.21.15.17:0/2536106305 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76140b9eb0 con 0x7f761c106fb0 2024-02-14T15:00:22.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.058+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f761c02cde0 con 0x7f761c106fb0 2024-02-14T15:00:22.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.058+0000 7f76137fe640 1 -- 172.21.15.17:0/2536106305 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3689 (secure 0 0 0) 0x7f76140bd570 con 0x7f761c106fb0 2024-02-14T15:00:22.059 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:22.059 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-14T15:00:22.059 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 5 2024-02-14T15:00:22.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76080a20c0 msgr2=0x7f76080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:22.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76080a20c0 0x7f76080a4580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7618009bd0 tx=0x7f7618005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 msgr2=0x7f761c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c19e820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f761402c700 tx=0x7f7614034a90 comp rx=0 tx=0).stop 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 shutdown_connections 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76080a20c0 0x7f76080a4580 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f761c106fb0 0x7f761c19e820 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c104680 0x7f761c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 --2- 172.21.15.17:0/2536106305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f761c101d50 0x7f761c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 >> 172.21.15.17:0/2536106305 conn(0x7f761c0fb740 msgr2=0x7f761c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 shutdown_connections 2024-02-14T15:00:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:22.060+0000 7f762474f640 1 -- 172.21.15.17:0/2536106305 wait complete. 2024-02-14T15:00:22.878 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:22 smithi050 ceph-mon[36512]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:22.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:22 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2536106305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:22.879 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:22 smithi050 systemd[1]: Starting Ceph mds.e for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:22.913 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:22 smithi076 ceph-mon[37057]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:22.913 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:22 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2536106305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:22 smithi017 ceph-mon[32549]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:22 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2536106305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:23.142 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:22 smithi050 podman[66576]: 2024-02-14 15:00:22.781498532 +0000 UTC m=+0.018615742 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:23.143 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:22 smithi050 podman[66576]: 2024-02-14 15:00:22.910836574 +0000 UTC m=+0.147953770 container create 6b8e546e9b1e32438b403f6cafafb615f872dfbc1c05f734ee0d6cc636d049f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, io.buildah.version=1.33.2) 2024-02-14T15:00:23.642 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:23 smithi050 podman[66576]: 2024-02-14 15:00:23.163501027 +0000 UTC m=+0.400618224 container init 6b8e546e9b1e32438b403f6cafafb615f872dfbc1c05f734ee0d6cc636d049f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, org.label-schema.schema-version=1.0) 2024-02-14T15:00:23.642 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:23 smithi050 podman[66576]: 2024-02-14 15:00:23.166252395 +0000 UTC m=+0.403369592 container start 6b8e546e9b1e32438b403f6cafafb615f872dfbc1c05f734ee0d6cc636d049f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-14T15:00:23.642 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:23 smithi050 ceph-mds[66595]: -- 172.21.15.50:0/3789980562 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a47488f520 con 0x55a474864800 2024-02-14T15:00:23.642 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e[66591]: starting mds.e at 2024-02-14T15:00:23.643 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:23 smithi050 bash[66576]: 6b8e546e9b1e32438b403f6cafafb615f872dfbc1c05f734ee0d6cc636d049f4 2024-02-14T15:00:23.643 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:00:23 smithi050 systemd[1]: Started Ceph mds.e for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:24.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 -- 172.21.15.17:0/122397417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101350 msgr2=0x7fcb081017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/122397417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101350 0x7fcb081017d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009900 tx=0x7fcaf402bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 -- 172.21.15.17:0/122397417 shutdown_connections 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/122397417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101350 0x7fcb081017d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/122397417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb08101d10 0x7fcb081085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/122397417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb08100160 0x7fcb08100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 -- 172.21.15.17:0/122397417 >> 172.21.15.17:0/122397417 conn(0x7fcb080fb6f0 msgr2=0x7fcb080fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 -- 172.21.15.17:0/122397417 shutdown_connections 2024-02-14T15:00:24.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.187+0000 7fcb0ed65640 1 -- 172.21.15.17:0/122397417 wait complete. 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 Processor -- start 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 -- start start 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb08100160 0x7fcb0810aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb08101350 0x7fcb08109060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101d10 0x7fcb081095a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08109d00 con 0x7fcb08101d10 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08109e70 con 0x7fcb08101350 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0ed65640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081a12a0 con 0x7fcb08100160 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101d10 0x7fcb081095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101d10 0x7fcb081095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41298/0 (socket says 172.21.15.17:41298) 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 -- 172.21.15.17:0/2474054964 learned_addr learned my addr 172.21.15.17:0/2474054964 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb07fff640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb08101350 0x7fcb08109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:24.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0cb01640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb08100160 0x7fcb0810aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 -- 172.21.15.17:0/2474054964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb08100160 msgr2=0x7fcb0810aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb08100160 0x7fcb0810aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 -- 172.21.15.17:0/2474054964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb08101350 msgr2=0x7fcb08109060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb08101350 0x7fcb08109060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 -- 172.21.15.17:0/2474054964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4009560 con 0x7fcb08101d10 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.188+0000 7fcb0d302640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101d10 0x7fcb081095a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000cb00 tx=0x7fcb0000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.189+0000 7fcb05ffb640 1 -- 172.21.15.17:0/2474054964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000ae90 con 0x7fcb08101d10 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.189+0000 7fcb05ffb640 1 -- 172.21.15.17:0/2474054964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb00005020 con 0x7fcb08101d10 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.189+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb081a1550 con 0x7fcb08101d10 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.189+0000 7fcb05ffb640 1 -- 172.21.15.17:0/2474054964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000f6b0 con 0x7fcb08101d10 2024-02-14T15:00:24.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.189+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb081a1a30 con 0x7fcb08101d10 2024-02-14T15:00:24.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.190+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb08107420 con 0x7fcb08101d10 2024-02-14T15:00:24.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.191+0000 7fcb05ffb640 1 -- 172.21.15.17:0/2474054964 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcb00005520 con 0x7fcb08101d10 2024-02-14T15:00:24.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.191+0000 7fcb05ffb640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcaf80a20c0 0x7fcaf80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:24.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.192+0000 7fcb0cb01640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcaf80a20c0 0x7fcaf80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:24.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.192+0000 7fcb05ffb640 1 -- 172.21.15.17:0/2474054964 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcb000cd890 con 0x7fcb08101d10 2024-02-14T15:00:24.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.192+0000 7fcb0cb01640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcaf80a20c0 0x7fcaf80a4580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0058c0 tx=0x7fcafc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:24.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.196+0000 7fcb05ffb640 1 -- 172.21.15.17:0/2474054964 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb00096c60 con 0x7fcb08101d10 2024-02-14T15:00:24.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.418+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb0810cc70 con 0x7fcb08101d10 2024-02-14T15:00:24.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.418+0000 7fcb05ffb640 1 -- 172.21.15.17:0/2474054964 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4480 (secure 0 0 0) 0x7fcb0009a320 con 0x7fcb08101d10 2024-02-14T15:00:24.419 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:24.419 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-14T15:00:24.419 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 6 2024-02-14T15:00:24.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcaf80a20c0 msgr2=0x7fcaf80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:24.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcaf80a20c0 0x7fcaf80a4580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0058c0 tx=0x7fcafc005850 comp rx=0 tx=0).stop 2024-02-14T15:00:24.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101d10 msgr2=0x7fcb081095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:24.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101d10 0x7fcb081095a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000cb00 tx=0x7fcb0000a250 comp rx=0 tx=0).stop 2024-02-14T15:00:24.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 shutdown_connections 2024-02-14T15:00:24.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcaf80a20c0 0x7fcaf80a4580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb08101d10 0x7fcb081095a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb08101350 0x7fcb08109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 --2- 172.21.15.17:0/2474054964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb08100160 0x7fcb0810aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:24.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 >> 172.21.15.17:0/2474054964 conn(0x7fcb080fb6f0 msgr2=0x7fcb081045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:24.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 shutdown_connections 2024-02-14T15:00:24.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:24.420+0000 7fcb0ed65640 1 -- 172.21.15.17:0/2474054964 wait complete. 2024-02-14T15:00:24.430 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:24.430 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.430 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.430 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.430 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:24.431 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:24.431 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:24.431 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: Deploying daemon mds.f on smithi076 2024-02-14T15:00:24.431 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6834/3474801477,v1:172.21.15.50:6835/3474801477] up:boot 2024-02-14T15:00:24.431 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: fsmap 5 up:standby 2024-02-14T15:00:24.431 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:24 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: Deploying daemon mds.f on smithi076 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6834/3474801477,v1:172.21.15.50:6835/3474801477] up:boot 2024-02-14T15:00:24.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: fsmap 5 up:standby 2024-02-14T15:00:24.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:24 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:00:24.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:24.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: Deploying daemon mds.f on smithi076 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6834/3474801477,v1:172.21.15.50:6835/3474801477] up:boot 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: fsmap 5 up:standby 2024-02-14T15:00:24.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:24 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:00:25.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:25 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2474054964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:25.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:25 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2474054964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:25.734 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:25 smithi076 systemd[1]: Starting Ceph mds.f for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:25.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:25 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2474054964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:26.009 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:25 smithi076 podman[67274]: 2024-02-14 15:00:25.654564059 +0000 UTC m=+0.017288185 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:26.009 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:25 smithi076 podman[67274]: 2024-02-14 15:00:25.772868637 +0000 UTC m=+0.135592755 container create fbbd60dda08a5b307eaaf0f675463797033482397c13103e75e18e1004896835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS) 2024-02-14T15:00:26.009 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:26 smithi076 podman[67274]: 2024-02-14 15:00:26.009335499 +0000 UTC m=+0.372059640 container init fbbd60dda08a5b307eaaf0f675463797033482397c13103e75e18e1004896835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:00:26.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:26.484 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:26 smithi076 podman[67274]: 2024-02-14 15:00:26.011887842 +0000 UTC m=+0.374611970 container start fbbd60dda08a5b307eaaf0f675463797033482397c13103e75e18e1004896835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:00:26.484 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mds[67294]: -- 172.21.15.76:0/1265680295 <== mon.1 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5634f6b63520 con 0x5634f6b38800 2024-02-14T15:00:26.484 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f[67290]: starting mds.f at 2024-02-14T15:00:26.484 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:26 smithi076 bash[67274]: fbbd60dda08a5b307eaaf0f675463797033482397c13103e75e18e1004896835 2024-02-14T15:00:26.484 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:00:26 smithi076 systemd[1]: Started Ceph mds.f for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 -- 172.21.15.17:0/2045147183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4100160 msgr2=0x7f5ea4100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/2045147183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4100160 0x7f5ea4100580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009900 tx=0x7f5e9402bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 -- 172.21.15.17:0/2045147183 shutdown_connections 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/2045147183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea4101d10 0x7f5ea41085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/2045147183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea4101350 0x7f5ea41017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/2045147183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4100160 0x7f5ea4100580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 -- 172.21.15.17:0/2045147183 >> 172.21.15.17:0/2045147183 conn(0x7f5ea40fb6f0 msgr2=0x7f5ea40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 -- 172.21.15.17:0/2045147183 shutdown_connections 2024-02-14T15:00:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 -- 172.21.15.17:0/2045147183 wait complete. 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.551+0000 7f5ea92c0640 1 Processor -- start 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea92c0640 1 -- start start 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea92c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea4100160 0x7f5ea4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea92c0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea4101350 0x7f5ea4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea92c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4101d10 0x7f5ea419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea92c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea419ee00 con 0x7f5ea4101d10 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea92c0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea419ef70 con 0x7f5ea4101350 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea92c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea419f130 con 0x7f5ea4100160 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4101d10 0x7f5ea419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4101d10 0x7f5ea419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41308/0 (socket says 172.21.15.17:41308) 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 -- 172.21.15.17:0/1220855348 learned_addr learned my addr 172.21.15.17:0/1220855348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea2ffd640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea4100160 0x7f5ea4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5e9bfff640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea4101350 0x7f5ea4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 -- 172.21.15.17:0/1220855348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea4100160 msgr2=0x7f5ea4197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea4100160 0x7f5ea4197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 -- 172.21.15.17:0/1220855348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea4101350 msgr2=0x7f5ea4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea4101350 0x7f5ea4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 -- 172.21.15.17:0/1220855348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e94009560 con 0x7f5ea4101d10 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea2ffd640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea4100160 0x7f5ea4197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.552+0000 7f5ea37fe640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4101d10 0x7f5ea419e8c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00eaa0 tx=0x7f5e9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.553+0000 7f5ea0ff9640 1 -- 172.21.15.17:0/1220855348 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00cdb0 con 0x7f5ea4101d10 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.553+0000 7f5ea0ff9640 1 -- 172.21.15.17:0/1220855348 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e9c005020 con 0x7f5ea4101d10 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.553+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea419f410 con 0x7f5ea4101d10 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.553+0000 7f5ea0ff9640 1 -- 172.21.15.17:0/1220855348 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0116b0 con 0x7f5ea4101d10 2024-02-14T15:00:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.553+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea419f8d0 con 0x7f5ea4101d10 2024-02-14T15:00:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.554+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea410bbc0 con 0x7f5ea4101d10 2024-02-14T15:00:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.555+0000 7f5ea0ff9640 1 -- 172.21.15.17:0/1220855348 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5e9c005520 con 0x7f5ea4101d10 2024-02-14T15:00:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.555+0000 7f5ea0ff9640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5e900a20c0 0x7f5e900a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.555+0000 7f5ea2ffd640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5e900a20c0 0x7f5e900a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.555+0000 7f5ea0ff9640 1 -- 172.21.15.17:0/1220855348 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5e9c0cd930 con 0x7f5ea4101d10 2024-02-14T15:00:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.556+0000 7f5ea2ffd640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5e900a20c0 0x7f5e900a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5e9402c240 tx=0x7f5e9402c7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:26.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.560+0000 7f5ea0ff9640 1 -- 172.21.15.17:0/1220855348 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e9c096e50 con 0x7f5ea4101d10 2024-02-14T15:00:26.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:26 smithi017 ceph-mon[32549]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:26.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:26.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:26.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.778+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ea419fdd0 con 0x7f5ea4101d10 2024-02-14T15:00:26.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.779+0000 7f5ea0ff9640 1 -- 172.21.15.17:0/1220855348 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5271 (secure 0 0 0) 0x7f5e9c09a510 con 0x7f5ea4101d10 2024-02-14T15:00:26.780 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:26.780 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-14T15:00:26.780 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5e900a20c0 msgr2=0x7f5e900a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5e900a20c0 0x7f5e900a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5e9402c240 tx=0x7f5e9402c7e0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4101d10 msgr2=0x7f5ea419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4101d10 0x7f5ea419e8c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00eaa0 tx=0x7f5e9c00ef60 comp rx=0 tx=0).stop 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 shutdown_connections 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5e900a20c0 0x7f5e900a4580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ea4101d10 0x7f5ea419e8c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ea4101350 0x7f5ea4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 --2- 172.21.15.17:0/1220855348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea4100160 0x7f5ea4197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 >> 172.21.15.17:0/1220855348 conn(0x7f5ea40fb6f0 msgr2=0x7f5ea40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 shutdown_connections 2024-02-14T15:00:26.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:26.781+0000 7f5ea92c0640 1 -- 172.21.15.17:0/1220855348 wait complete. 2024-02-14T15:00:26.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mon[37057]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:26.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.794 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:26.795 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:26.795 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:26 smithi050 ceph-mon[36512]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:27.795 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:27 smithi076 ceph-mon[37057]: Deploying daemon mds.c on smithi076 2024-02-14T15:00:27.796 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:27 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6834/1787917010,v1:172.21.15.76:6835/1787917010] up:boot 2024-02-14T15:00:27.796 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:27 smithi076 ceph-mon[37057]: fsmap 6 up:standby 2024-02-14T15:00:27.796 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:27 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:00:27.796 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:27 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1220855348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:27.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:27 smithi050 ceph-mon[36512]: Deploying daemon mds.c on smithi076 2024-02-14T15:00:27.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:27 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6834/1787917010,v1:172.21.15.76:6835/1787917010] up:boot 2024-02-14T15:00:27.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:27 smithi050 ceph-mon[36512]: fsmap 6 up:standby 2024-02-14T15:00:27.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:27 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:00:27.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:27 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1220855348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:27 smithi017 ceph-mon[32549]: Deploying daemon mds.c on smithi076 2024-02-14T15:00:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:27 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6834/1787917010,v1:172.21.15.76:6835/1787917010] up:boot 2024-02-14T15:00:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:27 smithi017 ceph-mon[32549]: fsmap 6 up:standby 2024-02-14T15:00:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:27 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:00:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:27 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1220855348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:28.484 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 systemd[1]: Starting Ceph mds.c for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:28.768 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:28 smithi076 ceph-mon[37057]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:28.768 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 podman[67586]: 2024-02-14 15:00:28.398053877 +0000 UTC m=+0.017333796 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:28.768 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 podman[67586]: 2024-02-14 15:00:28.514666495 +0000 UTC m=+0.133946459 container create 2323ce9ebc75637b95dbfbb00eb3e808160b8258d826201f4c3edfbaa7948c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-14T15:00:28.769 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 podman[67586]: 2024-02-14 15:00:28.76803817 +0000 UTC m=+0.387318068 container init 2323ce9ebc75637b95dbfbb00eb3e808160b8258d826201f4c3edfbaa7948c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131) 2024-02-14T15:00:28.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:28.821 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:28 smithi017 ceph-mon[32549]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:28.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:28 smithi050 ceph-mon[36512]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.906+0000 7f9811316640 1 -- 172.21.15.17:0/86989191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c104690 msgr2=0x7f980c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.906+0000 7f9811316640 1 --2- 172.21.15.17:0/86989191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c104690 0x7f980c106a80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009900 tx=0x7f97f402bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- 172.21.15.17:0/86989191 shutdown_connections 2024-02-14T15:00:28.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 --2- 172.21.15.17:0/86989191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c104690 0x7f980c106a80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 --2- 172.21.15.17:0/86989191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f980c106fc0 0x7f980c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 --2- 172.21.15.17:0/86989191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f980c101d60 0x7f980c104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- 172.21.15.17:0/86989191 >> 172.21.15.17:0/86989191 conn(0x7f980c0fb6d0 msgr2=0x7f980c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- 172.21.15.17:0/86989191 shutdown_connections 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- 172.21.15.17:0/86989191 wait complete. 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 Processor -- start 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- start start 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c101d60 0x7f980c1016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f980c104690 0x7f980c0ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f980c106fc0 0x7f980c100250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1009b0 con 0x7f980c101d60 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c100b20 con 0x7f980c106fc0 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.907+0000 7f9811316640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1a1330 con 0x7f980c104690 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c101d60 0x7f980c1016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:28.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c101d60 0x7f980c1016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41332/0 (socket says 172.21.15.17:41332) 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 -- 172.21.15.17:0/2441661127 learned_addr learned my addr 172.21.15.17:0/2441661127 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980a7fc640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f980c104690 0x7f980c0ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980b7fe640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f980c106fc0 0x7f980c100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 -- 172.21.15.17:0/2441661127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f980c104690 msgr2=0x7f980c0ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f980c104690 0x7f980c0ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 -- 172.21.15.17:0/2441661127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f980c106fc0 msgr2=0x7f980c100250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f980c106fc0 0x7f980c100250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 -- 172.21.15.17:0/2441661127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4009560 con 0x7f980c101d60 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980b7fe640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f980c106fc0 0x7f980c100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f980affd640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c101d60 0x7f980c1016d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f980400d7a0 tx=0x7f980400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f9803fff640 1 -- 172.21.15.17:0/2441661127 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804009c30 con 0x7f980c101d60 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f9803fff640 1 -- 172.21.15.17:0/2441661127 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f980400deb0 con 0x7f980c101d60 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c1a15e0 con 0x7f980c101d60 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f9803fff640 1 -- 172.21.15.17:0/2441661127 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804004ea0 con 0x7f980c101d60 2024-02-14T15:00:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.908+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c1a1a70 con 0x7f980c101d60 2024-02-14T15:00:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.909+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c0fd1c0 con 0x7f980c101d60 2024-02-14T15:00:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.910+0000 7f9803fff640 1 -- 172.21.15.17:0/2441661127 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9804005020 con 0x7f980c101d60 2024-02-14T15:00:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.911+0000 7f9803fff640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97f80a20c0 0x7f97f80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.911+0000 7f980a7fc640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97f80a20c0 0x7f97f80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.911+0000 7f9803fff640 1 -- 172.21.15.17:0/2441661127 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f98040cd240 con 0x7f980c101d60 2024-02-14T15:00:28.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.911+0000 7f980a7fc640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97f80a20c0 0x7f97f80a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97f4005ab0 tx=0x7f97f4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:28.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:28.916+0000 7f9803fff640 1 -- 172.21.15.17:0/2441661127 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98040966b0 con 0x7f980c101d60 2024-02-14T15:00:29.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.139+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f980c10cc70 con 0x7f980c101d60 2024-02-14T15:00:29.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.139+0000 7f9803fff640 1 -- 172.21.15.17:0/2441661127 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5271 (secure 0 0 0) 0x7f9804099d70 con 0x7f980c101d60 2024-02-14T15:00:29.140 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:29.140 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-14T15:00:29.141 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2024-02-14T15:00:29.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97f80a20c0 msgr2=0x7f97f80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:29.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97f80a20c0 0x7f97f80a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97f4005ab0 tx=0x7f97f4005fd0 comp rx=0 tx=0).stop 2024-02-14T15:00:29.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c101d60 msgr2=0x7f980c1016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:29.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c101d60 0x7f980c1016d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f980400d7a0 tx=0x7f980400dc60 comp rx=0 tx=0).stop 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 shutdown_connections 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f97f80a20c0 0x7f97f80a4580 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f980c106fc0 0x7f980c100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f980c104690 0x7f980c0ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 --2- 172.21.15.17:0/2441661127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c101d60 0x7f980c1016d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 >> 172.21.15.17:0/2441661127 conn(0x7f980c0fb6d0 msgr2=0x7f980c102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 shutdown_connections 2024-02-14T15:00:29.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:29.142+0000 7f9811316640 1 -- 172.21.15.17:0/2441661127 wait complete. 2024-02-14T15:00:29.234 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 podman[67586]: 2024-02-14 15:00:28.77113404 +0000 UTC m=+0.390413952 container start 2323ce9ebc75637b95dbfbb00eb3e808160b8258d826201f4c3edfbaa7948c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True) 2024-02-14T15:00:29.234 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 ceph-mds[67606]: -- 172.21.15.76:0/3129661132 <== mon.1 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564dbed23520 con 0x564dbecf8800 2024-02-14T15:00:29.234 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: starting mds.c at 2024-02-14T15:00:29.234 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 bash[67586]: 2323ce9ebc75637b95dbfbb00eb3e808160b8258d826201f4c3edfbaa7948c9c 2024-02-14T15:00:29.234 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:00:28 smithi076 systemd[1]: Started Ceph mds.c for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: Deploying daemon mds.h on smithi050 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2441661127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:boot 2024-02-14T15:00:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: fsmap 7 up:standby 2024-02-14T15:00:30.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:29 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:00:30.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:30.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: Deploying daemon mds.h on smithi050 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2441661127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:boot 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: fsmap 7 up:standby 2024-02-14T15:00:30.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:29 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:00:30.313 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:30.313 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.313 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: Deploying daemon mds.h on smithi050 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2441661127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:boot 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: fsmap 7 up:standby 2024-02-14T15:00:30.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:29 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:00:31.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 -- 172.21.15.17:0/1913585535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c102570 msgr2=0x7f500c1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 --2- 172.21.15.17:0/1913585535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c102570 0x7f500c1029f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc02bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 -- 172.21.15.17:0/1913585535 shutdown_connections 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 --2- 172.21.15.17:0/1913585535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c102570 0x7f500c1029f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 --2- 172.21.15.17:0/1913585535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f500c102f30 0x7f500c1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 --2- 172.21.15.17:0/1913585535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f500c0ff5a0 0x7f500c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 -- 172.21.15.17:0/1913585535 >> 172.21.15.17:0/1913585535 conn(0x7f500c0faeb0 msgr2=0x7f500c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 -- 172.21.15.17:0/1913585535 shutdown_connections 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.269+0000 7f501293e640 1 -- 172.21.15.17:0/1913585535 wait complete. 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 Processor -- start 2024-02-14T15:00:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 -- start start 2024-02-14T15:00:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c0ff5a0 0x7f500c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f500c102570 0x7f500c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f500c102f30 0x7f500c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c19edb0 con 0x7f500c0ff5a0 2024-02-14T15:00:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c19ef20 con 0x7f500c102570 2024-02-14T15:00:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f501293e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c19f0e0 con 0x7f500c102f30 2024-02-14T15:00:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f500bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c0ff5a0 0x7f500c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f500bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c0ff5a0 0x7f500c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39698/0 (socket says 172.21.15.17:39698) 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f500bfff640 1 -- 172.21.15.17:0/1516478829 learned_addr learned my addr 172.21.15.17:0/1516478829 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.270+0000 7f500b7fe640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f500c102570 0x7f500c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f5010edb640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f500c102f30 0x7f500c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f500bfff640 1 -- 172.21.15.17:0/1516478829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f500c102f30 msgr2=0x7f500c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f500bfff640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f500c102f30 0x7f500c19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f500bfff640 1 -- 172.21.15.17:0/1516478829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f500c102570 msgr2=0x7f500c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f500bfff640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f500c102570 0x7f500c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f500bfff640 1 -- 172.21.15.17:0/1516478829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc009560 con 0x7f500c0ff5a0 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f500bfff640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c0ff5a0 0x7f500c1972c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f500000d950 tx=0x7f500000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f50097fa640 1 -- 172.21.15.17:0/1516478829 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000009c30 con 0x7f500c0ff5a0 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f50097fa640 1 -- 172.21.15.17:0/1516478829 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5000009d90 con 0x7f500c0ff5a0 2024-02-14T15:00:31.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c19f3c0 con 0x7f500c0ff5a0 2024-02-14T15:00:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f50097fa640 1 -- 172.21.15.17:0/1516478829 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500000f720 con 0x7f500c0ff5a0 2024-02-14T15:00:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.271+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c19f8a0 con 0x7f500c0ff5a0 2024-02-14T15:00:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.272+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c108640 con 0x7f500c0ff5a0 2024-02-14T15:00:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.273+0000 7f50097fa640 1 -- 172.21.15.17:0/1516478829 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5000010a60 con 0x7f500c0ff5a0 2024-02-14T15:00:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.274+0000 7f50097fa640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4ff40a2140 0x7f4ff40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.274+0000 7f500b7fe640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4ff40a2140 0x7f4ff40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.274+0000 7f50097fa640 1 -- 172.21.15.17:0/1516478829 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f50000cd3f0 con 0x7f500c0ff5a0 2024-02-14T15:00:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.274+0000 7f500b7fe640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4ff40a2140 0x7f4ff40a4600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009230 tx=0x7f4ffc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.278+0000 7f50097fa640 1 -- 172.21.15.17:0/1516478829 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5000096990 con 0x7f500c0ff5a0 2024-02-14T15:00:31.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 systemd[1]: Starting Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:31.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.496+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f500c10cdb0 con 0x7f500c0ff5a0 2024-02-14T15:00:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.497+0000 7f50097fa640 1 -- 172.21.15.17:0/1516478829 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6062 (secure 0 0 0) 0x7f5000017090 con 0x7f500c0ff5a0 2024-02-14T15:00:31.498 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:31.498 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-14T15:00:31.498 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4ff40a2140 msgr2=0x7f4ff40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4ff40a2140 0x7f4ff40a4600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009230 tx=0x7f4ffc005fd0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c0ff5a0 msgr2=0x7f500c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c0ff5a0 0x7f500c1972c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f500000d950 tx=0x7f500000de10 comp rx=0 tx=0).stop 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 shutdown_connections 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4ff40a2140 0x7f4ff40a4600 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f500c102f30 0x7f500c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f500c102570 0x7f500c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 --2- 172.21.15.17:0/1516478829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f500c0ff5a0 0x7f500c1972c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:31.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 >> 172.21.15.17:0/1516478829 conn(0x7f500c0faeb0 msgr2=0x7f500c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:31.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 shutdown_connections 2024-02-14T15:00:31.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:31.499+0000 7f501293e640 1 -- 172.21.15.17:0/1516478829 wait complete. 2024-02-14T15:00:31.730 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 podman[66882]: 2024-02-14 15:00:31.366865599 +0000 UTC m=+0.018390507 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:31.730 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 podman[66882]: 2024-02-14 15:00:31.484043133 +0000 UTC m=+0.135568027 container create 62c016a31def367d2ea291198dba761de1da3779c686af7340a1bdd0390f4038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-14T15:00:31.730 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 podman[66882]: 2024-02-14 15:00:31.703422741 +0000 UTC m=+0.354947637 container init 62c016a31def367d2ea291198dba761de1da3779c686af7340a1bdd0390f4038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS) 2024-02-14T15:00:31.731 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 podman[66882]: 2024-02-14 15:00:31.706284096 +0000 UTC m=+0.357808997 container start 62c016a31def367d2ea291198dba761de1da3779c686af7340a1bdd0390f4038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.build-date=20240131, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:00:31.731 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:31 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1516478829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:31.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:31 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1516478829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:31.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:31 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1516478829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:32.142 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 ceph-mds[66901]: -- 172.21.15.50:0/588553466 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ebf38bd520 con 0x55ebf3892800 2024-02-14T15:00:32.142 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: starting mds.h at 2024-02-14T15:00:32.142 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 bash[66882]: 62c016a31def367d2ea291198dba761de1da3779c686af7340a1bdd0390f4038 2024-02-14T15:00:32.142 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:00:31 smithi050 systemd[1]: Started Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:32.790 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:32 smithi076 ceph-mon[37057]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:32.790 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.790 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.790 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.790 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:32.790 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:32.790 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:32 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:32.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:32 smithi050 ceph-mon[36512]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:32.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:32.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:32.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:32 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:32 smithi017 ceph-mon[32549]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:32.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:32 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:33.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 -- 172.21.15.17:0/2473947556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 msgr2=0x7f37bc100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/2473947556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc100580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009900 tx=0x7f37b002bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 -- 172.21.15.17:0/2473947556 shutdown_connections 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/2473947556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37bc101d10 0x7f37bc1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/2473947556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37bc101350 0x7f37bc1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/2473947556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc100580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 -- 172.21.15.17:0/2473947556 >> 172.21.15.17:0/2473947556 conn(0x7f37bc0fb6d0 msgr2=0x7f37bc0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 -- 172.21.15.17:0/2473947556 shutdown_connections 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.617+0000 7f37c2bc9640 1 -- 172.21.15.17:0/2473947556 wait complete. 2024-02-14T15:00:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 Processor -- start 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 -- start start 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc069070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37bc101350 0x7f37bc19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37bc101d10 0x7f37bc1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc1a11a0 con 0x7f37bc100160 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc1a1310 con 0x7f37bc101350 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c2bc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc1a14d0 con 0x7f37bc101d10 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc069070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39718/0 (socket says 172.21.15.17:39718) 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 -- 172.21.15.17:0/3412226509 learned_addr learned my addr 172.21.15.17:0/3412226509 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c1166640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37bc101d10 0x7f37bc1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37bbfff640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37bc101350 0x7f37bc19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 -- 172.21.15.17:0/3412226509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37bc101d10 msgr2=0x7f37bc1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37bc101d10 0x7f37bc1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 -- 172.21.15.17:0/3412226509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37bc101350 msgr2=0x7f37bc19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37bc101350 0x7f37bc19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 -- 172.21.15.17:0/3412226509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b0009560 con 0x7f37bc100160 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37bbfff640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37bc101350 0x7f37bc19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c1166640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37bc101d10 0x7f37bc1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.618+0000 7f37c0965640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc069070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f37b0036040 tx=0x7f37b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.619+0000 7f37b9ffb640 1 -- 172.21.15.17:0/3412226509 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00044e0 con 0x7f37bc100160 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.619+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc1a16f0 con 0x7f37bc100160 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.619+0000 7f37b9ffb640 1 -- 172.21.15.17:0/3412226509 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37b00054e0 con 0x7f37bc100160 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.619+0000 7f37b9ffb640 1 -- 172.21.15.17:0/3412226509 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b003e450 con 0x7f37bc100160 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.619+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1a1c80 con 0x7f37bc100160 2024-02-14T15:00:33.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.620+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc1074f0 con 0x7f37bc100160 2024-02-14T15:00:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.621+0000 7f37b9ffb640 1 -- 172.21.15.17:0/3412226509 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f37b0003710 con 0x7f37bc100160 2024-02-14T15:00:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.622+0000 7f37b9ffb640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37ac0a20f0 0x7f37ac0a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.622+0000 7f37bbfff640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37ac0a20f0 0x7f37ac0a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.622+0000 7f37b9ffb640 1 -- 172.21.15.17:0/3412226509 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f37b00f0290 con 0x7f37bc100160 2024-02-14T15:00:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.622+0000 7f37bbfff640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37ac0a20f0 0x7f37ac0a45b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006fd0 tx=0x7f37a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:33.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.629+0000 7f37b9ffb640 1 -- 172.21.15.17:0/3412226509 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37b00b9700 con 0x7f37bc100160 2024-02-14T15:00:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:33 smithi076 ceph-mon[37057]: Deploying daemon mds.i on smithi076 2024-02-14T15:00:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:33 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6836/1706654420,v1:172.21.15.50:6837/1706654420] up:boot 2024-02-14T15:00:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:33 smithi076 ceph-mon[37057]: fsmap 8 up:standby 2024-02-14T15:00:33.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:33 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:00:33.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.848+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37bc10bc30 con 0x7f37bc100160 2024-02-14T15:00:33.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.849+0000 7f37b9ffb640 1 -- 172.21.15.17:0/3412226509 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6853 (secure 0 0 0) 0x7f37b00bcdc0 con 0x7f37bc100160 2024-02-14T15:00:33.850 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:33.850 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-14T15:00:33.850 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37ac0a20f0 msgr2=0x7f37ac0a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37ac0a20f0 0x7f37ac0a45b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006fd0 tx=0x7f37a8008040 comp rx=0 tx=0).stop 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 msgr2=0x7f37bc069070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc069070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f37b0036040 tx=0x7f37b00042f0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 shutdown_connections 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37ac0a20f0 0x7f37ac0a45b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37bc101d10 0x7f37bc1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37bc101350 0x7f37bc19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 --2- 172.21.15.17:0/3412226509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37bc100160 0x7f37bc069070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:33.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 >> 172.21.15.17:0/3412226509 conn(0x7f37bc0fb6d0 msgr2=0x7f37bc1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:33.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 shutdown_connections 2024-02-14T15:00:33.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:33.851+0000 7f37c2bc9640 1 -- 172.21.15.17:0/3412226509 wait complete. 2024-02-14T15:00:33.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:33 smithi017 ceph-mon[32549]: Deploying daemon mds.i on smithi076 2024-02-14T15:00:33.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:33 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6836/1706654420,v1:172.21.15.50:6837/1706654420] up:boot 2024-02-14T15:00:33.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:33 smithi017 ceph-mon[32549]: fsmap 8 up:standby 2024-02-14T15:00:33.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:33 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:00:33.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:33 smithi050 ceph-mon[36512]: Deploying daemon mds.i on smithi076 2024-02-14T15:00:33.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:33 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6836/1706654420,v1:172.21.15.50:6837/1706654420] up:boot 2024-02-14T15:00:33.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:33 smithi050 ceph-mon[36512]: fsmap 8 up:standby 2024-02-14T15:00:33.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:33 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:00:34.158 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:33 smithi076 systemd[1]: Starting Ceph mds.i for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:34.158 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 podman[67894]: 2024-02-14 15:00:34.059557744 +0000 UTC m=+0.017304796 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:34.435 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 podman[67894]: 2024-02-14 15:00:34.16531036 +0000 UTC m=+0.123057397 container create 4a5faa0da1a299c92d6c186571cf0af68ef77b8e369a4f521f81b8b0f745e010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-14T15:00:34.435 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 podman[67894]: 2024-02-14 15:00:34.435355846 +0000 UTC m=+0.393102907 container init 4a5faa0da1a299c92d6c186571cf0af68ef77b8e369a4f521f81b8b0f745e010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-14T15:00:34.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:34 smithi050 ceph-mon[36512]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:34.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:34 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3412226509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:34.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:34 smithi076 ceph-mon[37057]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:34.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:34 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3412226509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:34.734 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 podman[67894]: 2024-02-14 15:00:34.437943869 +0000 UTC m=+0.395690909 container start 4a5faa0da1a299c92d6c186571cf0af68ef77b8e369a4f521f81b8b0f745e010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:00:34.734 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 ceph-mds[67913]: -- 172.21.15.76:0/3206168298 <== mon.1 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563004923520 con 0x5630048f8c00 2024-02-14T15:00:34.734 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i[67909]: starting mds.i at 2024-02-14T15:00:34.734 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 bash[67894]: 4a5faa0da1a299c92d6c186571cf0af68ef77b8e369a4f521f81b8b0f745e010 2024-02-14T15:00:34.734 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:00:34 smithi076 systemd[1]: Started Ceph mds.i for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:34 smithi017 ceph-mon[32549]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:34 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3412226509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:35.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] up:boot 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: fsmap 9 up:standby 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:35.891 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:35 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] up:boot 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: fsmap 9 up:standby 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:35.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:35.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:35 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:35.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 -- 172.21.15.17:0/4074502745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f8102e70 msgr2=0x7fb4f8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:35.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 --2- 172.21.15.17:0/4074502745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f8102e70 0x7fb4f8109700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009ca0 tx=0x7fb4f002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:35.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 -- 172.21.15.17:0/4074502745 shutdown_connections 2024-02-14T15:00:35.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 --2- 172.21.15.17:0/4074502745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f8102e70 0x7fb4f8109700 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 --2- 172.21.15.17:0/4074502745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4f81024b0 0x7fb4f8102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 --2- 172.21.15.17:0/4074502745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f81012c0 0x7fb4f81016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 -- 172.21.15.17:0/4074502745 >> 172.21.15.17:0/4074502745 conn(0x7fb4f80fc850 msgr2=0x7fb4f80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 -- 172.21.15.17:0/4074502745 shutdown_connections 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.975+0000 7fb4febe2640 1 -- 172.21.15.17:0/4074502745 wait complete. 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 Processor -- start 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 -- start start 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f81012c0 0x7fb4f8074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f81024b0 0x7fb4f8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4f8102e70 0x7fb4f8073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80742c0 con 0x7fb4f81012c0 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f8074430 con 0x7fb4f8102e70 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4febe2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a12b0 con 0x7fb4f81024b0 2024-02-14T15:00:35.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f81012c0 0x7fb4f8074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f81012c0 0x7fb4f8074fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39744/0 (socket says 172.21.15.17:39744) 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 -- 172.21.15.17:0/3001862214 learned_addr learned my addr 172.21.15.17:0/3001862214 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fd17f640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4f8102e70 0x7fb4f8073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4f7fff640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f81024b0 0x7fb4f8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 -- 172.21.15.17:0/3001862214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f81024b0 msgr2=0x7fb4f8073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f81024b0 0x7fb4f8073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 -- 172.21.15.17:0/3001862214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4f8102e70 msgr2=0x7fb4f8073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4f8102e70 0x7fb4f8073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 -- 172.21.15.17:0/3001862214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009900 con 0x7fb4f81012c0 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4f7fff640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f81024b0 0x7fb4f8073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.976+0000 7fb4fc97e640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f81012c0 0x7fb4f8074fe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800d980 tx=0x7fb4e800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.977+0000 7fb4f5ffb640 1 -- 172.21.15.17:0/3001862214 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8009c30 con 0x7fb4f81012c0 2024-02-14T15:00:35.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.977+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f81a1560 con 0x7fb4f81012c0 2024-02-14T15:00:35.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.977+0000 7fb4f5ffb640 1 -- 172.21.15.17:0/3001862214 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4e8009d90 con 0x7fb4f81012c0 2024-02-14T15:00:35.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.977+0000 7fb4f5ffb640 1 -- 172.21.15.17:0/3001862214 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800f6b0 con 0x7fb4f81012c0 2024-02-14T15:00:35.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.977+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f81a1a70 con 0x7fb4f81012c0 2024-02-14T15:00:35.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.978+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f81085e0 con 0x7fb4f81012c0 2024-02-14T15:00:35.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.979+0000 7fb4f5ffb640 1 -- 172.21.15.17:0/3001862214 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb4e8010460 con 0x7fb4f81012c0 2024-02-14T15:00:35.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.979+0000 7fb4f5ffb640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb4e40a20c0 0x7fb4e40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:35.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.980+0000 7fb4f7fff640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb4e40a20c0 0x7fb4e40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:35.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.980+0000 7fb4f5ffb640 1 -- 172.21.15.17:0/3001862214 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb4e80cd740 con 0x7fb4f81012c0 2024-02-14T15:00:35.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.980+0000 7fb4f7fff640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb4e40a20c0 0x7fb4e40a4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005d00 tx=0x7fb4e0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] up:boot 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: fsmap 9 up:standby 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:35 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:35.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:35.984+0000 7fb4f5ffb640 1 -- 172.21.15.17:0/3001862214 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4e8096b10 con 0x7fb4f81012c0 2024-02-14T15:00:36.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.205+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4f802ce80 con 0x7fb4f81012c0 2024-02-14T15:00:36.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.206+0000 7fb4f5ffb640 1 -- 172.21.15.17:0/3001862214 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7643 (secure 0 0 0) 0x7fb4e809a1d0 con 0x7fb4f81012c0 2024-02-14T15:00:36.207 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:36.207 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-14T15:00:36.207 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-02-14T15:00:36.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb4e40a20c0 msgr2=0x7fb4e40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:36.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb4e40a20c0 0x7fb4e40a4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005d00 tx=0x7fb4e0009550 comp rx=0 tx=0).stop 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f81012c0 msgr2=0x7fb4f8074fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f81012c0 0x7fb4f8074fe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800d980 tx=0x7fb4e800de40 comp rx=0 tx=0).stop 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 shutdown_connections 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb4e40a20c0 0x7fb4e40a4580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4f8102e70 0x7fb4f8073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4f81024b0 0x7fb4f8073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 --2- 172.21.15.17:0/3001862214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f81012c0 0x7fb4f8074fe0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.208+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 >> 172.21.15.17:0/3001862214 conn(0x7fb4f80fc850 msgr2=0x7fb4f81057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.209+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 shutdown_connections 2024-02-14T15:00:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:36.209+0000 7fb4febe2640 1 -- 172.21.15.17:0/3001862214 wait complete. 2024-02-14T15:00:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:36 smithi050 ceph-mon[36512]: Deploying daemon mds.g on smithi017 2024-02-14T15:00:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:36 smithi050 ceph-mon[36512]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:36 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3001862214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:36 smithi076 ceph-mon[37057]: Deploying daemon mds.g on smithi017 2024-02-14T15:00:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:36 smithi076 ceph-mon[37057]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:36 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3001862214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:36.988 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:36 smithi017 ceph-mon[32549]: Deploying daemon mds.g on smithi017 2024-02-14T15:00:36.988 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:36 smithi017 ceph-mon[32549]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:36.988 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:36 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3001862214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:36.988 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:36 smithi017 systemd[1]: Starting Ceph mds.g for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:37.266 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:36 smithi017 podman[73796]: 2024-02-14 15:00:36.89034511 +0000 UTC m=+0.017782009 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:37.266 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:37 smithi017 podman[73796]: 2024-02-14 15:00:37.068566893 +0000 UTC m=+0.196003822 container create f8d1ceeaf43d4716689b0a1df871ba624495cb747b129383f895144b3df9df64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-14T15:00:37.766 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:37 smithi017 podman[73796]: 2024-02-14 15:00:37.304962929 +0000 UTC m=+0.432399813 container init f8d1ceeaf43d4716689b0a1df871ba624495cb747b129383f895144b3df9df64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-14T15:00:37.766 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:37 smithi017 podman[73796]: 2024-02-14 15:00:37.307582469 +0000 UTC m=+0.435019352 container start f8d1ceeaf43d4716689b0a1df871ba624495cb747b129383f895144b3df9df64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:00:37.767 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:37 smithi017 ceph-mds[73815]: -- 172.21.15.17:0/1433828544 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1d4747520 con 0x55d1d471d000 2024-02-14T15:00:37.767 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g[73811]: starting mds.g at 2024-02-14T15:00:37.767 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:37 smithi017 bash[73796]: f8d1ceeaf43d4716689b0a1df871ba624495cb747b129383f895144b3df9df64 2024-02-14T15:00:37.767 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:00:37 smithi017 systemd[1]: Started Ceph mds.g for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:38.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.337+0000 7fccd255c640 1 -- 172.21.15.17:0/861350613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc101d10 msgr2=0x7fcccc1085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.337+0000 7fccd255c640 1 --2- 172.21.15.17:0/861350613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc101d10 0x7fcccc1085a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009ca0 tx=0x7fccc402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.337+0000 7fccd255c640 1 -- 172.21.15.17:0/861350613 shutdown_connections 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.337+0000 7fccd255c640 1 --2- 172.21.15.17:0/861350613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc101d10 0x7fcccc1085a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.337+0000 7fccd255c640 1 --2- 172.21.15.17:0/861350613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcccc101350 0x7fcccc1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.337+0000 7fccd255c640 1 --2- 172.21.15.17:0/861350613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcccc100160 0x7fcccc100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.337+0000 7fccd255c640 1 -- 172.21.15.17:0/861350613 >> 172.21.15.17:0/861350613 conn(0x7fcccc0fb6d0 msgr2=0x7fcccc0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 -- 172.21.15.17:0/861350613 shutdown_connections 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 -- 172.21.15.17:0/861350613 wait complete. 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 Processor -- start 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 -- start start 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc100160 0x7fcccc069070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcccc101350 0x7fcccc19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcccc101d10 0x7fcccc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc19ed90 con 0x7fcccc100160 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc19ef00 con 0x7fcccc101350 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fccd255c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc19f0c0 con 0x7fcccc101d10 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fcccbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc100160 0x7fcccc069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fcccbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc100160 0x7fcccc069070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39788/0 (socket says 172.21.15.17:39788) 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.338+0000 7fcccbfff640 1 -- 172.21.15.17:0/267736547 learned_addr learned my addr 172.21.15.17:0/267736547 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fccd0af9640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcccc101d10 0x7fcccc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fcccbfff640 1 -- 172.21.15.17:0/267736547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcccc101d10 msgr2=0x7fcccc19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fcccbfff640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcccc101d10 0x7fcccc19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fcccbfff640 1 -- 172.21.15.17:0/267736547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcccc101350 msgr2=0x7fcccc19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fcccbfff640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcccc101350 0x7fcccc19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fcccbfff640 1 -- 172.21.15.17:0/267736547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4009900 con 0x7fcccc100160 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fcccbfff640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc100160 0x7fcccc069070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c970 tx=0x7fccbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fccc97fa640 1 -- 172.21.15.17:0/267736547 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc004220 con 0x7fcccc100160 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fccc97fa640 1 -- 172.21.15.17:0/267736547 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccbc007e90 con 0x7fcccc100160 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc19f340 con 0x7fcccc100160 2024-02-14T15:00:38.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fccc97fa640 1 -- 172.21.15.17:0/267736547 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00f450 con 0x7fcccc100160 2024-02-14T15:00:38.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.339+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc19f850 con 0x7fcccc100160 2024-02-14T15:00:38.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.340+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc107480 con 0x7fcccc100160 2024-02-14T15:00:38.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.341+0000 7fccc97fa640 1 -- 172.21.15.17:0/267736547 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fccbc003610 con 0x7fcccc100160 2024-02-14T15:00:38.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.342+0000 7fccc97fa640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fccb40a20c0 0x7fccb40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:38.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.342+0000 7fcccb7fe640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fccb40a20c0 0x7fccb40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:38.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.342+0000 7fccc97fa640 1 -- 172.21.15.17:0/267736547 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fccbc0cd620 con 0x7fcccc100160 2024-02-14T15:00:38.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.342+0000 7fcccb7fe640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fccb40a20c0 0x7fccb40a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcccc068de0 tx=0x7fccc0005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:38.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.346+0000 7fccc97fa640 1 -- 172.21.15.17:0/267736547 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fccbc0969f0 con 0x7fcccc100160 2024-02-14T15:00:38.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.572+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcccc02ceb0 con 0x7fcccc100160 2024-02-14T15:00:38.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.573+0000 7fccc97fa640 1 -- 172.21.15.17:0/267736547 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8432 (secure 0 0 0) 0x7fccbc09a0b0 con 0x7fcccc100160 2024-02-14T15:00:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:38.574 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-14T15:00:38.574 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 11 2024-02-14T15:00:38.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fccb40a20c0 msgr2=0x7fccb40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:38.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fccb40a20c0 0x7fccb40a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcccc068de0 tx=0x7fccc0005bf0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc100160 msgr2=0x7fcccc069070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:38.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc100160 0x7fcccc069070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c970 tx=0x7fccbc00ce30 comp rx=0 tx=0).stop 2024-02-14T15:00:38.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 shutdown_connections 2024-02-14T15:00:38.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fccb40a20c0 0x7fccb40a4580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcccc101d10 0x7fcccc19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcccc101350 0x7fcccc19bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 --2- 172.21.15.17:0/267736547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc100160 0x7fcccc069070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 >> 172.21.15.17:0/267736547 conn(0x7fcccc0fb6d0 msgr2=0x7fcccc104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 shutdown_connections 2024-02-14T15:00:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:38.575+0000 7fccd255c640 1 -- 172.21.15.17:0/267736547 wait complete. 2024-02-14T15:00:38.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:38.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:38.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:38.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:38.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: Deploying daemon mds.a on smithi017 2024-02-14T15:00:38.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.17:6838/516562430,v1:172.21.15.17:6839/516562430] up:boot 2024-02-14T15:00:38.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: fsmap 10 up:standby 2024-02-14T15:00:38.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:38 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: Deploying daemon mds.a on smithi017 2024-02-14T15:00:38.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.17:6838/516562430,v1:172.21.15.17:6839/516562430] up:boot 2024-02-14T15:00:38.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: fsmap 10 up:standby 2024-02-14T15:00:38.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:38 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-14T15:00:38.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:38.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:38.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:38.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:38.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:38.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: Deploying daemon mds.a on smithi017 2024-02-14T15:00:38.898 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.17:6838/516562430,v1:172.21.15.17:6839/516562430] up:boot 2024-02-14T15:00:38.898 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: fsmap 10 up:standby 2024-02-14T15:00:38.898 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:38 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-14T15:00:39.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:39 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/267736547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:39.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:39 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/267736547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:39.766 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:39 smithi017 systemd[1]: Starting Ceph mds.a for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:00:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:39 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/267736547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:40.068 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:39 smithi017 podman[74145]: 2024-02-14 15:00:39.718340385 +0000 UTC m=+0.018504785 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:00:40.068 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:39 smithi017 podman[74145]: 2024-02-14 15:00:39.823807557 +0000 UTC m=+0.123971943 container create 5695e59a76e79d52b60b3d9b1865a1c6019bdfc0bad79085a25197c5acb3f485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131) 2024-02-14T15:00:40.068 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:40 smithi017 podman[74145]: 2024-02-14 15:00:40.068026978 +0000 UTC m=+0.368191364 container init 5695e59a76e79d52b60b3d9b1865a1c6019bdfc0bad79085a25197c5acb3f485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-14T15:00:40.462 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:40 smithi017 podman[74145]: 2024-02-14 15:00:40.07057235 +0000 UTC m=+0.370736739 container start 5695e59a76e79d52b60b3d9b1865a1c6019bdfc0bad79085a25197c5acb3f485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2) 2024-02-14T15:00:40.462 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mds[74164]: -- 172.21.15.17:0/2729027665 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc733a7520 con 0x55bc7337cc00 2024-02-14T15:00:40.462 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a[74160]: starting mds.a at 2024-02-14T15:00:40.463 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:40 smithi017 bash[74145]: 5695e59a76e79d52b60b3d9b1865a1c6019bdfc0bad79085a25197c5acb3f485 2024-02-14T15:00:40.463 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:00:40 smithi017 systemd[1]: Started Ceph mds.a for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:40.588 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 -- 172.21.15.17:0/946793614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102450 msgr2=0x7f536c1028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 --2- 172.21.15.17:0/946793614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102450 0x7f536c1028d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f535c009900 tx=0x7f535c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 -- 172.21.15.17:0/946793614 shutdown_connections 2024-02-14T15:00:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 --2- 172.21.15.17:0/946793614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102450 0x7f536c1028d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 --2- 172.21.15.17:0/946793614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c102e10 0x7f536c1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 --2- 172.21.15.17:0/946793614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c101260 0x7f536c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 -- 172.21.15.17:0/946793614 >> 172.21.15.17:0/946793614 conn(0x7f536c0fc810 msgr2=0x7f536c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 -- 172.21.15.17:0/946793614 shutdown_connections 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.696+0000 7f53747a1640 1 -- 172.21.15.17:0/946793614 wait complete. 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 Processor -- start 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 -- start start 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c101260 0x7f536c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c102450 0x7f536c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102e10 0x7f536c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c19ed90 con 0x7f536c102e10 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c19ef00 con 0x7f536c101260 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f53747a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c19f0c0 con 0x7f536c102450 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102e10 0x7f536c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102e10 0x7f536c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39220/0 (socket says 172.21.15.17:39220) 2024-02-14T15:00:40.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 -- 172.21.15.17:0/2630749059 learned_addr learned my addr 172.21.15.17:0/2630749059 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f537253d640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c101260 0x7f536c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5371d3c640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c102450 0x7f536c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 -- 172.21.15.17:0/2630749059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c102450 msgr2=0x7f536c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c102450 0x7f536c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 -- 172.21.15.17:0/2630749059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c101260 msgr2=0x7f536c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c101260 0x7f536c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 -- 172.21.15.17:0/2630749059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c009560 con 0x7f536c102e10 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5371d3c640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c102450 0x7f536c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.697+0000 7f5372d3e640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102e10 0x7f536c19e850 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f536400eaa0 tx=0x7f536400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.698+0000 7f53637fe640 1 -- 172.21.15.17:0/2630749059 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536400cdb0 con 0x7f536c102e10 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.698+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c19f3a0 con 0x7f536c102e10 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.698+0000 7f53637fe640 1 -- 172.21.15.17:0/2630749059 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5364005020 con 0x7f536c102e10 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.698+0000 7f53637fe640 1 -- 172.21.15.17:0/2630749059 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53640116b0 con 0x7f536c102e10 2024-02-14T15:00:40.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.698+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c19f8b0 con 0x7f536c102e10 2024-02-14T15:00:40.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.699+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c108580 con 0x7f536c102e10 2024-02-14T15:00:40.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.700+0000 7f53637fe640 1 -- 172.21.15.17:0/2630749059 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5364005520 con 0x7f536c102e10 2024-02-14T15:00:40.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.700+0000 7f53637fe640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f53580a20c0 0x7f53580a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:40.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.700+0000 7f537253d640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f53580a20c0 0x7f53580a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:40.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.700+0000 7f53637fe640 1 -- 172.21.15.17:0/2630749059 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f53640cda20 con 0x7f536c102e10 2024-02-14T15:00:40.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.701+0000 7f537253d640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f53580a20c0 0x7f53580a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f536c1022b0 tx=0x7f5368005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:40.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.705+0000 7f53637fe640 1 -- 172.21.15.17:0/2630749059 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5364096e90 con 0x7f536c102e10 2024-02-14T15:00:40.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:40 smithi076 ceph-mon[37057]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:40.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:40.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:40.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:40 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mon[32549]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:40.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:40 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:40 smithi050 ceph-mon[36512]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-14T15:00:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-14T15:00:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:40 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:40.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.924+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f536c10cc70 con 0x7f536c102e10 2024-02-14T15:00:40.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.925+0000 7f53637fe640 1 -- 172.21.15.17:0/2630749059 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9224 (secure 0 0 0) 0x7f536409a550 con 0x7f536c102e10 2024-02-14T15:00:40.925 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:40.926 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-14T15:00:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 12 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.926+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f53580a20c0 msgr2=0x7f53580a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.926+0000 7f53747a1640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f53580a20c0 0x7f53580a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f536c1022b0 tx=0x7f5368005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.926+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102e10 msgr2=0x7f536c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.926+0000 7f53747a1640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102e10 0x7f536c19e850 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f536400eaa0 tx=0x7f536400ef60 comp rx=0 tx=0).stop 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 shutdown_connections 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f53580a20c0 0x7f53580a4580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f536c102e10 0x7f536c19e850 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f536c102450 0x7f536c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 --2- 172.21.15.17:0/2630749059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f536c101260 0x7f536c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 >> 172.21.15.17:0/2630749059 conn(0x7f536c0fc810 msgr2=0x7f536c105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 shutdown_connections 2024-02-14T15:00:40.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:40.927+0000 7f53747a1640 1 -- 172.21.15.17:0/2630749059 wait complete. 2024-02-14T15:00:41.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:41 smithi050 ceph-mon[36512]: Deploying daemon mds.b on smithi050 2024-02-14T15:00:41.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:41 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.17:6840/3297312936,v1:172.21.15.17:6841/3297312936] up:boot 2024-02-14T15:00:41.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:41 smithi050 ceph-mon[36512]: fsmap 11 up:standby 2024-02-14T15:00:41.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:41 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-14T15:00:41.612 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:41 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2630749059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:41.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:41 smithi076 ceph-mon[37057]: Deploying daemon mds.b on smithi050 2024-02-14T15:00:41.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:41 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.17:6840/3297312936,v1:172.21.15.17:6841/3297312936] up:boot 2024-02-14T15:00:41.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:41 smithi076 ceph-mon[37057]: fsmap 11 up:standby 2024-02-14T15:00:41.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:41 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-14T15:00:41.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:41 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2630749059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:41 smithi017 ceph-mon[32549]: Deploying daemon mds.b on smithi050 2024-02-14T15:00:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:41 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.17:6840/3297312936,v1:172.21.15.17:6841/3297312936] up:boot 2024-02-14T15:00:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:41 smithi017 ceph-mon[32549]: fsmap 11 up:standby 2024-02-14T15:00:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:41 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-14T15:00:41.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:41 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2630749059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:42.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:42 smithi050 ceph-mon[36512]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:42.939 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:42.968 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:42 smithi017 ceph-mon[32549]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:42.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:42 smithi076 ceph-mon[37057]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:43.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 -- 172.21.15.17:0/569972131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d241046d0 msgr2=0x7f3d24104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:43.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 --2- 172.21.15.17:0/569972131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d241046d0 0x7f3d24104b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009900 tx=0x7f3d1802bc50 comp rx=0 tx=0).stop 2024-02-14T15:00:43.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 -- 172.21.15.17:0/569972131 shutdown_connections 2024-02-14T15:00:43.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 --2- 172.21.15.17:0/569972131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d241046d0 0x7f3d24104b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 --2- 172.21.15.17:0/569972131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d24105090 0x7f3d2410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 --2- 172.21.15.17:0/569972131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d241034e0 0x7f3d24103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 -- 172.21.15.17:0/569972131 >> 172.21.15.17:0/569972131 conn(0x7f3d240fea90 msgr2=0x7f3d24100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:43.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 -- 172.21.15.17:0/569972131 shutdown_connections 2024-02-14T15:00:43.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.053+0000 7f3d2a815640 1 -- 172.21.15.17:0/569972131 wait complete. 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 Processor -- start 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 -- start start 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d241034e0 0x7f3d24199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d241046d0 0x7f3d24199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105090 0x7f3d241a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241a1080 con 0x7f3d24105090 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241a11f0 con 0x7f3d241034e0 2024-02-14T15:00:43.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.054+0000 7f3d2a815640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241a13b0 con 0x7f3d241046d0 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105090 0x7f3d241a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105090 0x7f3d241a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39242/0 (socket says 172.21.15.17:39242) 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 -- 172.21.15.17:0/2475646538 learned_addr learned my addr 172.21.15.17:0/2475646538 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d237fe640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d241046d0 0x7f3d24199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 -- 172.21.15.17:0/2475646538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d241046d0 msgr2=0x7f3d24199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d241046d0 0x7f3d24199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 -- 172.21.15.17:0/2475646538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d241034e0 msgr2=0x7f3d24199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d241034e0 0x7f3d24199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 -- 172.21.15.17:0/2475646538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d18009560 con 0x7f3d24105090 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d28db2640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105090 0x7f3d241a0b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00eaa0 tx=0x7f3d1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d217fa640 1 -- 172.21.15.17:0/2475646538 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00cdb0 con 0x7f3d24105090 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d217fa640 1 -- 172.21.15.17:0/2475646538 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d1c005020 con 0x7f3d24105090 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d241a1690 con 0x7f3d24105090 2024-02-14T15:00:43.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d217fa640 1 -- 172.21.15.17:0/2475646538 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0116b0 con 0x7f3d24105090 2024-02-14T15:00:43.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.055+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241a1bf0 con 0x7f3d24105090 2024-02-14T15:00:43.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.056+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2410a7a0 con 0x7f3d24105090 2024-02-14T15:00:43.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.057+0000 7f3d217fa640 1 -- 172.21.15.17:0/2475646538 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3d1c005520 con 0x7f3d24105090 2024-02-14T15:00:43.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.058+0000 7f3d217fa640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3d0c0a20c0 0x7f3d0c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:43.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.058+0000 7f3d23fff640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3d0c0a20c0 0x7f3d0c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.058+0000 7f3d217fa640 1 -- 172.21.15.17:0/2475646538 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3d1c0cda20 con 0x7f3d24105090 2024-02-14T15:00:43.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.059+0000 7f3d23fff640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3d0c0a20c0 0x7f3d0c0a4580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d24104530 tx=0x7f3d14006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:43.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.063+0000 7f3d217fa640 1 -- 172.21.15.17:0/2475646538 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3d1c096e90 con 0x7f3d24105090 2024-02-14T15:00:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.288+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d2410ef50 con 0x7f3d24105090 2024-02-14T15:00:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.289+0000 7f3d217fa640 1 -- 172.21.15.17:0/2475646538 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9224 (secure 0 0 0) 0x7f3d1c09a550 con 0x7f3d24105090 2024-02-14T15:00:43.289 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:43.290 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-14T15:00:43.290 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 12 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3d0c0a20c0 msgr2=0x7f3d0c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3d0c0a20c0 0x7f3d0c0a4580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d24104530 tx=0x7f3d14006c40 comp rx=0 tx=0).stop 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105090 msgr2=0x7f3d241a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105090 0x7f3d241a0b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00eaa0 tx=0x7f3d1c00ef60 comp rx=0 tx=0).stop 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 shutdown_connections 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3d0c0a20c0 0x7f3d0c0a4580 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d24105090 0x7f3d241a0b40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d241046d0 0x7f3d24199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 --2- 172.21.15.17:0/2475646538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3d241034e0 0x7f3d24199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 >> 172.21.15.17:0/2475646538 conn(0x7f3d240fea90 msgr2=0x7f3d24107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 shutdown_connections 2024-02-14T15:00:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:43.291+0000 7f3d2a815640 1 -- 172.21.15.17:0/2475646538 wait complete. 2024-02-14T15:00:43.392 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:00:43 smithi050 systemd[1]: Started Ceph mds.b for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:00:44.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:44.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2475646538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] up:boot 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: fsmap 12 up:standby 2024-02-14T15:00:44.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2475646538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] up:boot 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: fsmap 12 up:standby 2024-02-14T15:00:44.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:00:44.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:44.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2475646538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:44.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:44.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:44.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] up:boot 2024-02-14T15:00:44.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: fsmap 12 up:standby 2024-02-14T15:00:44.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:00:45.305 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.409+0000 7fd79024e640 1 -- 172.21.15.17:0/4033972297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 msgr2=0x7fd788106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.409+0000 7fd79024e640 1 --2- 172.21.15.17:0/4033972297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788106aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009900 tx=0x7fd77c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.409+0000 7fd79024e640 1 -- 172.21.15.17:0/4033972297 shutdown_connections 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.409+0000 7fd79024e640 1 --2- 172.21.15.17:0/4033972297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788106aa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.409+0000 7fd79024e640 1 --2- 172.21.15.17:0/4033972297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd788106fe0 0x7fd7881093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.409+0000 7fd79024e640 1 --2- 172.21.15.17:0/4033972297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd788101d80 0x7fd788104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.409+0000 7fd79024e640 1 -- 172.21.15.17:0/4033972297 >> 172.21.15.17:0/4033972297 conn(0x7fd7880fb6d0 msgr2=0x7fd7880fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 -- 172.21.15.17:0/4033972297 shutdown_connections 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 -- 172.21.15.17:0/4033972297 wait complete. 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 Processor -- start 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 -- start start 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd788101d80 0x7fd7881972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd788106fe0 0x7fd78819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78819edb0 con 0x7fd7881046b0 2024-02-14T15:00:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78819ef20 con 0x7fd788101d80 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd79024e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78819f0e0 con 0x7fd788106fe0 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd78d7e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd78d7e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39256/0 (socket says 172.21.15.17:39256) 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd78d7e9640 1 -- 172.21.15.17:0/382471097 learned_addr learned my addr 172.21.15.17:0/382471097 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd78e7eb640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd788106fe0 0x7fd78819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.410+0000 7fd78dfea640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd788101d80 0x7fd7881972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd78d7e9640 1 -- 172.21.15.17:0/382471097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd788106fe0 msgr2=0x7fd78819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd78d7e9640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd788106fe0 0x7fd78819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd78d7e9640 1 -- 172.21.15.17:0/382471097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd788101d80 msgr2=0x7fd7881972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd78d7e9640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd788101d80 0x7fd7881972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd78d7e9640 1 -- 172.21.15.17:0/382471097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009490 con 0x7fd7881046b0 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd78e7eb640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd788106fe0 0x7fd78819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd78d7e9640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788197800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005ab0 tx=0x7fd77c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd786ffd640 1 -- 172.21.15.17:0/382471097 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0043d0 con 0x7fd7881046b0 2024-02-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd786ffd640 1 -- 172.21.15.17:0/382471097 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd77c0052d0 con 0x7fd7881046b0 2024-02-14T15:00:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd786ffd640 1 -- 172.21.15.17:0/382471097 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c003d40 con 0x7fd7881046b0 2024-02-14T15:00:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c009560 con 0x7fd7881046b0 2024-02-14T15:00:45.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.411+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78819f6e0 con 0x7fd7881046b0 2024-02-14T15:00:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.412+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7880fd1c0 con 0x7fd7881046b0 2024-02-14T15:00:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.413+0000 7fd786ffd640 1 -- 172.21.15.17:0/382471097 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd77c005440 con 0x7fd7881046b0 2024-02-14T15:00:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.414+0000 7fd786ffd640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd7700a20c0 0x7fd7700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.414+0000 7fd78dfea640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd7700a20c0 0x7fd7700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.414+0000 7fd786ffd640 1 -- 172.21.15.17:0/382471097 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd77c0ef920 con 0x7fd7881046b0 2024-02-14T15:00:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.414+0000 7fd78dfea640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd7700a20c0 0x7fd7700a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd780009d60 tx=0x7fd780005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:45.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.418+0000 7fd786ffd640 1 -- 172.21.15.17:0/382471097 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd77c0b8d90 con 0x7fd7881046b0 2024-02-14T15:00:45.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.638+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd78810ceb0 con 0x7fd7881046b0 2024-02-14T15:00:45.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.639+0000 7fd786ffd640 1 -- 172.21.15.17:0/382471097 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10013 (secure 0 0 0) 0x7fd77c0bc450 con 0x7fd7881046b0 2024-02-14T15:00:45.640 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:45.640 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-14T15:00:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd7700a20c0 msgr2=0x7fd7700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd7700a20c0 0x7fd7700a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd780009d60 tx=0x7fd780005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 msgr2=0x7fd788197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788197800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005ab0 tx=0x7fd77c005e50 comp rx=0 tx=0).stop 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 shutdown_connections 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd7700a20c0 0x7fd7700a4580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7881046b0 0x7fd788197800 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd788106fe0 0x7fd78819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 --2- 172.21.15.17:0/382471097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd788101d80 0x7fd7881972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 >> 172.21.15.17:0/382471097 conn(0x7fd7880fb6d0 msgr2=0x7fd788102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 shutdown_connections 2024-02-14T15:00:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.641+0000 7fd79024e640 1 -- 172.21.15.17:0/382471097 wait complete. 2024-02-14T15:00:45.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:00:45.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 -- 172.21.15.17:0/1684213589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8104670 msgr2=0x7ff4b8106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/1684213589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8104670 0x7ff4b8106a60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009900 tx=0x7ff4ac02bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:45.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 -- 172.21.15.17:0/1684213589 shutdown_connections 2024-02-14T15:00:45.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/1684213589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8104670 0x7ff4b8106a60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/1684213589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b8106fa0 0x7ff4b8109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/1684213589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b8101d40 0x7ff4b8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 -- 172.21.15.17:0/1684213589 >> 172.21.15.17:0/1684213589 conn(0x7ff4b80fb6d0 msgr2=0x7ff4b80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 -- 172.21.15.17:0/1684213589 shutdown_connections 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.756+0000 7ff4bca6b640 1 -- 172.21.15.17:0/1684213589 wait complete. 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 Processor -- start 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 -- start start 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b8101d40 0x7ff4b8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b8104670 0x7ff4b81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8106fa0 0x7ff4b819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b819ed80 con 0x7ff4b8106fa0 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b819eef0 con 0x7ff4b8101d40 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4bca6b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b819f0b0 con 0x7ff4b8104670 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4b6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8106fa0 0x7ff4b819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4b5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b8104670 0x7ff4b81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4b6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8106fa0 0x7ff4b819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39274/0 (socket says 172.21.15.17:39274) 2024-02-14T15:00:45.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4b6d9d640 1 -- 172.21.15.17:0/2732374474 learned_addr learned my addr 172.21.15.17:0/2732374474 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.757+0000 7ff4b659c640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b8101d40 0x7ff4b8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b5d9b640 1 -- 172.21.15.17:0/2732374474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b8101d40 msgr2=0x7ff4b8197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b5d9b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b8101d40 0x7ff4b8197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b5d9b640 1 -- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8106fa0 msgr2=0x7ff4b819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b5d9b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8106fa0 0x7ff4b819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b5d9b640 1 -- 172.21.15.17:0/2732374474 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac009560 con 0x7ff4b8104670 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b6d9d640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8106fa0 0x7ff4b819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b659c640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b8101d40 0x7ff4b8197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4b5d9b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b8104670 0x7ff4b81977d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac036040 tx=0x7ff4ac005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4a77fe640 1 -- 172.21.15.17:0/2732374474 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0043d0 con 0x7ff4b8104670 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b819f330 con 0x7ff4b8104670 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4a77fe640 1 -- 172.21.15.17:0/2732374474 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4ac0052d0 con 0x7ff4b8104670 2024-02-14T15:00:45.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4a77fe640 1 -- 172.21.15.17:0/2732374474 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac003cd0 con 0x7ff4b8104670 2024-02-14T15:00:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.758+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b819f860 con 0x7ff4b8104670 2024-02-14T15:00:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.759+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b80fd1c0 con 0x7ff4b8104670 2024-02-14T15:00:45.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.760+0000 7ff4a77fe640 1 -- 172.21.15.17:0/2732374474 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff4ac005440 con 0x7ff4b8104670 2024-02-14T15:00:45.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.761+0000 7ff4a77fe640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:45.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.761+0000 7ff4b659c640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:45.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.761+0000 7ff4a77fe640 1 -- 172.21.15.17:0/2732374474 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff4ac0efb40 con 0x7ff4b8104670 2024-02-14T15:00:45.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.761+0000 7ff4b659c640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009fc0 tx=0x7ff4a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:45.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.764+0000 7ff4a77fe640 1 -- 172.21.15.17:0/2732374474 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4ac0b8fb0 con 0x7ff4b8104670 2024-02-14T15:00:45.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.954+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4b810ce20 con 0x7ff4b8104670 2024-02-14T15:00:45.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.955+0000 7ff4a77fe640 1 -- 172.21.15.17:0/2732374474 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18350 (secure 0 0 0) 0x7ff4ac0bc670 con 0x7ff4b8104670 2024-02-14T15:00:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:45.956 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":81,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T14:59:59.565307+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:00:45.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 msgr2=0x7ff4a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009fc0 tx=0x7ff4a8005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:45.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b8104670 msgr2=0x7ff4b81977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:45.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b8104670 0x7ff4b81977d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac036040 tx=0x7ff4ac005e50 comp rx=0 tx=0).stop 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 shutdown_connections 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b8106fa0 0x7ff4b819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b8104670 0x7ff4b81977d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 --2- 172.21.15.17:0/2732374474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b8101d40 0x7ff4b8197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.957+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 >> 172.21.15.17:0/2732374474 conn(0x7ff4b80fb6d0 msgr2=0x7ff4b81027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.958+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 shutdown_connections 2024-02-14T15:00:45.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:45.958+0000 7ff4bca6b640 1 -- 172.21.15.17:0/2732374474 wait complete. 2024-02-14T15:00:45.970 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-14T15:00:45.971 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:00:46.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 -- 172.21.15.17:0/1111776463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8102430 msgr2=0x7efff81028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 --2- 172.21.15.17:0/1111776463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8102430 0x7efff81028b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7effe8009900 tx=0x7effe802bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 -- 172.21.15.17:0/1111776463 shutdown_connections 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 --2- 172.21.15.17:0/1111776463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8102430 0x7efff81028b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 --2- 172.21.15.17:0/1111776463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efff8102df0 0x7efff8109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 --2- 172.21.15.17:0/1111776463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efff8101240 0x7efff8101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 -- 172.21.15.17:0/1111776463 >> 172.21.15.17:0/1111776463 conn(0x7efff80fc7b0 msgr2=0x7efff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 -- 172.21.15.17:0/1111776463 shutdown_connections 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 -- 172.21.15.17:0/1111776463 wait complete. 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 Processor -- start 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.076+0000 7f00001e2640 1 -- start start 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8101240 0x7efff8074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efff8102430 0x7efff8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efff8102df0 0x7efff8073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff80742c0 con 0x7efff8101240 2024-02-14T15:00:46.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff8074430 con 0x7efff8102df0 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff81a1330 con 0x7efff8102430 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8101240 0x7efff8074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8101240 0x7efff8074fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39282/0 (socket says 172.21.15.17:39282) 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 -- 172.21.15.17:0/4051031814 learned_addr learned my addr 172.21.15.17:0/4051031814 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffd77d640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efff8102430 0x7efff8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 -- 172.21.15.17:0/4051031814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efff8102430 msgr2=0x7efff8073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffe77f640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efff8102df0 0x7efff8073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efff8102430 0x7efff8073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 -- 172.21.15.17:0/4051031814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efff8102df0 msgr2=0x7efff8073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efff8102df0 0x7efff8073b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 -- 172.21.15.17:0/4051031814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe8009560 con 0x7efff8101240 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffd77d640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efff8102430 0x7efff8073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7efffdf7e640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8101240 0x7efff8074fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efff400e9f0 tx=0x7efff400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:46.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7effeeffd640 1 -- 172.21.15.17:0/4051031814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4009700 con 0x7efff8101240 2024-02-14T15:00:46.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7effeeffd640 1 -- 172.21.15.17:0/4051031814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efff400cd80 con 0x7efff8101240 2024-02-14T15:00:46.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff81a15e0 con 0x7efff8101240 2024-02-14T15:00:46.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7effeeffd640 1 -- 172.21.15.17:0/4051031814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4010690 con 0x7efff8101240 2024-02-14T15:00:46.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.077+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff81a1b40 con 0x7efff8101240 2024-02-14T15:00:46.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.078+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff8108560 con 0x7efff8101240 2024-02-14T15:00:46.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.079+0000 7effeeffd640 1 -- 172.21.15.17:0/4051031814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efff4003710 con 0x7efff8101240 2024-02-14T15:00:46.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.080+0000 7effeeffd640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7effe40a2140 0x7effe40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.080+0000 7effeeffd640 1 -- 172.21.15.17:0/4051031814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efff4014070 con 0x7efff8101240 2024-02-14T15:00:46.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.080+0000 7efffd77d640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7effe40a2140 0x7effe40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.080+0000 7efffd77d640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7effe40a2140 0x7effe40a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effe8036040 tx=0x7effe8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:46.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.084+0000 7effeeffd640 1 -- 172.21.15.17:0/4051031814 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efff4097020 con 0x7efff8101240 2024-02-14T15:00:46.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:46.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/382471097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2732374474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.268+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efff802ce50 con 0x7efff8101240 2024-02-14T15:00:46.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.268+0000 7effeeffd640 1 -- 172.21.15.17:0/4051031814 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7efff409a6e0 con 0x7efff8101240 2024-02-14T15:00:46.269 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:00:46.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7effe40a2140 msgr2=0x7effe40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7effe40a2140 0x7effe40a4600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effe8036040 tx=0x7effe8000bc0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8101240 msgr2=0x7efff8074fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8101240 0x7efff8074fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efff400e9f0 tx=0x7efff400eeb0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 shutdown_connections 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7effe40a2140 0x7effe40a4600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efff8102df0 0x7efff8073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efff8102430 0x7efff8073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 --2- 172.21.15.17:0/4051031814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efff8101240 0x7efff8074fe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.270+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 >> 172.21.15.17:0/4051031814 conn(0x7efff80fc7b0 msgr2=0x7efff8105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.271+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 shutdown_connections 2024-02-14T15:00:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.271+0000 7f00001e2640 1 -- 172.21.15.17:0/4051031814 wait complete. 2024-02-14T15:00:46.286 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-14T15:00:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 -- 172.21.15.17:0/867700784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe474102ec0 msgr2=0x7fe474109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 --2- 172.21.15.17:0/867700784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe474102ec0 0x7fe474109750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009ca0 tx=0x7fe46c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 -- 172.21.15.17:0/867700784 shutdown_connections 2024-02-14T15:00:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 --2- 172.21.15.17:0/867700784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe474102ec0 0x7fe474109750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 --2- 172.21.15.17:0/867700784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe474102500 0x7fe474102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 --2- 172.21.15.17:0/867700784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4740ff530 0x7fe4740ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 -- 172.21.15.17:0/867700784 >> 172.21.15.17:0/867700784 conn(0x7fe4740fae60 msgr2=0x7fe4740fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 -- 172.21.15.17:0/867700784 shutdown_connections 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 -- 172.21.15.17:0/867700784 wait complete. 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.390+0000 7fe4798eb640 1 Processor -- start 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 -- start start 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4740ff530 0x7fe474197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe474102500 0x7fe4741977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe474102ec0 0x7fe47419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419ed80 con 0x7fe4740ff530 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419eef0 con 0x7fe474102ec0 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419f0b0 con 0x7fe474102500 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4740ff530 0x7fe474197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4740ff530 0x7fe474197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39296/0 (socket says 172.21.15.17:39296) 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 -- 172.21.15.17:0/421737802 learned_addr learned my addr 172.21.15.17:0/421737802 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4737fe640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe474102ec0 0x7fe47419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4727fc640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe474102500 0x7fe4741977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 -- 172.21.15.17:0/421737802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe474102500 msgr2=0x7fe4741977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe474102500 0x7fe4741977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 -- 172.21.15.17:0/421737802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe474102ec0 msgr2=0x7fe47419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe474102ec0 0x7fe47419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 -- 172.21.15.17:0/421737802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c009900 con 0x7fe4740ff530 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4727fc640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe474102500 0x7fe4741977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe472ffd640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4740ff530 0x7fe474197290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00d950 tx=0x7fe45c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4788e9640 1 -- 172.21.15.17:0/421737802 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c009c30 con 0x7fe4740ff530 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4788e9640 1 -- 172.21.15.17:0/421737802 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe45c009d90 con 0x7fe4740ff530 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.391+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47419f390 con 0x7fe4740ff530 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.392+0000 7fe4788e9640 1 -- 172.21.15.17:0/421737802 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00f720 con 0x7fe4740ff530 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.392+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47419f8a0 con 0x7fe4740ff530 2024-02-14T15:00:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.392+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4741085d0 con 0x7fe4740ff530 2024-02-14T15:00:46.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:46.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:46.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:46.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/382471097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2732374474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.394+0000 7fe4788e9640 1 -- 172.21.15.17:0/421737802 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe45c010a60 con 0x7fe4740ff530 2024-02-14T15:00:46.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.394+0000 7fe4788e9640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe4600a2140 0x7fe4600a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.394+0000 7fe4727fc640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe4600a2140 0x7fe4600a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.394+0000 7fe4788e9640 1 -- 172.21.15.17:0/421737802 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe45c0cd4a0 con 0x7fe4740ff530 2024-02-14T15:00:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.395+0000 7fe4727fc640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe4600a2140 0x7fe4600a4600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe4741987a0 tx=0x7fe464005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.398+0000 7fe4788e9640 1 -- 172.21.15.17:0/421737802 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe45c096990 con 0x7fe4740ff530 2024-02-14T15:00:46.428 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:46.428 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.428 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/382471097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.429 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2732374474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:46.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.581+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe47410ce30 con 0x7fe4740ff530 2024-02-14T15:00:46.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.582+0000 7fe4788e9640 1 -- 172.21.15.17:0/421737802 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe47410ce30 con 0x7fe4740ff530 2024-02-14T15:00:46.582 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:46.583 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-02-14T15:00:46.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:00:46.583 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-02-14T15:00:46.583 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-02-14T15:00:46.583 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:00:46.583 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe4600a2140 msgr2=0x7fe4600a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe4600a2140 0x7fe4600a4600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe4741987a0 tx=0x7fe464005d30 comp rx=0 tx=0).stop 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4740ff530 msgr2=0x7fe474197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4740ff530 0x7fe474197290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00d950 tx=0x7fe45c00de10 comp rx=0 tx=0).stop 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 shutdown_connections 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe4600a2140 0x7fe4600a4600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe474102ec0 0x7fe47419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe474102500 0x7fe4741977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 --2- 172.21.15.17:0/421737802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4740ff530 0x7fe474197290 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 >> 172.21.15.17:0/421737802 conn(0x7fe4740fae60 msgr2=0x7fe4741057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 shutdown_connections 2024-02-14T15:00:46.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.584+0000 7fe4798eb640 1 -- 172.21.15.17:0/421737802 wait complete. 2024-02-14T15:00:46.595 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-14T15:00:46.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-14T15:00:46.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.705+0000 7f8c04302640 1 -- 172.21.15.17:0/1267228541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 msgr2=0x7f8bfc10c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.705+0000 7f8c04302640 1 --2- 172.21.15.17:0/1267228541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc10c080 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009900 tx=0x7f8bf802bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:46.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.705+0000 7f8c04302640 1 -- 172.21.15.17:0/1267228541 shutdown_connections 2024-02-14T15:00:46.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.705+0000 7f8c04302640 1 --2- 172.21.15.17:0/1267228541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bfc10d810 0x7f8bfc1140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.705+0000 7f8c04302640 1 --2- 172.21.15.17:0/1267228541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bfc10ce50 0x7f8bfc10d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.705+0000 7f8c04302640 1 --2- 172.21.15.17:0/1267228541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc10c080 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.705+0000 7f8c04302640 1 -- 172.21.15.17:0/1267228541 >> 172.21.15.17:0/1267228541 conn(0x7f8bfc06bd20 msgr2=0x7f8bfc06c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 -- 172.21.15.17:0/1267228541 shutdown_connections 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 -- 172.21.15.17:0/1267228541 wait complete. 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 Processor -- start 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 -- start start 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bfc10ce50 0x7f8bfc1a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bfc10d810 0x7f8bfc1a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a9830 con 0x7f8bfc10bc60 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a99a0 con 0x7f8bfc10d810 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c04302640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a9b60 con 0x7f8bfc10ce50 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c0209e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c0209e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc1a1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39314/0 (socket says 172.21.15.17:39314) 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c0189d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bfc10ce50 0x7f8bfc1a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.706+0000 7f8c0209e640 1 -- 172.21.15.17:0/1038125690 learned_addr learned my addr 172.21.15.17:0/1038125690 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0289f640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bfc10d810 0x7f8bfc1a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0209e640 1 -- 172.21.15.17:0/1038125690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bfc10ce50 msgr2=0x7f8bfc1a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0209e640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bfc10ce50 0x7f8bfc1a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0209e640 1 -- 172.21.15.17:0/1038125690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bfc10d810 msgr2=0x7f8bfc1a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0209e640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bfc10d810 0x7f8bfc1a92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0209e640 1 -- 172.21.15.17:0/1038125690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf8009560 con 0x7f8bfc10bc60 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0189d640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bfc10ce50 0x7f8bfc1a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c0209e640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc1a1d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005ab0 tx=0x7f8bf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8bf37fe640 1 -- 172.21.15.17:0/1038125690 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8004aa0 con 0x7f8bfc10bc60 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc1a9de0 con 0x7f8bfc10bc60 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8bf37fe640 1 -- 172.21.15.17:0/1038125690 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bf80052d0 con 0x7f8bfc10bc60 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8bf37fe640 1 -- 172.21.15.17:0/1038125690 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8003e80 con 0x7f8bfc10bc60 2024-02-14T15:00:46.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.707+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc1aa210 con 0x7f8bfc10bc60 2024-02-14T15:00:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.708+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc113000 con 0x7f8bfc10bc60 2024-02-14T15:00:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.709+0000 7f8bf37fe640 1 -- 172.21.15.17:0/1038125690 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8bf8005440 con 0x7f8bfc10bc60 2024-02-14T15:00:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.710+0000 7f8bf37fe640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8be80a20c0 0x7f8be80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.710+0000 7f8bf37fe640 1 -- 172.21.15.17:0/1038125690 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8bf80ef710 con 0x7f8bfc10bc60 2024-02-14T15:00:46.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.710+0000 7f8c0189d640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8be80a20c0 0x7f8be80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:46.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.710+0000 7f8c0189d640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8be80a20c0 0x7f8be80a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc1a3250 tx=0x7f8bec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.715+0000 7f8bf37fe640 1 -- 172.21.15.17:0/1038125690 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bf80b8b80 con 0x7f8bfc10bc60 2024-02-14T15:00:46.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:46.934+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8bfc02cdb0 con 0x7f8bfc10bc60 2024-02-14T15:00:47.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.247+0000 7f8bf37fe640 1 -- 172.21.15.17:0/1038125690 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8bf80bc240 con 0x7f8bfc10bc60 2024-02-14T15:00:47.248 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-02-14T15:00:47.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.251+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8be80a20c0 msgr2=0x7f8be80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.251+0000 7f8c04302640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8be80a20c0 0x7f8be80a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc1a3250 tx=0x7f8bec00b390 comp rx=0 tx=0).stop 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.251+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 msgr2=0x7f8bfc1a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.251+0000 7f8c04302640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc1a1d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005ab0 tx=0x7f8bf8004270 comp rx=0 tx=0).stop 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.251+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 shutdown_connections 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.251+0000 7f8c04302640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8be80a20c0 0x7f8be80a4580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.251+0000 7f8c04302640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bfc10d810 0x7f8bfc1a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.252+0000 7f8c04302640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bfc10ce50 0x7f8bfc1a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.252+0000 7f8c04302640 1 --2- 172.21.15.17:0/1038125690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bfc10bc60 0x7f8bfc1a1d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.252+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 >> 172.21.15.17:0/1038125690 conn(0x7f8bfc06bd20 msgr2=0x7f8bfc1101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.252+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 shutdown_connections 2024-02-14T15:00:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.252+0000 7f8c04302640 1 -- 172.21.15.17:0/1038125690 wait complete. 2024-02-14T15:00:47.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 -- 172.21.15.17:0/1350993418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102de0 msgr2=0x7fedc0109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/1350993418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102de0 0x7fedc0109670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009ca0 tx=0x7fedb802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 -- 172.21.15.17:0/1350993418 shutdown_connections 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/1350993418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102de0 0x7fedc0109670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/1350993418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedc0102420 0x7fedc01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/1350993418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedc0101230 0x7fedc0101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 -- 172.21.15.17:0/1350993418 >> 172.21.15.17:0/1350993418 conn(0x7fedc00fc760 msgr2=0x7fedc00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 -- 172.21.15.17:0/1350993418 shutdown_connections 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 -- 172.21.15.17:0/1350993418 wait complete. 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 Processor -- start 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.375+0000 7fedc7a4e640 1 -- start start 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedc0101230 0x7fedc01971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102420 0x7fedc0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:47.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedc0102de0 0x7fedc019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019ecd0 con 0x7fedc0102420 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019ee40 con 0x7fedc0101230 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc019f000 con 0x7fedc0102de0 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102420 0x7fedc0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102420 0x7fedc0197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39328/0 (socket says 172.21.15.17:39328) 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 -- 172.21.15.17:0/2819183672 learned_addr learned my addr 172.21.15.17:0/2819183672 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc57ea640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedc0101230 0x7fedc01971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc5feb640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedc0102de0 0x7fedc019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 -- 172.21.15.17:0/2819183672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedc0102de0 msgr2=0x7fedc019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedc0102de0 0x7fedc019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 -- 172.21.15.17:0/2819183672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedc0101230 msgr2=0x7fedc01971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedc0101230 0x7fedc01971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 -- 172.21.15.17:0/2819183672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009900 con 0x7fedc0102420 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc4fe9640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102420 0x7fedc0197720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fedb000d950 tx=0x7fedb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedb67fc640 1 -- 172.21.15.17:0/2819183672 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0009c30 con 0x7fedc0102420 2024-02-14T15:00:47.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedb67fc640 1 -- 172.21.15.17:0/2819183672 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedb0009d90 con 0x7fedc0102420 2024-02-14T15:00:47.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc019f2e0 con 0x7fedc0102420 2024-02-14T15:00:47.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedb67fc640 1 -- 172.21.15.17:0/2819183672 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb000f7e0 con 0x7fedc0102420 2024-02-14T15:00:47.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.376+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc019f840 con 0x7fedc0102420 2024-02-14T15:00:47.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.378+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc0108480 con 0x7fedc0102420 2024-02-14T15:00:47.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.379+0000 7fedb67fc640 1 -- 172.21.15.17:0/2819183672 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fedb0010ad0 con 0x7fedc0102420 2024-02-14T15:00:47.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.379+0000 7fedb67fc640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedac0a2140 0x7fedac0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:47.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.380+0000 7fedc57ea640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedac0a2140 0x7fedac0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:47.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.380+0000 7fedb67fc640 1 -- 172.21.15.17:0/2819183672 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fedb00cd440 con 0x7fedc0102420 2024-02-14T15:00:47.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.380+0000 7fedc57ea640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedac0a2140 0x7fedac0a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fedc0102280 tx=0x7fedbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.383+0000 7fedb67fc640 1 -- 172.21.15.17:0/2819183672 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fedb0096870 con 0x7fedc0102420 2024-02-14T15:00:47.483 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:47 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4051031814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:47.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:47 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/421737802' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-14T15:00:47.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:47 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4051031814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:47.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:47 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/421737802' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-14T15:00:47.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:47.565+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fedc010cdd0 con 0x7fedc0102420 2024-02-14T15:00:47.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:47 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4051031814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:00:47.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:47 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/421737802' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-14T15:00:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.250+0000 7fedb67fc640 1 -- 172.21.15.17:0/2819183672 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7fedb00200d0 con 0x7fedc0102420 2024-02-14T15:00:48.251 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-02-14T15:00:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedac0a2140 msgr2=0x7fedac0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedac0a2140 0x7fedac0a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fedc0102280 tx=0x7fedbc005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102420 msgr2=0x7fedc0197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102420 0x7fedc0197720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fedb000d950 tx=0x7fedb000de10 comp rx=0 tx=0).stop 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 shutdown_connections 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedac0a2140 0x7fedac0a4600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedc0102420 0x7fedc0197720 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedc0102de0 0x7fedc019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 --2- 172.21.15.17:0/2819183672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedc0101230 0x7fedc01971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 >> 172.21.15.17:0/2819183672 conn(0x7fedc00fc760 msgr2=0x7fedc0105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 shutdown_connections 2024-02-14T15:00:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.253+0000 7fedc7a4e640 1 -- 172.21.15.17:0/2819183672 wait complete. 2024-02-14T15:00:48.278 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-14T15:00:48.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 -- 172.21.15.17:0/1102041684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 msgr2=0x7f36641028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:48.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 --2- 172.21.15.17:0/1102041684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f36641028b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f364c009900 tx=0x7f364c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:48.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 -- 172.21.15.17:0/1102041684 shutdown_connections 2024-02-14T15:00:48.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 --2- 172.21.15.17:0/1102041684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f36641028b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 --2- 172.21.15.17:0/1102041684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3664102df0 0x7f3664109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 --2- 172.21.15.17:0/1102041684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3664101240 0x7f3664101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 -- 172.21.15.17:0/1102041684 >> 172.21.15.17:0/1102041684 conn(0x7f36640fc7b0 msgr2=0x7f36640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 -- 172.21.15.17:0/1102041684 shutdown_connections 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.389+0000 7f366956c640 1 -- 172.21.15.17:0/1102041684 wait complete. 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 Processor -- start 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 -- start start 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3664101240 0x7f36641972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f3664197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3664102df0 0x7f366419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419edd0 con 0x7f3664102430 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419ef40 con 0x7f3664101240 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f366956c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366419f100 con 0x7f3664102df0 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f3664197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f3664197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39346/0 (socket says 172.21.15.17:39346) 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 -- 172.21.15.17:0/4151408847 learned_addr learned my addr 172.21.15.17:0/4151408847 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f3662ffd640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3664101240 0x7f36641972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f36637fe640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3664102df0 0x7f366419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 -- 172.21.15.17:0/4151408847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3664102df0 msgr2=0x7f366419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3664102df0 0x7f366419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 -- 172.21.15.17:0/4151408847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3664101240 msgr2=0x7f36641972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3664101240 0x7f36641972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f365bfff640 1 -- 172.21.15.17:0/4151408847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c009560 con 0x7f3664102430 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.390+0000 7f3662ffd640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3664101240 0x7f36641972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.391+0000 7f365bfff640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f3664197820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f364c006010 tx=0x7f364c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.391+0000 7f3660ff9640 1 -- 172.21.15.17:0/4151408847 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0044c0 con 0x7f3664102430 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.391+0000 7f3660ff9640 1 -- 172.21.15.17:0/4151408847 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f364c0052d0 con 0x7f3664102430 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.391+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366419f380 con 0x7f3664102430 2024-02-14T15:00:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.391+0000 7f3660ff9640 1 -- 172.21.15.17:0/4151408847 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c003e10 con 0x7f3664102430 2024-02-14T15:00:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.391+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366419f890 con 0x7f3664102430 2024-02-14T15:00:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.392+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3664108560 con 0x7f3664102430 2024-02-14T15:00:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.393+0000 7f3660ff9640 1 -- 172.21.15.17:0/4151408847 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f364c005440 con 0x7f3664102430 2024-02-14T15:00:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.393+0000 7f3660ff9640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36500a20c0 0x7f36500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.394+0000 7f3662ffd640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36500a20c0 0x7f36500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.394+0000 7f3660ff9640 1 -- 172.21.15.17:0/4151408847 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f364c0efea0 con 0x7f3664102430 2024-02-14T15:00:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.394+0000 7f3662ffd640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36500a20c0 0x7f36500a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3664102290 tx=0x7f3654005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.398+0000 7f3660ff9640 1 -- 172.21.15.17:0/4151408847 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f364c0b9020 con 0x7f3664102430 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1038125690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:48.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:48.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1038125690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-14T15:00:48.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: osdmap e82: 12 total, 12 up, 12 in 2024-02-14T15:00:48.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2819183672' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-14T15:00:48.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:48 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:48.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1038125690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-14T15:00:48.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1038125690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: osdmap e82: 12 total, 12 up, 12 in 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2819183672' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-14T15:00:48.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:48 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:48.618+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f366402cec0 con 0x7f3664102430 2024-02-14T15:00:48.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:48.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1038125690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-14T15:00:48.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1038125690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: osdmap e82: 12 total, 12 up, 12 in 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2819183672' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-14T15:00:48.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:48 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:00:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.257+0000 7f3660ff9640 1 -- 172.21.15.17:0/4151408847 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f364c0bc6e0 con 0x7f3664102430 2024-02-14T15:00:49.258 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 2 and data pool 3 2024-02-14T15:00:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.262+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36500a20c0 msgr2=0x7f36500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.262+0000 7f366956c640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36500a20c0 0x7f36500a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3664102290 tx=0x7f3654005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.262+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 msgr2=0x7f3664197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.262+0000 7f366956c640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f3664197820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f364c006010 tx=0x7f364c0040c0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.263+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 shutdown_connections 2024-02-14T15:00:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.263+0000 7f366956c640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36500a20c0 0x7f36500a4580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.263+0000 7f366956c640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3664102430 0x7f3664197820 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.263+0000 7f366956c640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3664102df0 0x7f366419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.263+0000 7f366956c640 1 --2- 172.21.15.17:0/4151408847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3664101240 0x7f36641972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.263+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 >> 172.21.15.17:0/4151408847 conn(0x7f36640fc7b0 msgr2=0x7f3664105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:49.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.263+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 shutdown_connections 2024-02-14T15:00:49.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.264+0000 7f366956c640 1 -- 172.21.15.17:0/4151408847 wait complete. 2024-02-14T15:00:49.285 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-02-14T15:00:49.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-02-14T15:00:49.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 -- 172.21.15.17:0/904119343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c101340 msgr2=0x7fa13c1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:49.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 --2- 172.21.15.17:0/904119343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c101340 0x7fa13c1017c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa130009900 tx=0x7fa13002bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:49.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 -- 172.21.15.17:0/904119343 shutdown_connections 2024-02-14T15:00:49.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 --2- 172.21.15.17:0/904119343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c101340 0x7fa13c1017c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 --2- 172.21.15.17:0/904119343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101d00 0x7fa13c108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 --2- 172.21.15.17:0/904119343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c100150 0x7fa13c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 -- 172.21.15.17:0/904119343 >> 172.21.15.17:0/904119343 conn(0x7fa13c0fb6c0 msgr2=0x7fa13c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 -- 172.21.15.17:0/904119343 shutdown_connections 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.394+0000 7fa144525640 1 -- 172.21.15.17:0/904119343 wait complete. 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 Processor -- start 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 -- start start 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c100150 0x7fa13c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c101340 0x7fa13c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101d00 0x7fa13c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19eda0 con 0x7fa13c100150 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19ef10 con 0x7fa13c101340 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa144525640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19f0d0 con 0x7fa13c101d00 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c100150 0x7fa13c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:49.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c100150 0x7fa13c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39360/0 (socket says 172.21.15.17:39360) 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 -- 172.21.15.17:0/3634079535 learned_addr learned my addr 172.21.15.17:0/3634079535 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa141ac0640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c101340 0x7fa13c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 -- 172.21.15.17:0/3634079535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101d00 msgr2=0x7fa13c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa142ac2640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101d00 0x7fa13c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101d00 0x7fa13c19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 -- 172.21.15.17:0/3634079535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c101340 msgr2=0x7fa13c19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c101340 0x7fa13c19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa1422c1640 1 -- 172.21.15.17:0/3634079535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa130009560 con 0x7fa13c100150 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.395+0000 7fa141ac0640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c101340 0x7fa13c19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.396+0000 7fa1422c1640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c100150 0x7fa13c069e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c970 tx=0x7fa13400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.396+0000 7fa13b7fe640 1 -- 172.21.15.17:0/3634079535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134007c10 con 0x7fa13c100150 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.396+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c19f350 con 0x7fa13c100150 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.396+0000 7fa13b7fe640 1 -- 172.21.15.17:0/3634079535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa134007d70 con 0x7fa13c100150 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.396+0000 7fa13b7fe640 1 -- 172.21.15.17:0/3634079535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13400f870 con 0x7fa13c100150 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.396+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c19f8b0 con 0x7fa13c100150 2024-02-14T15:00:49.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.397+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c107470 con 0x7fa13c100150 2024-02-14T15:00:49.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.398+0000 7fa13b7fe640 1 -- 172.21.15.17:0/3634079535 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa134018a60 con 0x7fa13c100150 2024-02-14T15:00:49.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.398+0000 7fa13b7fe640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa1240a20c0 0x7fa1240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:49.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.398+0000 7fa141ac0640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa1240a20c0 0x7fa1240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:49.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.398+0000 7fa13b7fe640 1 -- 172.21.15.17:0/3634079535 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa134013070 con 0x7fa13c100150 2024-02-14T15:00:49.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.399+0000 7fa141ac0640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa1240a20c0 0x7fa1240a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa13c069b90 tx=0x7fa130005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:49.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.403+0000 7fa13b7fe640 1 -- 172.21.15.17:0/3634079535 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1340970d0 con 0x7fa13c100150 2024-02-14T15:00:49.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:49 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2819183672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-14T15:00:49.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:49 smithi017 ceph-mon[32549]: osdmap e83: 12 total, 12 up, 12 in 2024-02-14T15:00:49.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:49 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4151408847' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-14T15:00:49.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:49 smithi017 ceph-mon[32549]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-14T15:00:49.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:49.596+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa13c02cdb0 con 0x7fa13c100150 2024-02-14T15:00:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:49 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2819183672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-14T15:00:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:49 smithi050 ceph-mon[36512]: osdmap e83: 12 total, 12 up, 12 in 2024-02-14T15:00:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:49 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4151408847' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-14T15:00:49.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:49 smithi050 ceph-mon[36512]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-14T15:00:49.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:49 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2819183672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-14T15:00:49.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:49 smithi076 ceph-mon[37057]: osdmap e83: 12 total, 12 up, 12 in 2024-02-14T15:00:49.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:49 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4151408847' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-14T15:00:49.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:49 smithi076 ceph-mon[37057]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-14T15:00:50.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.253+0000 7fa13b7fe640 1 -- 172.21.15.17:0/3634079535 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa13409a790 con 0x7fa13c100150 2024-02-14T15:00:50.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.256+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa1240a20c0 msgr2=0x7fa1240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.256+0000 7fa144525640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa1240a20c0 0x7fa1240a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa13c069b90 tx=0x7fa130005be0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.256+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c100150 msgr2=0x7fa13c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.256+0000 7fa144525640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c100150 0x7fa13c069e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c970 tx=0x7fa13400ce30 comp rx=0 tx=0).stop 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 shutdown_connections 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa1240a20c0 0x7fa1240a4580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa13c101d00 0x7fa13c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c101340 0x7fa13c19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 --2- 172.21.15.17:0/3634079535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c100150 0x7fa13c069e20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 >> 172.21.15.17:0/3634079535 conn(0x7fa13c0fb6c0 msgr2=0x7fa13c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 shutdown_connections 2024-02-14T15:00:50.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.257+0000 7fa144525640 1 -- 172.21.15.17:0/3634079535 wait complete. 2024-02-14T15:00:50.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-02-14T15:00:50.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- 172.21.15.17:0/1761803931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 msgr2=0x7f1fc41017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:50.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 --2- 172.21.15.17:0/1761803931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc41017d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009900 tx=0x7f1fb802bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:50.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- 172.21.15.17:0/1761803931 shutdown_connections 2024-02-14T15:00:50.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 --2- 172.21.15.17:0/1761803931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc41017d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 --2- 172.21.15.17:0/1761803931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc4101d10 0x7f1fc41085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 --2- 172.21.15.17:0/1761803931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc4100160 0x7f1fc4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- 172.21.15.17:0/1761803931 >> 172.21.15.17:0/1761803931 conn(0x7f1fc40fb6d0 msgr2=0x7f1fc40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- 172.21.15.17:0/1761803931 shutdown_connections 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- 172.21.15.17:0/1761803931 wait complete. 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 Processor -- start 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- start start 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc4100160 0x7f1fc4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc4101d10 0x7f1fc419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc419edb0 con 0x7f1fc4101350 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc419ef20 con 0x7f1fc4101d10 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.389+0000 7f1fcb650640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc419f0e0 con 0x7f1fc4100160 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:50.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc4197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39372/0 (socket says 172.21.15.17:39372) 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 -- 172.21.15.17:0/2897506452 learned_addr learned my addr 172.21.15.17:0/2897506452 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc93ec640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc4100160 0x7f1fc4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc9bed640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc4101d10 0x7f1fc419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 -- 172.21.15.17:0/2897506452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc4100160 msgr2=0x7f1fc4197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc4100160 0x7f1fc4197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 -- 172.21.15.17:0/2897506452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc4101d10 msgr2=0x7f1fc419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc4101d10 0x7f1fc419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 -- 172.21.15.17:0/2897506452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb8009560 con 0x7f1fc4101350 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc8beb640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc4197850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006010 tx=0x7f1fb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc27fc640 1 -- 172.21.15.17:0/2897506452 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80044e0 con 0x7f1fc4101350 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc419f360 con 0x7f1fc4101350 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc27fc640 1 -- 172.21.15.17:0/2897506452 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb80054e0 con 0x7f1fc4101350 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fc27fc640 1 -- 172.21.15.17:0/2897506452 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb803e450 con 0x7f1fc4101350 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.390+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc419f8c0 con 0x7f1fc4101350 2024-02-14T15:00:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.391+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc410bb70 con 0x7f1fc4101350 2024-02-14T15:00:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.392+0000 7f1fc27fc640 1 -- 172.21.15.17:0/2897506452 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1fb80035f0 con 0x7f1fc4101350 2024-02-14T15:00:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.393+0000 7f1fc27fc640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1fac0a20c0 0x7f1fac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.393+0000 7f1fc93ec640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1fac0a20c0 0x7f1fac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.393+0000 7f1fc27fc640 1 -- 172.21.15.17:0/2897506452 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f1fb80f0010 con 0x7f1fc4101350 2024-02-14T15:00:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.393+0000 7f1fc93ec640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1fac0a20c0 0x7f1fac0a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fc41011b0 tx=0x7f1fbc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:50.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.397+0000 7f1fc27fc640 1 -- 172.21.15.17:0/2897506452 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fb80b9000 con 0x7f1fc4101350 2024-02-14T15:00:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: pgmap v286: 97 pgs: 7 creating+peering, 4 active+clean, 86 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T15:00:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: osdmap e84: 12 total, 12 up, 12 in 2024-02-14T15:00:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4151408847' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-14T15:00:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-14T15:00:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-14T15:00:50.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3634079535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T15:00:50.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3634079535' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-14T15:00:50.517 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:50 smithi017 ceph-mon[32549]: osdmap e85: 12 total, 12 up, 12 in 2024-02-14T15:00:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:50.582+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1fc419fcf0 con 0x7f1fc4101350 2024-02-14T15:00:50.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: pgmap v286: 97 pgs: 7 creating+peering, 4 active+clean, 86 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:50.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T15:00:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: osdmap e84: 12 total, 12 up, 12 in 2024-02-14T15:00:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4151408847' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-14T15:00:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-14T15:00:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-14T15:00:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3634079535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T15:00:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3634079535' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-14T15:00:50.643 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:50 smithi050 ceph-mon[36512]: osdmap e85: 12 total, 12 up, 12 in 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: pgmap v286: 97 pgs: 7 creating+peering, 4 active+clean, 86 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: osdmap e84: 12 total, 12 up, 12 in 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4151408847' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3634079535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T15:00:50.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3634079535' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-14T15:00:50.735 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:50 smithi076 ceph-mon[37057]: osdmap e85: 12 total, 12 up, 12 in 2024-02-14T15:00:51.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:51 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] up:active 2024-02-14T15:00:51.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:51 smithi017 ceph-mon[32549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:51.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:51 smithi017 ceph-mon[32549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:51.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:51 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2897506452' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-14T15:00:51.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:51 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] up:active 2024-02-14T15:00:51.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:51 smithi050 ceph-mon[36512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:51.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:51 smithi050 ceph-mon[36512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:51.642 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:51 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2897506452' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-14T15:00:51.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:51 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] up:active 2024-02-14T15:00:51.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:51 smithi076 ceph-mon[37057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:51.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:51 smithi076 ceph-mon[37057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:51.734 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:51 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2897506452' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-14T15:00:52.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:52 smithi050 ceph-mon[36512]: pgmap v289: 97 pgs: 19 creating+peering, 68 active+clean, 10 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-02-14T15:00:52.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:52 smithi050 ceph-mon[36512]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-14T15:00:52.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:52 smithi050 ceph-mon[36512]: Cluster is now healthy 2024-02-14T15:00:52.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:52 smithi050 ceph-mon[36512]: osdmap e86: 12 total, 12 up, 12 in 2024-02-14T15:00:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.556+0000 7f1fc27fc640 1 -- 172.21.15.17:0/2897506452 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f1fb80bc6c0 con 0x7f1fc4101350 2024-02-14T15:00:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' created 2024-02-14T15:00:52.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.559+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1fac0a20c0 msgr2=0x7f1fac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:52.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.559+0000 7f1fcb650640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1fac0a20c0 0x7f1fac0a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fc41011b0 tx=0x7f1fbc00b390 comp rx=0 tx=0).stop 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.559+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 msgr2=0x7f1fc4197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.559+0000 7f1fcb650640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc4197850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006010 tx=0x7f1fb80042f0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 shutdown_connections 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1fac0a20c0 0x7f1fac0a4580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fc4101350 0x7f1fc4197850 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006010 tx=0x7f1fb80042f0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc4101d10 0x7f1fc419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 --2- 172.21.15.17:0/2897506452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc4100160 0x7f1fc4197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 >> 172.21.15.17:0/2897506452 conn(0x7f1fc40fb6d0 msgr2=0x7f1fc40fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 shutdown_connections 2024-02-14T15:00:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.560+0000 7f1fcb650640 1 -- 172.21.15.17:0/2897506452 wait complete. 2024-02-14T15:00:52.578 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:52 smithi017 ceph-mon[32549]: pgmap v289: 97 pgs: 19 creating+peering, 68 active+clean, 10 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-02-14T15:00:52.578 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:52 smithi017 ceph-mon[32549]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-14T15:00:52.578 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:52 smithi017 ceph-mon[32549]: Cluster is now healthy 2024-02-14T15:00:52.578 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:52 smithi017 ceph-mon[32549]: osdmap e86: 12 total, 12 up, 12 in 2024-02-14T15:00:52.579 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-02-14T15:00:52.635 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:52 smithi076 ceph-mon[37057]: pgmap v289: 97 pgs: 19 creating+peering, 68 active+clean, 10 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-02-14T15:00:52.635 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:52 smithi076 ceph-mon[37057]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-14T15:00:52.636 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:52 smithi076 ceph-mon[37057]: Cluster is now healthy 2024-02-14T15:00:52.636 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:52 smithi076 ceph-mon[37057]: osdmap e86: 12 total, 12 up, 12 in 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 -- 172.21.15.17:0/556011752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102df0 msgr2=0x7fb530109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 --2- 172.21.15.17:0/556011752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102df0 0x7fb530109680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb528009ca0 tx=0x7fb52802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 -- 172.21.15.17:0/556011752 shutdown_connections 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 --2- 172.21.15.17:0/556011752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102df0 0x7fb530109680 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 --2- 172.21.15.17:0/556011752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb530102430 0x7fb5301028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 --2- 172.21.15.17:0/556011752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb530101240 0x7fb530101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 -- 172.21.15.17:0/556011752 >> 172.21.15.17:0/556011752 conn(0x7fb5300fc7b0 msgr2=0x7fb5300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 -- 172.21.15.17:0/556011752 shutdown_connections 2024-02-14T15:00:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.691+0000 7fb53873e640 1 -- 172.21.15.17:0/556011752 wait complete. 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 Processor -- start 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 -- start start 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb530101240 0x7fb530197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102430 0x7fb5301977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb530102df0 0x7fb53019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019ed80 con 0x7fb530102430 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019eef0 con 0x7fb530102df0 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb53873e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019f0b0 con 0x7fb530101240 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102430 0x7fb5301977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102430 0x7fb5301977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38466/0 (socket says 172.21.15.17:38466) 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 -- 172.21.15.17:0/3179463323 learned_addr learned my addr 172.21.15.17:0/3179463323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb5364da640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb530101240 0x7fb530197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb536cdb640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb530102df0 0x7fb53019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 -- 172.21.15.17:0/3179463323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb530101240 msgr2=0x7fb530197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb530101240 0x7fb530197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 -- 172.21.15.17:0/3179463323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb530102df0 msgr2=0x7fb53019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb530102df0 0x7fb53019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 -- 172.21.15.17:0/3179463323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528009900 con 0x7fb530102430 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.692+0000 7fb535cd9640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102430 0x7fb5301977d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb52000d950 tx=0x7fb52000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.693+0000 7fb5277fe640 1 -- 172.21.15.17:0/3179463323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520009a30 con 0x7fb530102430 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.693+0000 7fb5277fe640 1 -- 172.21.15.17:0/3179463323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb520009b90 con 0x7fb530102430 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.693+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53019f390 con 0x7fb530102430 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.693+0000 7fb5277fe640 1 -- 172.21.15.17:0/3179463323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520004a00 con 0x7fb530102430 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.693+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53019f8f0 con 0x7fb530102430 2024-02-14T15:00:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.694+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb530108560 con 0x7fb530102430 2024-02-14T15:00:52.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.695+0000 7fb5277fe640 1 -- 172.21.15.17:0/3179463323 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb520009d00 con 0x7fb530102430 2024-02-14T15:00:52.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.695+0000 7fb5277fe640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb51c0a2140 0x7fb51c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:52.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.695+0000 7fb5364da640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb51c0a2140 0x7fb51c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:52.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.695+0000 7fb5277fe640 1 -- 172.21.15.17:0/3179463323 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fb5200cdda0 con 0x7fb530102430 2024-02-14T15:00:52.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.696+0000 7fb5364da640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb51c0a2140 0x7fb51c0a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb530102290 tx=0x7fb52c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.699+0000 7fb5277fe640 1 -- 172.21.15.17:0/3179463323 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb520096c20 con 0x7fb530102430 2024-02-14T15:00:52.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:52.879+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb53010cc50 con 0x7fb530102430 2024-02-14T15:00:53.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.561+0000 7fb5277fe640 1 -- 172.21.15.17:0/3179463323 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb520017070 con 0x7fb530102430 2024-02-14T15:00:53.563 INFO:teuthology.orchestra.run.smithi017.stderr:set pool 4 allow_ec_overwrites to true 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb51c0a2140 msgr2=0x7fb51c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb51c0a2140 0x7fb51c0a4600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb530102290 tx=0x7fb52c005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102430 msgr2=0x7fb5301977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102430 0x7fb5301977d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb52000d950 tx=0x7fb52000de10 comp rx=0 tx=0).stop 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 shutdown_connections 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb51c0a2140 0x7fb51c0a4600 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb530102290 tx=0x7fb52c005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb530102430 0x7fb5301977d0 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb52000d950 tx=0x7fb52000de10 comp rx=0 tx=0).stop 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb530102df0 0x7fb53019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 --2- 172.21.15.17:0/3179463323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb530101240 0x7fb530197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 >> 172.21.15.17:0/3179463323 conn(0x7fb5300fc7b0 msgr2=0x7fb530105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:53.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 shutdown_connections 2024-02-14T15:00:53.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.566+0000 7fb53873e640 1 -- 172.21.15.17:0/3179463323 wait complete. 2024-02-14T15:00:53.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-02-14T15:00:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 -- 172.21.15.17:0/2838947663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 msgr2=0x7fcda40ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/2838947663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda40ff920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009900 tx=0x7fcd9402bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 -- 172.21.15.17:0/2838947663 shutdown_connections 2024-02-14T15:00:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/2838947663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcda4102e90 0x7fcda4109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:53.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/2838947663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcda41024d0 0x7fcda4102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/2838947663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda40ff920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 -- 172.21.15.17:0/2838947663 >> 172.21.15.17:0/2838947663 conn(0x7fcda40fae50 msgr2=0x7fcda40fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 -- 172.21.15.17:0/2838947663 shutdown_connections 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 -- 172.21.15.17:0/2838947663 wait complete. 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 Processor -- start 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 -- start start 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcda41024d0 0x7fcda4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.699+0000 7fcdaabbc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcda4102e90 0x7fcda419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcdaabbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda419ee00 con 0x7fcda40ff500 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcdaabbc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda419ef70 con 0x7fcda4102e90 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcdaabbc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda419f130 con 0x7fcda41024d0 2024-02-14T15:00:53.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda4197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38486/0 (socket says 172.21.15.17:38486) 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 -- 172.21.15.17:0/706813419 learned_addr learned my addr 172.21.15.17:0/706813419 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda3fff640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcda41024d0 0x7fcda4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda9159640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcda4102e90 0x7fcda419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 -- 172.21.15.17:0/706813419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcda41024d0 msgr2=0x7fcda4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcda41024d0 0x7fcda4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 -- 172.21.15.17:0/706813419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcda4102e90 msgr2=0x7fcda419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcda4102e90 0x7fcda419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 -- 172.21.15.17:0/706813419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94009560 con 0x7fcda40ff500 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda8958640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda4197310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd94000c00 tx=0x7fcd94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda1ffb640 1 -- 172.21.15.17:0/706813419 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94004df0 con 0x7fcda40ff500 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda1ffb640 1 -- 172.21.15.17:0/706813419 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd94005430 con 0x7fcda40ff500 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda419f3b0 con 0x7fcda40ff500 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.700+0000 7fcda1ffb640 1 -- 172.21.15.17:0/706813419 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9403e450 con 0x7fcda40ff500 2024-02-14T15:00:53.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.701+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda419f8c0 con 0x7fcda40ff500 2024-02-14T15:00:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.702+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda41086d0 con 0x7fcda40ff500 2024-02-14T15:00:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.702+0000 7fcda1ffb640 1 -- 172.21.15.17:0/706813419 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcd940035f0 con 0x7fcda40ff500 2024-02-14T15:00:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.703+0000 7fcda1ffb640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd8c0a20c0 0x7fcd8c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:53.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.703+0000 7fcda3fff640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd8c0a20c0 0x7fcd8c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.703+0000 7fcda1ffb640 1 -- 172.21.15.17:0/706813419 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fcd940f0070 con 0x7fcda40ff500 2024-02-14T15:00:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.703+0000 7fcda3fff640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd8c0a20c0 0x7fcd8c0a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcda4198820 tx=0x7fcd9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.708+0000 7fcda1ffb640 1 -- 172.21.15.17:0/706813419 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd940b8e10 con 0x7fcda40ff500 2024-02-14T15:00:53.838 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:53 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2897506452' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-14T15:00:53.838 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:53 smithi076 ceph-mon[37057]: osdmap e87: 12 total, 12 up, 12 in 2024-02-14T15:00:53.838 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:53 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3179463323' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-14T15:00:53.846 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:53 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2897506452' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-14T15:00:53.846 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:53 smithi050 ceph-mon[36512]: osdmap e87: 12 total, 12 up, 12 in 2024-02-14T15:00:53.846 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:53 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3179463323' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-14T15:00:53.847 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:00:53 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b[67204]: 2024-02-14T15:00:53.556+0000 7f6127ec9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-14T15:00:53.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:53.930+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fcda402cde0 con 0x7fcda40ff500 2024-02-14T15:00:54.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:53 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2897506452' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-14T15:00:54.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:53 smithi017 ceph-mon[32549]: osdmap e87: 12 total, 12 up, 12 in 2024-02-14T15:00:54.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:53 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3179463323' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-14T15:00:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.563+0000 7fcda1ffb640 1 -- 172.21.15.17:0/706813419 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fcd940bc4d0 con 0x7fcda40ff500 2024-02-14T15:00:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:added data pool 4 to fsmap 2024-02-14T15:00:54.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd8c0a20c0 msgr2=0x7fcd8c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:54.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd8c0a20c0 0x7fcd8c0a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcda4198820 tx=0x7fcd9800b410 comp rx=0 tx=0).stop 2024-02-14T15:00:54.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 msgr2=0x7fcda4197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:54.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda4197310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd94000c00 tx=0x7fcd94004270 comp rx=0 tx=0).stop 2024-02-14T15:00:54.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 shutdown_connections 2024-02-14T15:00:54.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd8c0a20c0 0x7fcd8c0a4580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcda4102e90 0x7fcda419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcda41024d0 0x7fcda4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 --2- 172.21.15.17:0/706813419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda40ff500 0x7fcda4197310 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.566+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 >> 172.21.15.17:0/706813419 conn(0x7fcda40fae50 msgr2=0x7fcda41058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:54.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.567+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 shutdown_connections 2024-02-14T15:00:54.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.567+0000 7fcdaabbc640 1 -- 172.21.15.17:0/706813419 wait complete. 2024-02-14T15:00:54.582 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 -- 172.21.15.17:0/218844336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc101d80 msgr2=0x7ff7dc108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 --2- 172.21.15.17:0/218844336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc101d80 0x7ff7dc108610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009ca0 tx=0x7ff7cc02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 -- 172.21.15.17:0/218844336 shutdown_connections 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 --2- 172.21.15.17:0/218844336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc101d80 0x7ff7dc108610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 --2- 172.21.15.17:0/218844336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7dc1013c0 0x7ff7dc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 --2- 172.21.15.17:0/218844336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7dc1001d0 0x7ff7dc1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 -- 172.21.15.17:0/218844336 >> 172.21.15.17:0/218844336 conn(0x7ff7dc0fb760 msgr2=0x7ff7dc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 -- 172.21.15.17:0/218844336 shutdown_connections 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 -- 172.21.15.17:0/218844336 wait complete. 2024-02-14T15:00:54.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.689+0000 7ff7e3630640 1 Processor -- start 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 -- start start 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc1001d0 0x7ff7dc197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7dc1013c0 0x7ff7dc1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7dc101d80 0x7ff7dc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc19ed60 con 0x7ff7dc1001d0 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc19eed0 con 0x7ff7dc101d80 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc19f090 con 0x7ff7dc1013c0 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc1001d0 0x7ff7dc197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc1001d0 0x7ff7dc197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38512/0 (socket says 172.21.15.17:38512) 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 -- 172.21.15.17:0/2681103394 learned_addr learned my addr 172.21.15.17:0/2681103394 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e1bcd640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7dc101d80 0x7ff7dc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e0bcb640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7dc1013c0 0x7ff7dc1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 -- 172.21.15.17:0/2681103394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7dc1013c0 msgr2=0x7ff7dc1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7dc1013c0 0x7ff7dc1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 -- 172.21.15.17:0/2681103394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7dc101d80 msgr2=0x7ff7dc19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7dc101d80 0x7ff7dc19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 -- 172.21.15.17:0/2681103394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009900 con 0x7ff7dc1001d0 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e13cc640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc1001d0 0x7ff7dc197270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400d950 tx=0x7ff7d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7da7fc640 1 -- 172.21.15.17:0/2681103394 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4009c30 con 0x7ff7dc1001d0 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc19f370 con 0x7ff7dc1001d0 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7da7fc640 1 -- 172.21.15.17:0/2681103394 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7d4009d90 con 0x7ff7dc1001d0 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7da7fc640 1 -- 172.21.15.17:0/2681103394 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400f720 con 0x7ff7dc1001d0 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.690+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc19f830 con 0x7ff7dc1001d0 2024-02-14T15:00:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.691+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc10bb70 con 0x7ff7dc1001d0 2024-02-14T15:00:54.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.692+0000 7ff7da7fc640 1 -- 172.21.15.17:0/2681103394 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff7d4010a60 con 0x7ff7dc1001d0 2024-02-14T15:00:54.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.693+0000 7ff7da7fc640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff7c40a20c0 0x7ff7c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:54.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.693+0000 7ff7e0bcb640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff7c40a20c0 0x7ff7c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:54.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.693+0000 7ff7da7fc640 1 -- 172.21.15.17:0/2681103394 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7d40cdb80 con 0x7ff7dc1001d0 2024-02-14T15:00:54.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.693+0000 7ff7e0bcb640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff7c40a20c0 0x7ff7c40a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc198780 tx=0x7ff7d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:54.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.697+0000 7ff7da7fc640 1 -- 172.21.15.17:0/2681103394 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7d4096960 con 0x7ff7dc1001d0 2024-02-14T15:00:54.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:54 smithi050 ceph-mon[36512]: pgmap v292: 129 pgs: 19 creating+peering, 76 active+clean, 34 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-02-14T15:00:54.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:54 smithi050 ceph-mon[36512]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T15:00:54.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:54 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3179463323' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-14T15:00:54.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:54 smithi050 ceph-mon[36512]: osdmap e88: 12 total, 12 up, 12 in 2024-02-14T15:00:54.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:54 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/706813419' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-14T15:00:54.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.915+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7dc19fa50 con 0x7ff7dc1001d0 2024-02-14T15:00:54.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.916+0000 7ff7da7fc640 1 -- 172.21.15.17:0/2681103394 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11256 (secure 0 0 0) 0x7ff7d4017090 con 0x7ff7dc1001d0 2024-02-14T15:00:54.917 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:54.917 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:00:53.931986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-14T15:00:54.917 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-02-14T15:00:54.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff7c40a20c0 msgr2=0x7ff7c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff7c40a20c0 0x7ff7c40a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc198780 tx=0x7ff7d0005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc1001d0 msgr2=0x7ff7dc197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc1001d0 0x7ff7dc197270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400d950 tx=0x7ff7d400de10 comp rx=0 tx=0).stop 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 shutdown_connections 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff7c40a20c0 0x7ff7c40a4580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7dc101d80 0x7ff7dc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7dc1013c0 0x7ff7dc1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 --2- 172.21.15.17:0/2681103394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7dc1001d0 0x7ff7dc197270 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 >> 172.21.15.17:0/2681103394 conn(0x7ff7dc0fb760 msgr2=0x7ff7dc069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 shutdown_connections 2024-02-14T15:00:54.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:54.919+0000 7ff7e3630640 1 -- 172.21.15.17:0/2681103394 wait complete. 2024-02-14T15:00:54.929 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:54 smithi017 ceph-mon[32549]: pgmap v292: 129 pgs: 19 creating+peering, 76 active+clean, 34 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-02-14T15:00:54.929 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:54 smithi017 ceph-mon[32549]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T15:00:54.929 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:54 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3179463323' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-14T15:00:54.929 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:54 smithi017 ceph-mon[32549]: osdmap e88: 12 total, 12 up, 12 in 2024-02-14T15:00:54.930 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:54 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/706813419' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-14T15:00:54.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:00:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:54 smithi076 ceph-mon[37057]: pgmap v292: 129 pgs: 19 creating+peering, 76 active+clean, 34 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-02-14T15:00:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:54 smithi076 ceph-mon[37057]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T15:00:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:54 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3179463323' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-14T15:00:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:54 smithi076 ceph-mon[37057]: osdmap e88: 12 total, 12 up, 12 in 2024-02-14T15:00:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:54 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/706813419' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-14T15:00:55.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 -- 172.21.15.17:0/2997312913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6341004a0 msgr2=0x7fd634100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 --2- 172.21.15.17:0/2997312913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6341004a0 0x7fd634100920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd624009930 tx=0x7fd62402bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:55.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 -- 172.21.15.17:0/2997312913 shutdown_connections 2024-02-14T15:00:55.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 --2- 172.21.15.17:0/2997312913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6341004a0 0x7fd634100920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 --2- 172.21.15.17:0/2997312913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd634100e60 0x7fd634108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 --2- 172.21.15.17:0/2997312913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd634101770 0x7fd634101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 -- 172.21.15.17:0/2997312913 >> 172.21.15.17:0/2997312913 conn(0x7fd6340faeb0 msgr2=0x7fd6340fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 -- 172.21.15.17:0/2997312913 shutdown_connections 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 -- 172.21.15.17:0/2997312913 wait complete. 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 Processor -- start 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.033+0000 7fd63c439640 1 -- start start 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6341004a0 0x7fd6340fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd634100e60 0x7fd63419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd634101770 0x7fd63419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419ee80 con 0x7fd634101770 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419eff0 con 0x7fd6341004a0 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419f1b0 con 0x7fd634100e60 2024-02-14T15:00:55.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd634101770 0x7fd63419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd634101770 0x7fd63419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38540/0 (socket says 172.21.15.17:38540) 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 -- 172.21.15.17:0/543951246 learned_addr learned my addr 172.21.15.17:0/543951246 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a1d5640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6341004a0 0x7fd6340fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd6399d4640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd634100e60 0x7fd63419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 -- 172.21.15.17:0/543951246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd634100e60 msgr2=0x7fd63419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd634100e60 0x7fd63419c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 -- 172.21.15.17:0/543951246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6341004a0 msgr2=0x7fd6340fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6341004a0 0x7fd6340fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 -- 172.21.15.17:0/543951246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd624009560 con 0x7fd634101770 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63a9d6640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd634101770 0x7fd63419e6a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009fd0 tx=0x7fd62c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd62b7fe640 1 -- 172.21.15.17:0/543951246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00bea0 con 0x7fd634101770 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd62b7fe640 1 -- 172.21.15.17:0/543951246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd62c009940 con 0x7fd634101770 2024-02-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63419f430 con 0x7fd634101770 2024-02-14T15:00:55.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd62b7fe640 1 -- 172.21.15.17:0/543951246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0106c0 con 0x7fd634101770 2024-02-14T15:00:55.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.034+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63419f910 con 0x7fd634101770 2024-02-14T15:00:55.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.035+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6341075e0 con 0x7fd634101770 2024-02-14T15:00:55.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.036+0000 7fd62b7fe640 1 -- 172.21.15.17:0/543951246 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd62c004240 con 0x7fd634101770 2024-02-14T15:00:55.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.037+0000 7fd62b7fe640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd6200a2140 0x7fd6200a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.037+0000 7fd63a1d5640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd6200a2140 0x7fd6200a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.037+0000 7fd62b7fe640 1 -- 172.21.15.17:0/543951246 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd62c0cdbf0 con 0x7fd634101770 2024-02-14T15:00:55.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.037+0000 7fd63a1d5640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd6200a2140 0x7fd6200a4600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd634069710 tx=0x7fd63000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:55.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.041+0000 7fd62b7fe640 1 -- 172.21.15.17:0/543951246 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd62c0969f0 con 0x7fd634101770 2024-02-14T15:00:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.224+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd63410bda0 con 0x7fd634101770 2024-02-14T15:00:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.225+0000 7fd62b7fe640 1 -- 172.21.15.17:0/543951246 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7fd62c017020 con 0x7fd634101770 2024-02-14T15:00:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:55.227 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:00:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd6200a2140 msgr2=0x7fd6200a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd6200a2140 0x7fd6200a4600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd634069710 tx=0x7fd63000b390 comp rx=0 tx=0).stop 2024-02-14T15:00:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd634101770 msgr2=0x7fd63419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd634101770 0x7fd63419e6a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009fd0 tx=0x7fd62c00b260 comp rx=0 tx=0).stop 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 shutdown_connections 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd6200a2140 0x7fd6200a4600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd634101770 0x7fd63419e6a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd634100e60 0x7fd63419c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 --2- 172.21.15.17:0/543951246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6341004a0 0x7fd6340fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 >> 172.21.15.17:0/543951246 conn(0x7fd6340faeb0 msgr2=0x7fd6341047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 shutdown_connections 2024-02-14T15:00:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.227+0000 7fd63c439640 1 -- 172.21.15.17:0/543951246 wait complete. 2024-02-14T15:00:55.240 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 -- 172.21.15.17:0/1732756244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 msgr2=0x7fe1801016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 --2- 172.21.15.17:0/1732756244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801016e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009ca0 tx=0x7fe17c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 -- 172.21.15.17:0/1732756244 shutdown_connections 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 --2- 172.21.15.17:0/1732756244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe180102e70 0x7fe180109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 --2- 172.21.15.17:0/1732756244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1801024b0 0x7fe180102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 --2- 172.21.15.17:0/1732756244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801016e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 -- 172.21.15.17:0/1732756244 >> 172.21.15.17:0/1732756244 conn(0x7fe1800fc850 msgr2=0x7fe1800fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 -- 172.21.15.17:0/1732756244 shutdown_connections 2024-02-14T15:00:55.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 -- 172.21.15.17:0/1732756244 wait complete. 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 Processor -- start 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 -- start start 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.341+0000 7fe187664640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe187664640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1801024b0 0x7fe180197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe187664640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe180102e70 0x7fe18019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe187664640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019edc0 con 0x7fe1801012c0 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe187664640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019ef30 con 0x7fe1801024b0 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe187664640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019f0f0 con 0x7fe180102e70 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38550/0 (socket says 172.21.15.17:38550) 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 -- 172.21.15.17:0/3793013712 learned_addr learned my addr 172.21.15.17:0/3793013712 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe184bff640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1801024b0 0x7fe180197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185c01640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe180102e70 0x7fe18019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 -- 172.21.15.17:0/3793013712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe180102e70 msgr2=0x7fe18019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe180102e70 0x7fe18019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 -- 172.21.15.17:0/3793013712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1801024b0 msgr2=0x7fe180197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1801024b0 0x7fe180197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 -- 172.21.15.17:0/3793013712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c009900 con 0x7fe1801012c0 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185c01640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe180102e70 0x7fe18019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe185400640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801972d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe17c037040 tx=0x7fe17c034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe1767fc640 1 -- 172.21.15.17:0/3793013712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c032eb0 con 0x7fe1801012c0 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18019f370 con 0x7fe1801012c0 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe1767fc640 1 -- 172.21.15.17:0/3793013712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe17c003e50 con 0x7fe1801012c0 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe1767fc640 1 -- 172.21.15.17:0/3793013712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c040470 con 0x7fe1801012c0 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.342+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18019f870 con 0x7fe1801012c0 2024-02-14T15:00:55.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.343+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe180108660 con 0x7fe1801012c0 2024-02-14T15:00:55.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.344+0000 7fe1767fc640 1 -- 172.21.15.17:0/3793013712 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe17c00a040 con 0x7fe1801012c0 2024-02-14T15:00:55.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.345+0000 7fe1767fc640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe16c0a20c0 0x7fe16c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.345+0000 7fe184bff640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe16c0a20c0 0x7fe16c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.345+0000 7fe1767fc640 1 -- 172.21.15.17:0/3793013712 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe17c0f0e00 con 0x7fe1801012c0 2024-02-14T15:00:55.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.345+0000 7fe184bff640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe16c0a20c0 0x7fe16c0a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe1801987e0 tx=0x7fe17000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:55.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.349+0000 7fe1767fc640 1 -- 172.21.15.17:0/3793013712 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe17c0b9b80 con 0x7fe1801012c0 2024-02-14T15:00:55.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.531+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe18002cdb0 con 0x7fe1801012c0 2024-02-14T15:00:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.533+0000 7fe1767fc640 1 -- 172.21.15.17:0/3793013712 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7fe17c0bd240 con 0x7fe1801012c0 2024-02-14T15:00:55.535 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:55.535 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:00:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.534+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe16c0a20c0 msgr2=0x7fe16c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe16c0a20c0 0x7fe16c0a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe1801987e0 tx=0x7fe17000b390 comp rx=0 tx=0).stop 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 msgr2=0x7fe1801972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801972d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe17c037040 tx=0x7fe17c034d40 comp rx=0 tx=0).stop 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 shutdown_connections 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe16c0a20c0 0x7fe16c0a4580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe180102e70 0x7fe18019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1801024b0 0x7fe180197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 --2- 172.21.15.17:0/3793013712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1801012c0 0x7fe1801972d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 >> 172.21.15.17:0/3793013712 conn(0x7fe1800fc850 msgr2=0x7fe180105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 shutdown_connections 2024-02-14T15:00:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.535+0000 7fe187664640 1 -- 172.21.15.17:0/3793013712 wait complete. 2024-02-14T15:00:55.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 -- 172.21.15.17:0/466158675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 msgr2=0x7fda84104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 --2- 172.21.15.17:0/466158675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda84104140 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009ca0 tx=0x7fda7c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 -- 172.21.15.17:0/466158675 shutdown_connections 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 --2- 172.21.15.17:0/466158675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda84106fb0 0x7fda84109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 --2- 172.21.15.17:0/466158675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda84104680 0x7fda84106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 --2- 172.21.15.17:0/466158675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda84104140 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 -- 172.21.15.17:0/466158675 >> 172.21.15.17:0/466158675 conn(0x7fda840fb740 msgr2=0x7fda840fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:55.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 -- 172.21.15.17:0/466158675 shutdown_connections 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.666+0000 7fda8bde2640 1 -- 172.21.15.17:0/466158675 wait complete. 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 Processor -- start 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 -- start start 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda841016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda84104680 0x7fda840ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda84106fb0 0x7fda84100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841009a0 con 0x7fda84101d50 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda84100b10 con 0x7fda84106fb0 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8bde2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841a12f0 con 0x7fda84104680 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda841016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda841016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38570/0 (socket says 172.21.15.17:38570) 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 -- 172.21.15.17:0/2464870699 learned_addr learned my addr 172.21.15.17:0/2464870699 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8a37f640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda84106fb0 0x7fda84100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8937d640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda84104680 0x7fda840ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 -- 172.21.15.17:0/2464870699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda84104680 msgr2=0x7fda840ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda84104680 0x7fda840ffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 -- 172.21.15.17:0/2464870699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda84106fb0 msgr2=0x7fda84100240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda84106fb0 0x7fda84100240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 -- 172.21.15.17:0/2464870699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c009900 con 0x7fda84101d50 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8a37f640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda84106fb0 0x7fda84100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda8937d640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda84104680 0x7fda840ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda89b7e640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda841016c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009dc0 tx=0x7fda7c034e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.667+0000 7fda82ffd640 1 -- 172.21.15.17:0/2464870699 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c003970 con 0x7fda84101d50 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.668+0000 7fda82ffd640 1 -- 172.21.15.17:0/2464870699 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda7c004040 con 0x7fda84101d50 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.668+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda841a1540 con 0x7fda84101d50 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.668+0000 7fda82ffd640 1 -- 172.21.15.17:0/2464870699 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0406b0 con 0x7fda84101d50 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.668+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda841a19f0 con 0x7fda84101d50 2024-02-14T15:00:55.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.668+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda840fd230 con 0x7fda84101d50 2024-02-14T15:00:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.670+0000 7fda82ffd640 1 -- 172.21.15.17:0/2464870699 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fda7c00a040 con 0x7fda84101d50 2024-02-14T15:00:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.671+0000 7fda82ffd640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fda6c0a20c0 0x7fda6c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.671+0000 7fda8937d640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fda6c0a20c0 0x7fda6c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.671+0000 7fda82ffd640 1 -- 172.21.15.17:0/2464870699 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fda7c0f1560 con 0x7fda84101d50 2024-02-14T15:00:55.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.671+0000 7fda8937d640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fda6c0a20c0 0x7fda6c0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda84101430 tx=0x7fda78005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:55.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.674+0000 7fda82ffd640 1 -- 172.21.15.17:0/2464870699 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda7c0ba3d0 con 0x7fda84101d50 2024-02-14T15:00:55.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:55 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/706813419' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-14T15:00:55.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:55 smithi050 ceph-mon[36512]: osdmap e89: 12 total, 12 up, 12 in 2024-02-14T15:00:55.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:55 smithi050 ceph-mon[36512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:55.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:55 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2681103394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:55 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/543951246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:55 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3793013712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.893+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda8410cc70 con 0x7fda84101d50 2024-02-14T15:00:55.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.894+0000 7fda82ffd640 1 -- 172.21.15.17:0/2464870699 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11256 (secure 0 0 0) 0x7fda7c0bda90 con 0x7fda84101d50 2024-02-14T15:00:55.896 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:55.896 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:00:53.931986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-14T15:00:55.896 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fda6c0a20c0 msgr2=0x7fda6c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fda6c0a20c0 0x7fda6c0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda84101430 tx=0x7fda78005d30 comp rx=0 tx=0).stop 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 msgr2=0x7fda841016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda841016c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009dc0 tx=0x7fda7c034e80 comp rx=0 tx=0).stop 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 shutdown_connections 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fda6c0a20c0 0x7fda6c0a4580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda84106fb0 0x7fda84100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda84104680 0x7fda840ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 --2- 172.21.15.17:0/2464870699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda84101d50 0x7fda841016c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 >> 172.21.15.17:0/2464870699 conn(0x7fda840fb740 msgr2=0x7fda84102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 shutdown_connections 2024-02-14T15:00:55.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:55.896+0000 7fda8bde2640 1 -- 172.21.15.17:0/2464870699 wait complete. 2024-02-14T15:00:55.907 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:55 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/706813419' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-14T15:00:55.907 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:55 smithi017 ceph-mon[32549]: osdmap e89: 12 total, 12 up, 12 in 2024-02-14T15:00:55.907 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:55 smithi017 ceph-mon[32549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:55.907 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:55 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2681103394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.908 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:55 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/543951246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.908 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:55 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3793013712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.909 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:00:55.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:55 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/706813419' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-14T15:00:55.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:55 smithi076 ceph-mon[37057]: osdmap e89: 12 total, 12 up, 12 in 2024-02-14T15:00:55.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:55 smithi076 ceph-mon[37057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:00:55.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:55 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2681103394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:55 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/543951246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:55.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:55 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3793013712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/2735217199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 msgr2=0x7fdd141005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/2735217199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141005d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009900 tx=0x7fdd1002bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/2735217199 shutdown_connections 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/2735217199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd14101d60 0x7fdd141085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/2735217199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd141013a0 0x7fdd14101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/2735217199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141005d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/2735217199 >> 172.21.15.17:0/2735217199 conn(0x7fdd140fb740 msgr2=0x7fdd140fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/2735217199 shutdown_connections 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.009+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/2735217199 wait complete. 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 Processor -- start 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 -- start start 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd141013a0 0x7fdd141966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd14101d60 0x7fdd1419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1419dc90 con 0x7fdd141001b0 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1419de00 con 0x7fdd14101d60 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1bd1e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1419dfc0 con 0x7fdd141013a0 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd19aba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd19aba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38588/0 (socket says 172.21.15.17:38588) 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd19aba640 1 -- 172.21.15.17:0/4237075207 learned_addr learned my addr 172.21.15.17:0/4237075207 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd1a2bb640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd14101d60 0x7fdd1419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd192b9640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd141013a0 0x7fdd141966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.010+0000 7fdd19aba640 1 -- 172.21.15.17:0/4237075207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd141013a0 msgr2=0x7fdd141966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd19aba640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd141013a0 0x7fdd141966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd19aba640 1 -- 172.21.15.17:0/4237075207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd14101d60 msgr2=0x7fdd1419d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd19aba640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd14101d60 0x7fdd1419d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd19aba640 1 -- 172.21.15.17:0/4237075207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10009560 con 0x7fdd141001b0 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd19aba640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141961a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005ab0 tx=0x7fdd10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd0affd640 1 -- 172.21.15.17:0/4237075207 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10004aa0 con 0x7fdd141001b0 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1419e240 con 0x7fdd141001b0 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd0affd640 1 -- 172.21.15.17:0/4237075207 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd100052d0 con 0x7fdd141001b0 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd0affd640 1 -- 172.21.15.17:0/4237075207 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10003e80 con 0x7fdd141001b0 2024-02-14T15:00:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.011+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1419e750 con 0x7fdd141001b0 2024-02-14T15:00:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.012+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd141074d0 con 0x7fdd141001b0 2024-02-14T15:00:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.013+0000 7fdd0affd640 1 -- 172.21.15.17:0/4237075207 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdd10005000 con 0x7fdd141001b0 2024-02-14T15:00:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.014+0000 7fdd0affd640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd000a20c0 0x7fdd000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.014+0000 7fdd192b9640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd000a20c0 0x7fdd000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.014+0000 7fdd0affd640 1 -- 172.21.15.17:0/4237075207 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdd100f0960 con 0x7fdd141001b0 2024-02-14T15:00:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.014+0000 7fdd192b9640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd000a20c0 0x7fdd000a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdd141976b0 tx=0x7fdd04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.018+0000 7fdd0affd640 1 -- 172.21.15.17:0/4237075207 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd100b9740 con 0x7fdd141001b0 2024-02-14T15:00:56.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.199+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd1402ce50 con 0x7fdd141001b0 2024-02-14T15:00:56.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.200+0000 7fdd0affd640 1 -- 172.21.15.17:0/4237075207 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7fdd100bce00 con 0x7fdd141001b0 2024-02-14T15:00:56.203 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:00:56.203 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:00:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd000a20c0 msgr2=0x7fdd000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd000a20c0 0x7fdd000a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdd141976b0 tx=0x7fdd04005f90 comp rx=0 tx=0).stop 2024-02-14T15:00:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 msgr2=0x7fdd141961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141961a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005ab0 tx=0x7fdd10004270 comp rx=0 tx=0).stop 2024-02-14T15:00:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 shutdown_connections 2024-02-14T15:00:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd000a20c0 0x7fdd000a4580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd14101d60 0x7fdd1419d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd141013a0 0x7fdd141966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 --2- 172.21.15.17:0/4237075207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd141001b0 0x7fdd141961a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 >> 172.21.15.17:0/4237075207 conn(0x7fdd140fb740 msgr2=0x7fdd141046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 shutdown_connections 2024-02-14T15:00:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.202+0000 7fdd1bd1e640 1 -- 172.21.15.17:0/4237075207 wait complete. 2024-02-14T15:00:56.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-14T15:00:48.619843+0000', 'modified': '2024-02-14T15:00:53.931986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6839/916171404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 916171404}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 916171404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-02-14T15:00:56.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-14T15:00:56.216 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-14T15:00:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 -- 172.21.15.17:0/115763822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c101340 msgr2=0x7f9c6c1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 --2- 172.21.15.17:0/115763822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c101340 0x7f9c6c1017c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009900 tx=0x7f9c5c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:00:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 -- 172.21.15.17:0/115763822 shutdown_connections 2024-02-14T15:00:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 --2- 172.21.15.17:0/115763822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c101340 0x7f9c6c1017c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 --2- 172.21.15.17:0/115763822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c101d00 0x7f9c6c108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 --2- 172.21.15.17:0/115763822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6c100150 0x7f9c6c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 -- 172.21.15.17:0/115763822 >> 172.21.15.17:0/115763822 conn(0x7f9c6c0fb6c0 msgr2=0x7f9c6c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 -- 172.21.15.17:0/115763822 shutdown_connections 2024-02-14T15:00:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.319+0000 7f9c7249a640 1 -- 172.21.15.17:0/115763822 wait complete. 2024-02-14T15:00:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 Processor -- start 2024-02-14T15:00:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 -- start start 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c100150 0x7f9c6c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6c101340 0x7f9c6c197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c101d00 0x7f9c6c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c19edc0 con 0x7f9c6c100150 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c19ef30 con 0x7f9c6c101d00 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c7249a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c19f0f0 con 0x7f9c6c101340 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c100150 0x7f9c6c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c100150 0x7f9c6c1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38608/0 (socket says 172.21.15.17:38608) 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 -- 172.21.15.17:0/998118745 learned_addr learned my addr 172.21.15.17:0/998118745 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6c101340 0x7f9c6c197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 -- 172.21.15.17:0/998118745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6c101340 msgr2=0x7f9c6c197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6c101340 0x7f9c6c197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 -- 172.21.15.17:0/998118745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c101d00 msgr2=0x7f9c6c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c101d00 0x7f9c6c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6bfff640 1 -- 172.21.15.17:0/998118745 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c009560 con 0x7f9c6c100150 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.320+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6c101340 0x7f9c6c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.321+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c100150 0x7f9c6c1972d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d950 tx=0x7f9c5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.321+0000 7f9c697fa640 1 -- 172.21.15.17:0/998118745 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54009c30 con 0x7f9c6c100150 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.321+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c19f3d0 con 0x7f9c6c100150 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.321+0000 7f9c697fa640 1 -- 172.21.15.17:0/998118745 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c54009d90 con 0x7f9c6c100150 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.321+0000 7f9c697fa640 1 -- 172.21.15.17:0/998118745 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400f720 con 0x7f9c6c100150 2024-02-14T15:00:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.321+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c19f890 con 0x7f9c6c100150 2024-02-14T15:00:56.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.323+0000 7f9c697fa640 1 -- 172.21.15.17:0/998118745 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9c54010a60 con 0x7f9c6c100150 2024-02-14T15:00:56.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.323+0000 7f9c7506d740 1 Processor -- start 2024-02-14T15:00:56.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.323+0000 7f9c697fa640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9c580a20c0 0x7f9c580a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.323+0000 7f9c7506d740 1 -- start start 2024-02-14T15:00:56.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.323+0000 7f9c70a37640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9c580a20c0 0x7f9c580a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c697fa640 1 -- 172.21.15.17:0/998118745 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c540cdcf0 con 0x7f9c6c100150 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 0x56363b0c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0cb3d0 0x56363b0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c7506d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b07dfd0 con 0x56363b0cb3d0 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c7506d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b030cb0 con 0x56363b0c6260 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c7506d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b02e1e0 con 0x56363b0c5870 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0cb3d0 0x56363b0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0cb3d0 0x56363b0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38622/0 (socket says 172.21.15.17:38622) 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 -- 172.21.15.17:0/3243475765 learned_addr learned my addr 172.21.15.17:0/3243475765 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 0x56363b0c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9c580a20c0 0x7f9c580a4580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9c640135f0 tx=0x7f9c640139f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 -- 172.21.15.17:0/3243475765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 msgr2=0x56363b0c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 0x56363b0c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 -- 172.21.15.17:0/3243475765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 msgr2=0x56363b0cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 -- 172.21.15.17:0/3243475765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56363afec100 con 0x56363b0cb3d0 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c70a37640 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0cb3d0 0x56363b0cd860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c640106c0 tx=0x7f9c6401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c08b6ce9013306f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c627fc640 1 -- 172.21.15.17:0/3243475765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64004b20 con 0x56363b0cb3d0 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c627fc640 1 -- 172.21.15.17:0/3243475765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c64004c80 con 0x56363b0cb3d0 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.324+0000 7f9c627fc640 1 -- 172.21.15.17:0/3243475765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c640053a0 con 0x56363b0cb3d0 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 -- 172.21.15.17:0/3243475765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0cb3d0 msgr2=0x56363b0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0cb3d0 0x56363b0cd860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c640106c0 tx=0x7f9c6401a000 comp rx=0 tx=0).stop 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 -- 172.21.15.17:0/3243475765 shutdown_connections 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0cb3d0 0x56363b0cd860 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3243475765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 0x56363b0c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 -- 172.21.15.17:0/3243475765 >> 172.21.15.17:0/3243475765 conn(0x56363b0bc9d0 msgr2=0x56363b0bcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 -- 172.21.15.17:0/3243475765 shutdown_connections 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 -- 172.21.15.17:0/3243475765 wait complete. 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.325+0000 7f9c7506d740 1 -- start start 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 1 Processor -- start 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 0x56363b0d6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b0299c0 con 0x56363b0c5870 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363afcecd0 con 0x56363b0c6260 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363acfd310 con 0x56363b0cb3d0 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c7506d740 10 client.0 _ll_register_callbacks cb 0x56363ad3aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38632/0 (socket says 172.21.15.17:38632) 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 -- 172.21.15.17:0/2628655986 learned_addr learned my addr 172.21.15.17:0/2628655986 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c70a37640 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 0x56363b0d6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 -- 172.21.15.17:0/2628655986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 msgr2=0x56363b0d6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 0x56363b0d6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 -- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 msgr2=0x56363b0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 -- 172.21.15.17:0/2628655986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c64019070 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0cf930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9c540964b0 tx=0x7f9c5409aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238adb77d4723ecc server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.326+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/2628655986 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5409e920 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56363ad296c0 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/2628655986 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c540a2f70 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/2628655986 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c54096e40 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 10 client.14787 fetch_fsmap learned FSMap version 17 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56363b028040 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/2628655986 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56363ad296c0 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/2628655986 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9c5409bb00 con 0x56363b0c5870 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 10 client.14787 fetch_fsmap finished waiting for FSMap version 17 2024-02-14T15:00:56.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 10 client.14787 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 20 client.14787 populate_metadata read hostname 'smithi017' 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b0dbf80 0x56363b0db250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 4 client.14787 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.327+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56363ad7a270 con 0x56363b0dbf80 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.328+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b0dbf80 0x56363b0db250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.328+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b0dbf80 0x56363b0db250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e75265dc0b737d server_cookie=e24f0f00df8b51ea in_seq=0 out_seq=0 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.328+0000 7f9c4b7fe640 10 client.14787 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/2628655986 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56363ad7a270 con 0x56363b0dbf80 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c4b7fe640 10 client.14787 handle_command_reply: tid=0 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 client.14787 shutdown 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 msgr2=0x56363b0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0cf930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9c540964b0 tx=0x7f9c5409aed0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 shutdown_connections 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b0dbf80 0x56363b0db250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 0x56363b0d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c4b7fe640 0 client.14787 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:00:56.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 --2- 172.21.15.17:0/2628655986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0cf930 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 >> 172.21.15.17:0/2628655986 conn(0x56363b0bc9d0 msgr2=0x56363b0bbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 shutdown_connections 2024-02-14T15:00:56.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 1 -- 172.21.15.17:0/2628655986 wait complete. 2024-02-14T15:00:56.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.329+0000 7f9c7506d740 20 client.14787 trim_cache size 0 max 16384 2024-02-14T15:00:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 Processor -- start 2024-02-14T15:00:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 -- start start 2024-02-14T15:00:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 0x56363b0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c6260 0x56363b0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0cb3d0 0x56363b0d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363afaf490 con 0x56363b0c6260 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b0caef0 con 0x56363b0cb3d0 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c7506d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b0ddf60 con 0x56363b0c5870 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c6260 0x56363b0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c6260 0x56363b0c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38646/0 (socket says 172.21.15.17:38646) 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 -- 172.21.15.17:0/1519990694 learned_addr learned my addr 172.21.15.17:0/1519990694 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c70a37640 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0cb3d0 0x56363b0d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 -- 172.21.15.17:0/1519990694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 msgr2=0x56363b0c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 0x56363b0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 -- 172.21.15.17:0/1519990694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0cb3d0 msgr2=0x56363b0d6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0cb3d0 0x56363b0d6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 -- 172.21.15.17:0/1519990694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56363afc4a90 con 0x56363b0c6260 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c6260 0x56363b0c7bc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c02c170 tx=0x7f9c5c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d15bc8247c0072e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.345+0000 7f9c61ffb640 1 -- 172.21.15.17:0/1519990694 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c046de0 con 0x56363b0c6260 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c61ffb640 1 -- 172.21.15.17:0/1519990694 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c004040 con 0x56363b0c6260 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c61ffb640 1 -- 172.21.15.17:0/1519990694 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0467c0 con 0x56363b0c6260 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 -- 172.21.15.17:0/1519990694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c6260 msgr2=0x56363b0c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c6260 0x56363b0c7bc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c02c170 tx=0x7f9c5c005e00 comp rx=0 tx=0).stop 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 -- 172.21.15.17:0/1519990694 shutdown_connections 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c6260 0x56363b0c7bc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0cb3d0 0x56363b0d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 --2- 172.21.15.17:0/1519990694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0c5870 0x56363b0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 -- 172.21.15.17:0/1519990694 >> 172.21.15.17:0/1519990694 conn(0x56363b0bc9d0 msgr2=0x56363b0bcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 -- 172.21.15.17:0/1519990694 shutdown_connections 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 -- 172.21.15.17:0/1519990694 wait complete. 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.346+0000 7f9c7506d740 1 -- start start 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 1 Processor -- start 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 0x56363b0d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b0d5fd0 con 0x56363b0c5870 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b0d6140 con 0x56363b0c6260 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363b0d6300 con 0x56363b0cb3d0 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c7506d740 10 client.0 _ll_register_callbacks cb 0x7f9c64007a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0d4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38656/0 (socket says 172.21.15.17:38656) 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 -- 172.21.15.17:0/3258310240 learned_addr learned my addr 172.21.15.17:0/3258310240 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 -- 172.21.15.17:0/3258310240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 msgr2=0x56363b0d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 0x56363b0d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 -- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 msgr2=0x56363b0d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 -- 172.21.15.17:0/3258310240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c108e90 con 0x56363b0c5870 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c6bfff640 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0d4540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c5409a760 tx=0x7f9c54005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfb5a61c9ebba84 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:56.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.347+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/3258310240 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5409a130 con 0x56363b0c5870 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/3258310240 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c540a2f70 con 0x56363b0c5870 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56363b0dcfb0 con 0x56363b0c5870 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/3258310240 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c5409e920 con 0x56363b0c5870 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 10 client.14793 fetch_fsmap learned FSMap version 17 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56363afea960 con 0x56363b0c5870 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/3258310240 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56363b0dcfb0 con 0x56363b0c5870 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/3258310240 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9c54099690 con 0x56363b0c5870 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 10 client.14793 fetch_fsmap finished waiting for FSMap version 17 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 10 client.14793 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 20 client.14793 populate_metadata read hostname 'smithi017' 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b137990 0x56363b133de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 4 client.14793 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9c64004c80 con 0x56363b137990 2024-02-14T15:00:56.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.348+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b137990 0x56363b133de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c6b7fe640 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b137990 0x56363b133de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f8485f66d13613 server_cookie=1b65cdfc83d61e3e in_seq=0 out_seq=0 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c4b7fe640 10 client.14793 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c4b7fe640 1 -- 172.21.15.17:0/3258310240 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7f9c64004c80 con 0x56363b137990 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c4b7fe640 10 client.14793 handle_command_reply: tid=0 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 client.14793 shutdown 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 msgr2=0x56363b0d4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0d4540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c5409a760 tx=0x7f9c54005f30 comp rx=0 tx=0).stop 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 shutdown_connections 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56363b137990 0x56363b133de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56363b0cb3d0 0x56363b0d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c4b7fe640 0 client.14793 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363b0c6260 0x56363b0d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 --2- 172.21.15.17:0/3258310240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56363b0c5870 0x56363b0d4540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 >> 172.21.15.17:0/3258310240 conn(0x56363b0bc9d0 msgr2=0x56363b0c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 shutdown_connections 2024-02-14T15:00:56.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 1 -- 172.21.15.17:0/3258310240 wait complete. 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.349+0000 7f9c7506d740 20 client.14793 trim_cache size 0 max 16384 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24509, 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 7.0858923069999999, 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 17, 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 89, 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 88, 2024-02-14T15:00:56.357 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 13.194626518 2024-02-14T15:00:56.358 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:00:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9c580a20c0 msgr2=0x7f9c580a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9c580a20c0 0x7f9c580a4580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9c640135f0 tx=0x7f9c640139f0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c100150 msgr2=0x7f9c6c1972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c100150 0x7f9c6c1972d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d950 tx=0x7f9c5400de10 comp rx=0 tx=0).stop 2024-02-14T15:00:56.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 shutdown_connections 2024-02-14T15:00:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9c580a20c0 0x7f9c580a4580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c6c101d00 0x7f9c6c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6c101340 0x7f9c6c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 --2- 172.21.15.17:0/998118745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6c100150 0x7f9c6c1972d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 >> 172.21.15.17:0/998118745 conn(0x7f9c6c0fb6c0 msgr2=0x7f9c6c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 shutdown_connections 2024-02-14T15:00:56.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:56.350+0000 7f9c7249a640 1 -- 172.21.15.17:0/998118745 wait complete. 2024-02-14T15:00:56.362 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-14T15:00:56.362 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:00:56.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph fs ls 2024-02-14T15:00:56.602 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:00:56.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:56 smithi017 ceph-mon[32549]: pgmap v295: 129 pgs: 4 creating+peering, 106 active+clean, 19 unknown; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2024-02-14T15:00:56.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:56 smithi017 ceph-mon[32549]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T15:00:56.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:56 smithi017 ceph-mon[32549]: Cluster is now healthy 2024-02-14T15:00:56.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:56 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2464870699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:56.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:56 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4237075207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:56 smithi050 ceph-mon[36512]: pgmap v295: 129 pgs: 4 creating+peering, 106 active+clean, 19 unknown; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2024-02-14T15:00:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:56 smithi050 ceph-mon[36512]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T15:00:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:56 smithi050 ceph-mon[36512]: Cluster is now healthy 2024-02-14T15:00:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:56 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2464870699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:56.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:56 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4237075207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:56 smithi076 ceph-mon[37057]: pgmap v295: 129 pgs: 4 creating+peering, 106 active+clean, 19 unknown; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2024-02-14T15:00:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:56 smithi076 ceph-mon[37057]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T15:00:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:56 smithi076 ceph-mon[37057]: Cluster is now healthy 2024-02-14T15:00:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:56 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2464870699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:00:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:56 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4237075207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 -- 172.21.15.17:0/1083638689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 msgr2=0x7fe1a81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 --2- 172.21.15.17:0/1083638689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a81021f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe198009990 tx=0x7fe198009ca0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 -- 172.21.15.17:0/1083638689 shutdown_connections 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 --2- 172.21.15.17:0/1083638689 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1a8102730 0x7fe1a810ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 --2- 172.21.15.17:0/1083638689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1a8107e20 0x7fe1a8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 --2- 172.21.15.17:0/1083638689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a81021f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 -- 172.21.15.17:0/1083638689 >> 172.21.15.17:0/1083638689 conn(0x7fe1a8078e50 msgr2=0x7fe1a8079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 -- 172.21.15.17:0/1083638689 shutdown_connections 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.272+0000 7fe1af569700 1 -- 172.21.15.17:0/1083638689 wait complete. 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 Processor -- start 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 -- start start 2024-02-14T15:00:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1a8102730 0x7fe1a8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1a8107e20 0x7fe1a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81a66a0 con 0x7fe1a8101d70 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81a6810 con 0x7fe1a8107e20 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1af569700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81a69d0 con 0x7fe1a8102730 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1ad305700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1ad305700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a8074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38672/0 (socket says 172.21.15.17:38672) 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1ad305700 1 -- 172.21.15.17:0/3029602454 learned_addr learned my addr 172.21.15.17:0/3029602454 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1adb06700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1a8107e20 0x7fe1a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.273+0000 7fe1acb04700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1a8102730 0x7fe1a8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1ad305700 1 -- 172.21.15.17:0/3029602454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1a8102730 msgr2=0x7fe1a8074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1ad305700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1a8102730 0x7fe1a8074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1ad305700 1 -- 172.21.15.17:0/3029602454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1a8107e20 msgr2=0x7fe1a8074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1ad305700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1a8107e20 0x7fe1a8074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1ad305700 1 -- 172.21.15.17:0/3029602454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980095f0 con 0x7fe1a8101d70 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1adb06700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1a8107e20 0x7fe1a8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1ad305700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a8074020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400b530 tx=0x7fe1a400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe19e7fc700 1 -- 172.21.15.17:0/3029602454 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4011830 con 0x7fe1a8101d70 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe19e7fc700 1 -- 172.21.15.17:0/3029602454 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1a4011e70 con 0x7fe1a8101d70 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a81a6cb0 con 0x7fe1a8101d70 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe19e7fc700 1 -- 172.21.15.17:0/3029602454 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a400f2e0 con 0x7fe1a8101d70 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.274+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a81a71b0 con 0x7fe1a8101d70 2024-02-14T15:00:58.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.275+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a810a290 con 0x7fe1a8101d70 2024-02-14T15:00:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.277+0000 7fe19e7fc700 1 -- 172.21.15.17:0/3029602454 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe1a40105f0 con 0x7fe1a8101d70 2024-02-14T15:00:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.277+0000 7fe19e7fc700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1940a21d0 0x7fe1940a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:00:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.277+0000 7fe1acb04700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1940a21d0 0x7fe1940a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:00:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.277+0000 7fe19e7fc700 1 -- 172.21.15.17:0/3029602454 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe1a40cddc0 con 0x7fe1a8101d70 2024-02-14T15:00:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.278+0000 7fe1acb04700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1940a21d0 0x7fe1940a4690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe198006010 tx=0x7fe19801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:00:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.281+0000 7fe19e7fc700 1 -- 172.21.15.17:0/3029602454 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1a4096ba0 con 0x7fe1a8101d70 2024-02-14T15:00:58.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.530+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1a80689e0 con 0x7fe1a8101d70 2024-02-14T15:00:58.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.531+0000 7fe19e7fc700 1 -- 172.21.15.17:0/3029602454 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe1a4015020 con 0x7fe1a8101d70 2024-02-14T15:00:58.533 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1940a21d0 msgr2=0x7fe1940a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1940a21d0 0x7fe1940a4690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe198006010 tx=0x7fe19801a040 comp rx=0 tx=0).stop 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 msgr2=0x7fe1a8074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a8074020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400b530 tx=0x7fe1a400b8f0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 shutdown_connections 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1940a21d0 0x7fe1940a4690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1a8107e20 0x7fe1a8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1a8101d70 0x7fe1a8074020 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 --2- 172.21.15.17:0/3029602454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1a8102730 0x7fe1a8074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 >> 172.21.15.17:0/3029602454 conn(0x7fe1a8078e50 msgr2=0x7fe1a8100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 shutdown_connections 2024-02-14T15:00:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:00:58.533+0000 7fe1af569700 1 -- 172.21.15.17:0/3029602454 wait complete. 2024-02-14T15:00:58.851 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:58 smithi017 ceph-mon[32549]: pgmap v296: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-02-14T15:00:58.851 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:00:58 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3029602454' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-14T15:00:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:58 smithi050 ceph-mon[36512]: pgmap v296: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-02-14T15:00:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:00:58 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3029602454' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-14T15:00:58.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:58 smithi076 ceph-mon[37057]: pgmap v296: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-02-14T15:00:58.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:00:58 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3029602454' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-14T15:00:59.163 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-14T15:00:59.163 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-14T15:00:59.164 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-02-14T15:00:59.164 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-02-14T15:00:59.164 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:00:59.164 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-14T15:00:59.164 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-14T15:00:59.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-14T15:00:59.164 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-02-14T15:00:59.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.165 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-14T15:00:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42218sec preferred_lft 42218sec 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:00:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-02-14T15:00:59.183 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-14T15:00:59.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.183 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:00:59.183 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:00:59.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-02-14T15:00:59.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-02-14T15:00:59.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-02-14T15:00:59.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-14T15:00:59.183 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:00:59.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.357 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-14T15:00:59.382 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-02-14T15:00:59.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.384 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-02-14T15:00:59.453 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-02-14T15:00:59.453 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-14T15:00:59.453 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-14T15:00:59.453 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-14T15:00:59.454 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-14T15:00:59.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.454 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:00:59.454 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:00:59.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-14T15:00:59.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-14T15:00:59.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-14T15:00:59.454 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:00:59.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.734 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-02-14T15:00:59.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.749 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-02-14T15:00:59.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.803 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:00:59.803 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:00:59.803 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-14T15:00:59.803 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-02-14T15:00:59.803 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:00:59.886 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-02-14T15:00:59.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-02-14T15:00:59.886 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:01:00.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:00.026 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:01:00.026 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:01:00.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-02-14T15:01:00.027 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-02-14T15:01:00.027 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:01:00.068 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-02-14T15:01:00.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-14T15:01:00.068 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:01:00.126 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-02-14T15:01:00.126 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-14T15:01:00.126 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:01:00.180 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-02-14T15:01:00.247 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/selinux 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-02-14T15:01:00.303 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysctl.service 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/nfs/rpc_pipefs 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay 2024-02-14T15:01:00.304 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/8b6734776db8aef51266f2a3d0160e888bd04b69e83d997f0b31f8a4cedcb4b3/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/d861576df424177cb8cdc863c94f9701ada21fe5141715de2dea90bb781e6154/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/0 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/0ae06949177ecab0ecef87cec4209170f8d5c28ba2f7e634c5a6cb40cc7c3397/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/6408aba98fd886742fd814fca128d424f64488294446768d78578460008dbbdb/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/4e9fe037f9ab03d830a6fdae261e5069fc336b2cf621fb68f8cc164f0ed07d46/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/2ff86006cb0ff0a4bff2d8ba8758700c1593c538c6cc4b4a28882bd157042932/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/b2ba981a99196d8d0ad5a545ddf127e148a9bda34e278ef7ac21cbbc5d408d10/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/b2cf7369bec566ea5e6aa214e613209caaa13d4039ec91ce165a4a27a777ed9c/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/4ea4454b7f04bb88c0089ee77463bb00245da1f898860442535e18f5fc143d18/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/0a56b646ee5b8ed0c5a636ceb609c05fd1c4a2c2a2d11313967f6ba76da383ed/merged 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-14T15:01:00.305 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-14T15:01:00.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:00.306 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-02-14T15:01:00.357 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-14T15:01:00.357 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-02-14T15:01:00.399 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-02-14T15:01:00.425 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/selinux 2024-02-14T15:01:00.482 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysctl.service 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/nfs/rpc_pipefs 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/8b6734776db8aef51266f2a3d0160e888bd04b69e83d997f0b31f8a4cedcb4b3/merged 2024-02-14T15:01:00.483 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/d861576df424177cb8cdc863c94f9701ada21fe5141715de2dea90bb781e6154/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/0 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/0ae06949177ecab0ecef87cec4209170f8d5c28ba2f7e634c5a6cb40cc7c3397/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/6408aba98fd886742fd814fca128d424f64488294446768d78578460008dbbdb/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/4e9fe037f9ab03d830a6fdae261e5069fc336b2cf621fb68f8cc164f0ed07d46/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/2ff86006cb0ff0a4bff2d8ba8758700c1593c538c6cc4b4a28882bd157042932/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/b2ba981a99196d8d0ad5a545ddf127e148a9bda34e278ef7ac21cbbc5d408d10/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/b2cf7369bec566ea5e6aa214e613209caaa13d4039ec91ce165a4a27a777ed9c/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/4ea4454b7f04bb88c0089ee77463bb00245da1f898860442535e18f5fc143d18/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/containers/storage/overlay/0a56b646ee5b8ed0c5a636ceb609c05fd1c4a2c2a2d11313967f6ba76da383ed/merged 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-14T15:01:00.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:00.485 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-02-14T15:01:00.498 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:2024-02-14T15:01:00.496+0000 7f0553c93480 -1 init, newargv = 0x55cb45acbf10 newargc=15 2024-02-14T15:01:00.498 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[76306]: starting ceph client 2024-02-14T15:01:00.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[76306]: starting fuse 2024-02-14T15:01:00.512 INFO:teuthology.orchestra.run.smithi017.stdout:119 2024-02-14T15:01:00.512 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-02-14T15:01:00.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo stdin-killer -- python3 -c ' 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> import re 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> import os 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2024-02-14T15:01:00.513 DEBUG:teuthology.orchestra.run.smithi017:> 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> return f 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.admin")) 2024-02-14T15:01:00.514 DEBUG:teuthology.orchestra.run.smithi017:> ' 2024-02-14T15:01:00.634 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.admin.76306.asok 2024-02-14T15:01:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-14T15:01:00.645 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76306.asok 2024-02-14T15:01:00.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:00.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76306.asok status 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0", 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev)", 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "admin", 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "76306", 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:01:00.782 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 14805, 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 14805 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 673516127 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-02-14T15:01:00.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 673516127 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.14805 192.168.0.1:0/673516127", 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/673516127", 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 17, 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 89, 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2024-02-14T15:01:00.784 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:01:00.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:00.790 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2024-02-14T15:01:00.790 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:01:00.805 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2024-02-14T15:01:00.805 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:01:00.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:00.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:01:00.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:00 smithi017 ceph-mon[32549]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-02-14T15:01:00.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:00.873 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-02-14T15:01:00.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:00 smithi050 ceph-mon[36512]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-02-14T15:01:00.945 INFO:teuthology.orchestra.run.smithi017.stdout:# file: . 2024-02-14T15:01:00.945 INFO:teuthology.orchestra.run.smithi017.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-02-14T15:01:00.945 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:00.945 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:01:00.945 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-14T15:01:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:00 smithi076 ceph-mon[37057]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-02-14T15:01:01.001 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/3248301702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 msgr2=0x7f0ee81017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/3248301702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee81017d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009900 tx=0x7f0ed402bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/3248301702 shutdown_connections 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/3248301702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee81017d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/3248301702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ee8101d10 0x7f0ee81085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/3248301702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee8100160 0x7f0ee8100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/3248301702 >> 172.21.15.17:0/3248301702 conn(0x7f0ee80fb6d0 msgr2=0x7f0ee80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/3248301702 shutdown_connections 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.142+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/3248301702 wait complete. 2024-02-14T15:01:01.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 Processor -- start 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 -- start start 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ee8100160 0x7f0ee80690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee8101d10 0x7f0ee819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819ed70 con 0x7f0ee8101350 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819eee0 con 0x7f0ee8101d10 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eeeb6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819f0a0 con 0x7f0ee8100160 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee819bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40448/0 (socket says 172.21.15.17:40448) 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 -- 172.21.15.17:0/1701461979 learned_addr learned my addr 172.21.15.17:0/1701461979 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eec908640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ee8100160 0x7f0ee80690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eed109640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee8101d10 0x7f0ee819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 -- 172.21.15.17:0/1701461979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ee8100160 msgr2=0x7f0ee80690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ee8100160 0x7f0ee80690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 -- 172.21.15.17:0/1701461979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee8101d10 msgr2=0x7f0ee819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee8101d10 0x7f0ee819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 -- 172.21.15.17:0/1701461979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed4009560 con 0x7f0ee8101350 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0eed109640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee8101d10 0x7f0ee819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.143+0000 7f0ee7fff640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee819bf70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4006010 tx=0x7f0ed40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.144+0000 7f0ee5ffb640 1 -- 172.21.15.17:0/1701461979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40044e0 con 0x7f0ee8101350 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.144+0000 7f0ee5ffb640 1 -- 172.21.15.17:0/1701461979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ed40054e0 con 0x7f0ee8101350 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.144+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee819f320 con 0x7f0ee8101350 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.144+0000 7f0ee5ffb640 1 -- 172.21.15.17:0/1701461979 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed403e450 con 0x7f0ee8101350 2024-02-14T15:01:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.144+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee819f830 con 0x7f0ee8101350 2024-02-14T15:01:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.145+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee8107480 con 0x7f0ee8101350 2024-02-14T15:01:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.146+0000 7f0ee5ffb640 1 -- 172.21.15.17:0/1701461979 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0ed40035f0 con 0x7f0ee8101350 2024-02-14T15:01:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.146+0000 7f0ee5ffb640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0ed80a20c0 0x7f0ed80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.146+0000 7f0ee5ffb640 1 -- 172.21.15.17:0/1701461979 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0ed40f0820 con 0x7f0ee8101350 2024-02-14T15:01:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.146+0000 7f0eec908640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0ed80a20c0 0x7f0ed80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.147+0000 7f0eec908640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0ed80a20c0 0x7f0ed80a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ee81011b0 tx=0x7f0edc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:01.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.150+0000 7f0ee5ffb640 1 -- 172.21.15.17:0/1701461979 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ed40b95e0 con 0x7f0ee8101350 2024-02-14T15:01:01.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.334+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ee810bdc0 con 0x7f0ee8101350 2024-02-14T15:01:01.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.335+0000 7f0ee5ffb640 1 -- 172.21.15.17:0/1701461979 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7f0ed40bcca0 con 0x7f0ee8101350 2024-02-14T15:01:01.338 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:01.338 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:01:01.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0ed80a20c0 msgr2=0x7f0ed80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0ed80a20c0 0x7f0ed80a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ee81011b0 tx=0x7f0edc00b390 comp rx=0 tx=0).stop 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 msgr2=0x7f0ee819bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee819bf70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4006010 tx=0x7f0ed40042f0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 shutdown_connections 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0ed80a20c0 0x7f0ed80a4580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ee8101350 0x7f0ee819bf70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ee8101d10 0x7f0ee819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.337+0000 7f0eeeb6c640 1 --2- 172.21.15.17:0/1701461979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ee8100160 0x7f0ee80690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.338+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 >> 172.21.15.17:0/1701461979 conn(0x7f0ee80fb6d0 msgr2=0x7f0ee8104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.338+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 shutdown_connections 2024-02-14T15:01:01.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.338+0000 7f0eeeb6c640 1 -- 172.21.15.17:0/1701461979 wait complete. 2024-02-14T15:01:01.350 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:01:01.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:01:01.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 -- 172.21.15.17:0/3083218547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 msgr2=0x7f1750102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 --2- 172.21.15.17:0/3083218547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f1750102970 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1744009900 tx=0x7f174402bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 -- 172.21.15.17:0/3083218547 shutdown_connections 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 --2- 172.21.15.17:0/3083218547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f1750102970 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 --2- 172.21.15.17:0/3083218547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1750102eb0 0x7f1750109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 --2- 172.21.15.17:0/3083218547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17500ff520 0x7f17500ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 -- 172.21.15.17:0/3083218547 >> 172.21.15.17:0/3083218547 conn(0x7f17500fae50 msgr2=0x7f17500fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 -- 172.21.15.17:0/3083218547 shutdown_connections 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 -- 172.21.15.17:0/3083218547 wait complete. 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.455+0000 7f17583e9640 1 Processor -- start 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 -- start start 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17500ff520 0x7f17501972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f17501977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1750102eb0 0x7f175019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175019eda0 con 0x7f17501024f0 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175019ef10 con 0x7f17500ff520 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175019f0d0 con 0x7f1750102eb0 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f17501977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f17501977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40464/0 (socket says 172.21.15.17:40464) 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 -- 172.21.15.17:0/1919381721 learned_addr learned my addr 172.21.15.17:0/1919381721 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1756185640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17500ff520 0x7f17501972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1756986640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1750102eb0 0x7f175019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 -- 172.21.15.17:0/1919381721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1750102eb0 msgr2=0x7f175019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1750102eb0 0x7f175019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 -- 172.21.15.17:0/1919381721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17500ff520 msgr2=0x7f17501972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17500ff520 0x7f17501972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 -- 172.21.15.17:0/1919381721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1744009560 con 0x7f17501024f0 2024-02-14T15:01:01.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f1755984640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f17501977f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1744006010 tx=0x7f17440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:01.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f174f7fe640 1 -- 172.21.15.17:0/1919381721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17440044e0 con 0x7f17501024f0 2024-02-14T15:01:01.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f174f7fe640 1 -- 172.21.15.17:0/1919381721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17440054e0 con 0x7f17501024f0 2024-02-14T15:01:01.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175019f350 con 0x7f17501024f0 2024-02-14T15:01:01.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f174f7fe640 1 -- 172.21.15.17:0/1919381721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174403e450 con 0x7f17501024f0 2024-02-14T15:01:01.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.456+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175019f860 con 0x7f17501024f0 2024-02-14T15:01:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.457+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17501086a0 con 0x7f17501024f0 2024-02-14T15:01:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.458+0000 7f174f7fe640 1 -- 172.21.15.17:0/1919381721 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1744003680 con 0x7f17501024f0 2024-02-14T15:01:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.459+0000 7f174f7fe640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17380a20c0 0x7f17380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.459+0000 7f1756185640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17380a20c0 0x7f17380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.459+0000 7f174f7fe640 1 -- 172.21.15.17:0/1919381721 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f17440f01c0 con 0x7f17501024f0 2024-02-14T15:01:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.459+0000 7f1756185640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17380a20c0 0x7f17380a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1748005fd0 tx=0x7f174800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.463+0000 7f174f7fe640 1 -- 172.21.15.17:0/1919381721 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17440b9040 con 0x7f17501024f0 2024-02-14T15:01:01.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.648+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f175002cdb0 con 0x7f17501024f0 2024-02-14T15:01:01.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.649+0000 7f174f7fe640 1 -- 172.21.15.17:0/1919381721 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17440bc700 con 0x7f17501024f0 2024-02-14T15:01:01.650 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:01:01.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17380a20c0 msgr2=0x7f17380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17380a20c0 0x7f17380a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1748005fd0 tx=0x7f174800b410 comp rx=0 tx=0).stop 2024-02-14T15:01:01.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 msgr2=0x7f17501977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f17501977f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1744006010 tx=0x7f17440042f0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 shutdown_connections 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17380a20c0 0x7f17380a4580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17501024f0 0x7f17501977f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1750102eb0 0x7f175019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 --2- 172.21.15.17:0/1919381721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17500ff520 0x7f17501972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.650+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 >> 172.21.15.17:0/1919381721 conn(0x7f17500fae50 msgr2=0x7f1750105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.651+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 shutdown_connections 2024-02-14T15:01:01.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.651+0000 7f17583e9640 1 -- 172.21.15.17:0/1919381721 wait complete. 2024-02-14T15:01:01.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:01:01.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.766+0000 7f7d743d0640 1 -- 172.21.15.17:0/1681510721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c101d00 msgr2=0x7f7d6c108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.766+0000 7f7d743d0640 1 --2- 172.21.15.17:0/1681510721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c101d00 0x7f7d6c108590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009ca0 tx=0x7f7d6402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- 172.21.15.17:0/1681510721 shutdown_connections 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 --2- 172.21.15.17:0/1681510721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c101d00 0x7f7d6c108590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 --2- 172.21.15.17:0/1681510721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d6c101340 0x7f7d6c1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 --2- 172.21.15.17:0/1681510721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d6c100150 0x7f7d6c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- 172.21.15.17:0/1681510721 >> 172.21.15.17:0/1681510721 conn(0x7f7d6c0fb6c0 msgr2=0x7f7d6c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- 172.21.15.17:0/1681510721 shutdown_connections 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- 172.21.15.17:0/1681510721 wait complete. 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 Processor -- start 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- start start 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c100150 0x7f7d6c069070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d6c101340 0x7f7d6c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d6c101d00 0x7f7d6c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c19ee30 con 0x7f7d6c100150 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c19efa0 con 0x7f7d6c101340 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.767+0000 7f7d743d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c19f160 con 0x7f7d6c101d00 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c100150 0x7f7d6c069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c100150 0x7f7d6c069070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40488/0 (socket says 172.21.15.17:40488) 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 -- 172.21.15.17:0/3300968240 learned_addr learned my addr 172.21.15.17:0/3300968240 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7196b640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d6c101340 0x7f7d6c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7296d640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d6c101d00 0x7f7d6c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 -- 172.21.15.17:0/3300968240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d6c101d00 msgr2=0x7f7d6c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d6c101d00 0x7f7d6c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 -- 172.21.15.17:0/3300968240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d6c101340 msgr2=0x7f7d6c19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d6c101340 0x7f7d6c19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 -- 172.21.15.17:0/3300968240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d64009900 con 0x7f7d6c100150 2024-02-14T15:01:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d7216c640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c100150 0x7f7d6c069070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c970 tx=0x7f7d6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:01.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d637fe640 1 -- 172.21.15.17:0/3300968240 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68004220 con 0x7f7d6c100150 2024-02-14T15:01:01.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d637fe640 1 -- 172.21.15.17:0/3300968240 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d68007e90 con 0x7f7d6c100150 2024-02-14T15:01:01.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c19f3e0 con 0x7f7d6c100150 2024-02-14T15:01:01.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d637fe640 1 -- 172.21.15.17:0/3300968240 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800f450 con 0x7f7d6c100150 2024-02-14T15:01:01.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.768+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c19f940 con 0x7f7d6c100150 2024-02-14T15:01:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.769+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c107410 con 0x7f7d6c100150 2024-02-14T15:01:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.770+0000 7f7d637fe640 1 -- 172.21.15.17:0/3300968240 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7d68003610 con 0x7f7d6c100150 2024-02-14T15:01:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.771+0000 7f7d637fe640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7d580a2140 0x7f7d580a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.771+0000 7f7d637fe640 1 -- 172.21.15.17:0/3300968240 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7d680cdac0 con 0x7f7d6c100150 2024-02-14T15:01:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.771+0000 7f7d7196b640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7d580a2140 0x7f7d580a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.771+0000 7f7d7196b640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7d580a2140 0x7f7d580a4600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c068de0 tx=0x7f7d5c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:01.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.775+0000 7f7d637fe640 1 -- 172.21.15.17:0/3300968240 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d680968a0 con 0x7f7d6c100150 2024-02-14T15:01:01.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:01 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1701461979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:01.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.957+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d6c10bb60 con 0x7f7d6c100150 2024-02-14T15:01:01.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.957+0000 7f7d637fe640 1 -- 172.21.15.17:0/3300968240 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d68096070 con 0x7f7d6c100150 2024-02-14T15:01:01.959 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7d580a2140 msgr2=0x7f7d580a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7d580a2140 0x7f7d580a4600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c068de0 tx=0x7f7d5c00b390 comp rx=0 tx=0).stop 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c100150 msgr2=0x7f7d6c069070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c100150 0x7f7d6c069070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c970 tx=0x7f7d6800ce30 comp rx=0 tx=0).stop 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 shutdown_connections 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7d580a2140 0x7f7d580a4600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d6c101d00 0x7f7d6c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d6c101340 0x7f7d6c19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 --2- 172.21.15.17:0/3300968240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d6c100150 0x7f7d6c069070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:01.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 >> 172.21.15.17:0/3300968240 conn(0x7f7d6c0fb6c0 msgr2=0x7f7d6c1045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:01.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 shutdown_connections 2024-02-14T15:01:01.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:01.959+0000 7f7d743d0640 1 -- 172.21.15.17:0/3300968240 wait complete. 2024-02-14T15:01:01.970 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:01 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1701461979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:01.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:01:01.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:01 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1701461979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.075+0000 7f3a8e15c640 1 -- 172.21.15.17:0/4127926067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88100150 msgr2=0x7f3a88100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.075+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/4127926067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88100150 0x7f3a88100570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a7802bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.075+0000 7f3a8e15c640 1 -- 172.21.15.17:0/4127926067 shutdown_connections 2024-02-14T15:01:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/4127926067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88101d00 0x7f3a88108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/4127926067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88101340 0x7f3a881017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/4127926067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88100150 0x7f3a88100570 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 -- 172.21.15.17:0/4127926067 >> 172.21.15.17:0/4127926067 conn(0x7f3a880fb6c0 msgr2=0x7f3a880fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 -- 172.21.15.17:0/4127926067 shutdown_connections 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 -- 172.21.15.17:0/4127926067 wait complete. 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 Processor -- start 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 -- start start 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88100150 0x7f3a88199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88101340 0x7f3a88199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88101d00 0x7f3a881a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881a1020 con 0x7f3a88101d00 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881a1190 con 0x7f3a88100150 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.076+0000 7f3a8e15c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881a1350 con 0x7f3a88101340 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88101d00 0x7f3a881a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88101d00 0x7f3a881a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40502/0 (socket says 172.21.15.17:40502) 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 -- 172.21.15.17:0/3309600379 learned_addr learned my addr 172.21.15.17:0/3309600379 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a86ffd640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88101340 0x7f3a88199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a877fe640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88100150 0x7f3a88199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 -- 172.21.15.17:0/3309600379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88101340 msgr2=0x7f3a88199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88101340 0x7f3a88199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 -- 172.21.15.17:0/3309600379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88100150 msgr2=0x7f3a88199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88100150 0x7f3a88199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 -- 172.21.15.17:0/3309600379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a78009560 con 0x7f3a88101d00 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a87fff640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88101d00 0x7f3a881a0ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000eaa0 tx=0x7f3a8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a84ff9640 1 -- 172.21.15.17:0/3309600379 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000cdb0 con 0x7f3a88101d00 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a881a1630 con 0x7f3a88101d00 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a84ff9640 1 -- 172.21.15.17:0/3309600379 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a80005020 con 0x7f3a88101d00 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a84ff9640 1 -- 172.21.15.17:0/3309600379 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a800116b0 con 0x7f3a88101d00 2024-02-14T15:01:02.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.077+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a881a1b40 con 0x7f3a88101d00 2024-02-14T15:01:02.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.078+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8810bb60 con 0x7f3a88101d00 2024-02-14T15:01:02.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.079+0000 7f3a84ff9640 1 -- 172.21.15.17:0/3309600379 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3a80005520 con 0x7f3a88101d00 2024-02-14T15:01:02.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.080+0000 7f3a84ff9640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3a700a20c0 0x7f3a700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.080+0000 7f3a877fe640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3a700a20c0 0x7f3a700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.080+0000 7f3a84ff9640 1 -- 172.21.15.17:0/3309600379 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3a800ce090 con 0x7f3a88101d00 2024-02-14T15:01:02.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.080+0000 7f3a877fe640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3a700a20c0 0x7f3a700a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a78005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:02.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.084+0000 7f3a84ff9640 1 -- 172.21.15.17:0/3309600379 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a80096ec0 con 0x7f3a88101d00 2024-02-14T15:01:02.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.264+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a881a1fd0 con 0x7f3a88101d00 2024-02-14T15:01:02.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.265+0000 7f3a84ff9640 1 -- 172.21.15.17:0/3309600379 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a8001f0a0 con 0x7f3a88101d00 2024-02-14T15:01:02.266 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:01:02.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3a700a20c0 msgr2=0x7f3a700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3a700a20c0 0x7f3a700a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a78005fd0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88101d00 msgr2=0x7f3a881a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88101d00 0x7f3a881a0ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000eaa0 tx=0x7f3a8000ef60 comp rx=0 tx=0).stop 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 shutdown_connections 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f3a700a20c0 0x7f3a700a4580 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a88101d00 0x7f3a881a0ae0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88101340 0x7f3a88199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.266+0000 7f3a8e15c640 1 --2- 172.21.15.17:0/3309600379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88100150 0x7f3a88199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.267+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 >> 172.21.15.17:0/3309600379 conn(0x7f3a880fb6c0 msgr2=0x7f3a88068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.267+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 shutdown_connections 2024-02-14T15:01:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.267+0000 7f3a8e15c640 1 -- 172.21.15.17:0/3309600379 wait complete. 2024-02-14T15:01:02.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:01:02.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 -- 172.21.15.17:0/4090418912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0106f30 msgr2=0x7f37a0109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 --2- 172.21.15.17:0/4090418912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0106f30 0x7f37a0109380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3798009ca0 tx=0x7f379802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 -- 172.21.15.17:0/4090418912 shutdown_connections 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 --2- 172.21.15.17:0/4090418912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0106f30 0x7f37a0109380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 --2- 172.21.15.17:0/4090418912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a0104600 0x7f37a01069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 --2- 172.21.15.17:0/4090418912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0101cd0 0x7f37a01040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 -- 172.21.15.17:0/4090418912 >> 172.21.15.17:0/4090418912 conn(0x7f37a00fb6c0 msgr2=0x7f37a00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 -- 172.21.15.17:0/4090418912 shutdown_connections 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 -- 172.21.15.17:0/4090418912 wait complete. 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 Processor -- start 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.384+0000 7f37a7064640 1 -- start start 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a7064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0101cd0 0x7f37a01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a7064640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0104600 0x7f37a01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a7064640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a0106f30 0x7f37a019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a7064640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a019ed90 con 0x7f37a0104600 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a7064640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a019ef00 con 0x7f37a0106f30 2024-02-14T15:01:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a7064640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a019f0c0 con 0x7f37a0101cd0 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0104600 0x7f37a01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0104600 0x7f37a01977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40526/0 (socket says 172.21.15.17:40526) 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 -- 172.21.15.17:0/3745837062 learned_addr learned my addr 172.21.15.17:0/3745837062 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a5601640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a0106f30 0x7f37a019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a4e00640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0101cd0 0x7f37a01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 -- 172.21.15.17:0/3745837062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0101cd0 msgr2=0x7f37a01972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0101cd0 0x7f37a01972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 -- 172.21.15.17:0/3745837062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a0106f30 msgr2=0x7f37a019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a0106f30 0x7f37a019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 -- 172.21.15.17:0/3745837062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798009900 con 0x7f37a0104600 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a4e00640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0101cd0 0x7f37a01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379ffff640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0104600 0x7f37a01977e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f379400e9c0 tx=0x7f379400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:02.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379dffb640 1 -- 172.21.15.17:0/3745837062 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794009700 con 0x7f37a0104600 2024-02-14T15:01:02.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a019f3a0 con 0x7f37a0104600 2024-02-14T15:01:02.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379dffb640 1 -- 172.21.15.17:0/3745837062 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f379400cd80 con 0x7f37a0104600 2024-02-14T15:01:02.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.385+0000 7f379dffb640 1 -- 172.21.15.17:0/3745837062 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794010430 con 0x7f37a0104600 2024-02-14T15:01:02.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.386+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a00ffca0 con 0x7f37a0104600 2024-02-14T15:01:02.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.387+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a00fd1b0 con 0x7f37a0104600 2024-02-14T15:01:02.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.387+0000 7f379dffb640 1 -- 172.21.15.17:0/3745837062 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3794005000 con 0x7f37a0104600 2024-02-14T15:01:02.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.388+0000 7f379dffb640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37900a64d0 0x7f37900a8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.388+0000 7f379dffb640 1 -- 172.21.15.17:0/3745837062 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3794014070 con 0x7f37a0104600 2024-02-14T15:01:02.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.388+0000 7f37a4e00640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37900a64d0 0x7f37900a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.388+0000 7f37a4e00640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37900a64d0 0x7f37900a8990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f378c009b50 tx=0x7f378c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:02.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.392+0000 7f379dffb640 1 -- 172.21.15.17:0/3745837062 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37940970e0 con 0x7f37a0104600 2024-02-14T15:01:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.586+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f37a002cde0 con 0x7f37a0104600 2024-02-14T15:01:02.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.589+0000 7f379dffb640 1 -- 172.21.15.17:0/3745837062 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f379409a7a0 con 0x7f37a0104600 2024-02-14T15:01:02.590 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37900a64d0 msgr2=0x7f37900a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37900a64d0 0x7f37900a8990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f378c009b50 tx=0x7f378c005f90 comp rx=0 tx=0).stop 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0104600 msgr2=0x7f37a01977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0104600 0x7f37a01977e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f379400e9c0 tx=0x7f379400ee80 comp rx=0 tx=0).stop 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 shutdown_connections 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37900a64d0 0x7f37900a8990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37a0104600 0x7f37a01977e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a0106f30 0x7f37a019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 --2- 172.21.15.17:0/3745837062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0101cd0 0x7f37a01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 >> 172.21.15.17:0/3745837062 conn(0x7f37a00fb6c0 msgr2=0x7f37a0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 shutdown_connections 2024-02-14T15:01:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.591+0000 7f37a7064640 1 -- 172.21.15.17:0/3745837062 wait complete. 2024-02-14T15:01:02.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 -- 172.21.15.17:0/2859786640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c101d70 msgr2=0x7fc64c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 --2- 172.21.15.17:0/2859786640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c101d70 0x7fc64c104160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009900 tx=0x7fc63c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 -- 172.21.15.17:0/2859786640 shutdown_connections 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 --2- 172.21.15.17:0/2859786640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc64c106fd0 0x7fc64c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 --2- 172.21.15.17:0/2859786640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64c1046a0 0x7fc64c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 --2- 172.21.15.17:0/2859786640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c101d70 0x7fc64c104160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 -- 172.21.15.17:0/2859786640 >> 172.21.15.17:0/2859786640 conn(0x7fc64c0fb6c0 msgr2=0x7fc64c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 -- 172.21.15.17:0/2859786640 shutdown_connections 2024-02-14T15:01:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.709+0000 7fc651f25640 1 -- 172.21.15.17:0/2859786640 wait complete. 2024-02-14T15:01:02.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 Processor -- start 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 -- start start 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64c101d70 0x7fc64c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc64c1046a0 0x7fc64c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c106fd0 0x7fc64c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c19ee00 con 0x7fc64c106fd0 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c19ef70 con 0x7fc64c101d70 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc651f25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c19f130 con 0x7fc64c1046a0 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c106fd0 0x7fc64c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c106fd0 0x7fc64c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40554/0 (socket says 172.21.15.17:40554) 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 -- 172.21.15.17:0/2539889050 learned_addr learned my addr 172.21.15.17:0/2539889050 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64b7fe640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64c101d70 0x7fc64c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64affd640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc64c1046a0 0x7fc64c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 -- 172.21.15.17:0/2539889050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc64c1046a0 msgr2=0x7fc64c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc64c1046a0 0x7fc64c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 -- 172.21.15.17:0/2539889050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64c101d70 msgr2=0x7fc64c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64c101d70 0x7fc64c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.710+0000 7fc64bfff640 1 -- 172.21.15.17:0/2539889050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c009560 con 0x7fc64c106fd0 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.711+0000 7fc64bfff640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c106fd0 0x7fc64c19e8c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc64400eb10 tx=0x7fc64400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.711+0000 7fc648ff9640 1 -- 172.21.15.17:0/2539889050 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644004b20 con 0x7fc64c106fd0 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.711+0000 7fc648ff9640 1 -- 172.21.15.17:0/2539889050 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc644004c80 con 0x7fc64c106fd0 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.711+0000 7fc648ff9640 1 -- 172.21.15.17:0/2539889050 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644010650 con 0x7fc64c106fd0 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.711+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c19f410 con 0x7fc64c106fd0 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.711+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c19f920 con 0x7fc64c106fd0 2024-02-14T15:01:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.712+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c0fd1b0 con 0x7fc64c106fd0 2024-02-14T15:01:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.713+0000 7fc648ff9640 1 -- 172.21.15.17:0/2539889050 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc644004df0 con 0x7fc64c106fd0 2024-02-14T15:01:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.713+0000 7fc648ff9640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6340a20c0 0x7fc6340a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.714+0000 7fc64b7fe640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6340a20c0 0x7fc6340a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.714+0000 7fc648ff9640 1 -- 172.21.15.17:0/2539889050 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc644014070 con 0x7fc64c106fd0 2024-02-14T15:01:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.714+0000 7fc64b7fe640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6340a20c0 0x7fc6340a4580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000c00 tx=0x7fc63c02c7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.719+0000 7fc648ff9640 1 -- 172.21.15.17:0/2539889050 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc644097390 con 0x7fc64c106fd0 2024-02-14T15:01:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:02 smithi050 ceph-mon[36512]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-02-14T15:01:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1919381721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3300968240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3309600379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.915+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc64c10ccd0 con 0x7fc64c106fd0 2024-02-14T15:01:02.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.915+0000 7fc648ff9640 1 -- 172.21.15.17:0/2539889050 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+592 (secure 0 0 0) 0x7fc64409aa50 con 0x7fc64c106fd0 2024-02-14T15:01:02.917 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3179868836 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:02.917 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3756063081 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:02.917 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/1044675107 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:02.917 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1117796267 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:02.918 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3241397077 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:02.918 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/2869822133 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:02.918 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/2869822133 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:02.918 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3371941300 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:02.918 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/1044675107 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:02.918 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2319986884 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:02.918 INFO:teuthology.orchestra.run.smithi017.stderr:listed 10 entries 2024-02-14T15:01:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6340a20c0 msgr2=0x7fc6340a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6340a20c0 0x7fc6340a4580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000c00 tx=0x7fc63c02c7e0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c106fd0 msgr2=0x7fc64c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c106fd0 0x7fc64c19e8c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc64400eb10 tx=0x7fc64400c2d0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 shutdown_connections 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6340a20c0 0x7fc6340a4580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc64c106fd0 0x7fc64c19e8c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc64c1046a0 0x7fc64c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 --2- 172.21.15.17:0/2539889050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc64c101d70 0x7fc64c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.917+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 >> 172.21.15.17:0/2539889050 conn(0x7fc64c0fb6c0 msgr2=0x7fc64c1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.918+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 shutdown_connections 2024-02-14T15:01:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:02.918+0000 7fc651f25640 1 -- 172.21.15.17:0/2539889050 wait complete. 2024-02-14T15:01:02.929 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:02 smithi017 ceph-mon[32549]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-02-14T15:01:02.929 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1919381721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.930 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3300968240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.930 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3309600379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.931 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:01:02.931 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-14T15:01:02.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:01:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:02 smithi076 ceph-mon[37057]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-02-14T15:01:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1919381721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3300968240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:01:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3309600379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 -- 172.21.15.17:0/4128182266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034101270 msgr2=0x7f8034101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 --2- 172.21.15.17:0/4128182266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034101270 0x7f8034101690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f801c009900 tx=0x7f801c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 -- 172.21.15.17:0/4128182266 shutdown_connections 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 --2- 172.21.15.17:0/4128182266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8034102e20 0x7f80341096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 --2- 172.21.15.17:0/4128182266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8034102460 0x7f80341028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 --2- 172.21.15.17:0/4128182266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034101270 0x7f8034101690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 -- 172.21.15.17:0/4128182266 >> 172.21.15.17:0/4128182266 conn(0x7f80340fc7c0 msgr2=0x7f80340fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 -- 172.21.15.17:0/4128182266 shutdown_connections 2024-02-14T15:01:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.089+0000 7f803ae5c640 1 -- 172.21.15.17:0/4128182266 wait complete. 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 Processor -- start 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 -- start start 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8034101270 0x7f80341972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034102460 0x7f8034197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8034102e20 0x7f803419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803419edc0 con 0x7f8034102460 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803419ef30 con 0x7f8034101270 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f803ae5c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803419f0f0 con 0x7f8034102e20 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034102460 0x7f8034197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034102460 0x7f8034197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40576/0 (socket says 172.21.15.17:40576) 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 -- 172.21.15.17:0/2343048999 learned_addr learned my addr 172.21.15.17:0/2343048999 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8038bf8640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8034101270 0x7f80341972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f80393f9640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8034102e20 0x7f803419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 -- 172.21.15.17:0/2343048999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8034102e20 msgr2=0x7f803419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8034102e20 0x7f803419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 -- 172.21.15.17:0/2343048999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8034101270 msgr2=0x7f80341972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8034101270 0x7f80341972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 -- 172.21.15.17:0/2343048999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c009560 con 0x7f8034102460 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f80393f9640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8034102e20 0x7f803419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.090+0000 7f8033fff640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034102460 0x7f8034197810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f802400eab0 tx=0x7f802400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.091+0000 7f8031ffb640 1 -- 172.21.15.17:0/2343048999 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802400ce40 con 0x7f8034102460 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.091+0000 7f8031ffb640 1 -- 172.21.15.17:0/2343048999 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8024011950 con 0x7f8034102460 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.091+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803419f3d0 con 0x7f8034102460 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.091+0000 7f8031ffb640 1 -- 172.21.15.17:0/2343048999 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024010aa0 con 0x7f8034102460 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.091+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803419f8e0 con 0x7f8034102460 2024-02-14T15:01:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.092+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8034108590 con 0x7f8034102460 2024-02-14T15:01:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.093+0000 7f8031ffb640 1 -- 172.21.15.17:0/2343048999 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8024011ac0 con 0x7f8034102460 2024-02-14T15:01:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.093+0000 7f8031ffb640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f80200a20c0 0x7f80200a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.093+0000 7f8038bf8640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f80200a20c0 0x7f80200a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.093+0000 7f8031ffb640 1 -- 172.21.15.17:0/2343048999 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f80240ce490 con 0x7f8034102460 2024-02-14T15:01:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.094+0000 7f8038bf8640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f80200a20c0 0x7f80200a4580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f801c005ab0 tx=0x7f801c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.098+0000 7f8031ffb640 1 -- 172.21.15.17:0/2343048999 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8024097270 con 0x7f8034102460 2024-02-14T15:01:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.281+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f803402cdb0 con 0x7f8034102460 2024-02-14T15:01:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.283+0000 7f8031ffb640 1 -- 172.21.15.17:0/2343048999 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7f802409a930 con 0x7f8034102460 2024-02-14T15:01:03.285 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:03.285 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f80200a20c0 msgr2=0x7f80200a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f80200a20c0 0x7f80200a4580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f801c005ab0 tx=0x7f801c005fd0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034102460 msgr2=0x7f8034197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034102460 0x7f8034197810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f802400eab0 tx=0x7f802400ef70 comp rx=0 tx=0).stop 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 shutdown_connections 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f80200a20c0 0x7f80200a4580 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8034102460 0x7f8034197810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8034102e20 0x7f803419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 --2- 172.21.15.17:0/2343048999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8034101270 0x7f80341972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 >> 172.21.15.17:0/2343048999 conn(0x7f80340fc7c0 msgr2=0x7f8034105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 shutdown_connections 2024-02-14T15:01:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.285+0000 7f803ae5c640 1 -- 172.21.15.17:0/2343048999 wait complete. 2024-02-14T15:01:03.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:01:03.299 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:01:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 -- 172.21.15.17:0/2709847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e640fff80 msgr2=0x7f2e641003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 --2- 172.21.15.17:0/2709847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e640fff80 0x7f2e641003a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009900 tx=0x7f2e5802bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 -- 172.21.15.17:0/2709847 shutdown_connections 2024-02-14T15:01:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 --2- 172.21.15.17:0/2709847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e64101b30 0x7f2e641083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 --2- 172.21.15.17:0/2709847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e64101170 0x7f2e641015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 --2- 172.21.15.17:0/2709847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e640fff80 0x7f2e641003a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 -- 172.21.15.17:0/2709847 >> 172.21.15.17:0/2709847 conn(0x7f2e640fb4f0 msgr2=0x7f2e640fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:03.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 -- 172.21.15.17:0/2709847 shutdown_connections 2024-02-14T15:01:03.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 -- 172.21.15.17:0/2709847 wait complete. 2024-02-14T15:01:03.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.403+0000 7f2e63fff640 1 Processor -- start 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e63fff640 1 -- start start 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e640fff80 0x7f2e64194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e64101170 0x7f2e64195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e64101b30 0x7f2e6419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e63fff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6419c9d0 con 0x7f2e64101170 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e63fff640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6419cb40 con 0x7f2e640fff80 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e63fff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6419cd00 con 0x7f2e64101b30 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e64101170 0x7f2e64195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e64101170 0x7f2e64195420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40592/0 (socket says 172.21.15.17:40592) 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 -- 172.21.15.17:0/4279215633 learned_addr learned my addr 172.21.15.17:0/4279215633 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e637fe640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e64101b30 0x7f2e6419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e62ffd640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e640fff80 0x7f2e64194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 -- 172.21.15.17:0/4279215633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e64101b30 msgr2=0x7f2e6419c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e64101b30 0x7f2e6419c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 -- 172.21.15.17:0/4279215633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e640fff80 msgr2=0x7f2e64194ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e640fff80 0x7f2e64194ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e627fc640 1 -- 172.21.15.17:0/4279215633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e58009560 con 0x7f2e64101170 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e637fe640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e64101b30 0x7f2e6419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.404+0000 7f2e62ffd640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e640fff80 0x7f2e64194ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.405+0000 7f2e627fc640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e64101170 0x7f2e64195420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400d770 tx=0x7f2e5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.405+0000 7f2e68800640 1 -- 172.21.15.17:0/4279215633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54009c30 con 0x7f2e64101170 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.405+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6419cfe0 con 0x7f2e64101170 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.405+0000 7f2e68800640 1 -- 172.21.15.17:0/4279215633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e5400de80 con 0x7f2e64101170 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.405+0000 7f2e68800640 1 -- 172.21.15.17:0/4279215633 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54004e80 con 0x7f2e64101170 2024-02-14T15:01:03.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.405+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6419d460 con 0x7f2e64101170 2024-02-14T15:01:03.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.406+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e641072f0 con 0x7f2e64101170 2024-02-14T15:01:03.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.407+0000 7f2e68800640 1 -- 172.21.15.17:0/4279215633 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2e54005020 con 0x7f2e64101170 2024-02-14T15:01:03.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.407+0000 7f2e68800640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2e4c0a42d0 0x7f2e4c0a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.408+0000 7f2e68800640 1 -- 172.21.15.17:0/4279215633 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2e540cd960 con 0x7f2e64101170 2024-02-14T15:01:03.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.408+0000 7f2e62ffd640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2e4c0a42d0 0x7f2e4c0a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.408+0000 7f2e62ffd640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2e4c0a42d0 0x7f2e4c0a6790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2e58036040 tx=0x7f2e58000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.412+0000 7f2e68800640 1 -- 172.21.15.17:0/4279215633 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e54096740 con 0x7f2e64101170 2024-02-14T15:01:03.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.594+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e6402ceb0 con 0x7f2e64101170 2024-02-14T15:01:03.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.595+0000 7f2e68800640 1 -- 172.21.15.17:0/4279215633 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e540200a0 con 0x7f2e64101170 2024-02-14T15:01:03.597 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2e4c0a42d0 msgr2=0x7f2e4c0a6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2e4c0a42d0 0x7f2e4c0a6790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2e58036040 tx=0x7f2e58000bc0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e64101170 msgr2=0x7f2e64195420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e64101170 0x7f2e64195420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400d770 tx=0x7f2e5400dc30 comp rx=0 tx=0).stop 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 shutdown_connections 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2e4c0a42d0 0x7f2e4c0a6790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e64101170 0x7f2e64195420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e64101b30 0x7f2e6419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 --2- 172.21.15.17:0/4279215633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e640fff80 0x7f2e64194ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 >> 172.21.15.17:0/4279215633 conn(0x7f2e640fb4f0 msgr2=0x7f2e641044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 shutdown_connections 2024-02-14T15:01:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.597+0000 7f2e63fff640 1 -- 172.21.15.17:0/4279215633 wait complete. 2024-02-14T15:01:03.609 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 -- 172.21.15.17:0/4108996064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c106fb0 msgr2=0x7f813c1093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 --2- 172.21.15.17:0/4108996064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c106fb0 0x7f813c1093a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8134009ca0 tx=0x7f813402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 -- 172.21.15.17:0/4108996064 shutdown_connections 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 --2- 172.21.15.17:0/4108996064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c106fb0 0x7f813c1093a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 --2- 172.21.15.17:0/4108996064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c104680 0x7f813c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 --2- 172.21.15.17:0/4108996064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f813c101d50 0x7f813c104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 -- 172.21.15.17:0/4108996064 >> 172.21.15.17:0/4108996064 conn(0x7f813c0fb6c0 msgr2=0x7f813c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 -- 172.21.15.17:0/4108996064 shutdown_connections 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.713+0000 7f8142577640 1 -- 172.21.15.17:0/4108996064 wait complete. 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 Processor -- start 2024-02-14T15:01:03.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 -- start start 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c101d50 0x7f813c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c104680 0x7f813c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f813c106fb0 0x7f813c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c19ed80 con 0x7f813c101d50 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c19eef0 con 0x7f813c106fb0 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8142577640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c19f0b0 con 0x7f813c104680 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f813bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c101d50 0x7f813c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f813bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c101d50 0x7f813c197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40610/0 (socket says 172.21.15.17:40610) 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f813bfff640 1 -- 172.21.15.17:0/3848292403 learned_addr learned my addr 172.21.15.17:0/3848292403 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.714+0000 7f8140b14640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f813c106fb0 0x7f813c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813b7fe640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c104680 0x7f813c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813bfff640 1 -- 172.21.15.17:0/3848292403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c104680 msgr2=0x7f813c1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813bfff640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c104680 0x7f813c1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813bfff640 1 -- 172.21.15.17:0/3848292403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f813c106fb0 msgr2=0x7f813c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813bfff640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f813c106fb0 0x7f813c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813bfff640 1 -- 172.21.15.17:0/3848292403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134009900 con 0x7f813c101d50 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813b7fe640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c104680 0x7f813c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f813bfff640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c101d50 0x7f813c197290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f812c00d950 tx=0x7f812c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f81397fa640 1 -- 172.21.15.17:0/3848292403 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c009c30 con 0x7f813c101d50 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c19f390 con 0x7f813c101d50 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f81397fa640 1 -- 172.21.15.17:0/3848292403 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f812c009d90 con 0x7f813c101d50 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f81397fa640 1 -- 172.21.15.17:0/3848292403 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c00f7e0 con 0x7f813c101d50 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.715+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c19f8a0 con 0x7f813c101d50 2024-02-14T15:01:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.716+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c0fd1b0 con 0x7f813c101d50 2024-02-14T15:01:03.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.717+0000 7f81397fa640 1 -- 172.21.15.17:0/3848292403 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f812c010ad0 con 0x7f813c101d50 2024-02-14T15:01:03.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.717+0000 7f81397fa640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f81240a20c0 0x7f81240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:03.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.718+0000 7f813b7fe640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f81240a20c0 0x7f81240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:03.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.718+0000 7f81397fa640 1 -- 172.21.15.17:0/3848292403 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f812c0cdb70 con 0x7f813c101d50 2024-02-14T15:01:03.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.718+0000 7f813b7fe640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f81240a20c0 0x7f81240a4580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f813c1987a0 tx=0x7f8130005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.722+0000 7f81397fa640 1 -- 172.21.15.17:0/3848292403 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f812c0968f0 con 0x7f813c101d50 2024-02-14T15:01:03.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:03 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3745837062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:01:03.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:03 smithi050 ceph-mon[36512]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-02-14T15:01:03.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:03 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2539889050' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:01:03.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:03 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2343048999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:03.904 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:03 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3745837062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:01:03.905 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:03 smithi076 ceph-mon[37057]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-02-14T15:01:03.905 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:03 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2539889050' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:01:03.905 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:03 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2343048999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.903+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f813c02ce50 con 0x7f813c101d50 2024-02-14T15:01:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.904+0000 7f81397fa640 1 -- 172.21.15.17:0/3848292403 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f812c099fb0 con 0x7f813c101d50 2024-02-14T15:01:03.906 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f81240a20c0 msgr2=0x7f81240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f81240a20c0 0x7f81240a4580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f813c1987a0 tx=0x7f8130005e50 comp rx=0 tx=0).stop 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c101d50 msgr2=0x7f813c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c101d50 0x7f813c197290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f812c00d950 tx=0x7f812c00de10 comp rx=0 tx=0).stop 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 shutdown_connections 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f81240a20c0 0x7f81240a4580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f813c106fb0 0x7f813c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c104680 0x7f813c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 --2- 172.21.15.17:0/3848292403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f813c101d50 0x7f813c197290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 >> 172.21.15.17:0/3848292403 conn(0x7f813c0fb6c0 msgr2=0x7f813c102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 shutdown_connections 2024-02-14T15:01:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:03.906+0000 7f8142577640 1 -- 172.21.15.17:0/3848292403 wait complete. 2024-02-14T15:01:03.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:03 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3745837062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:01:03.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:03 smithi017 ceph-mon[32549]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-02-14T15:01:03.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:03 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2539889050' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:01:03.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:03 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2343048999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:03.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:01:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.021+0000 7f06ed65c640 1 -- 172.21.15.17:0/2019998364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8102450 msgr2=0x7f06e81028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.021+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2019998364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8102450 0x7f06e81028d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009900 tx=0x7f06d002bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:04.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- 172.21.15.17:0/2019998364 shutdown_connections 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2019998364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8102450 0x7f06e81028d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2019998364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06e8102e10 0x7f06e81096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2019998364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e8101260 0x7f06e8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- 172.21.15.17:0/2019998364 >> 172.21.15.17:0/2019998364 conn(0x7f06e80fc7f0 msgr2=0x7f06e80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- 172.21.15.17:0/2019998364 shutdown_connections 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- 172.21.15.17:0/2019998364 wait complete. 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 Processor -- start 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- start start 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8101260 0x7f06e81972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06e8102450 0x7f06e8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e8102e10 0x7f06e819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e819edc0 con 0x7f06e8101260 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e819ef30 con 0x7f06e8102450 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.022+0000 7f06ed65c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e819f0f0 con 0x7f06e8102e10 2024-02-14T15:01:04.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8101260 0x7f06e81972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8101260 0x7f06e81972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40618/0 (socket says 172.21.15.17:40618) 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 -- 172.21.15.17:0/2426716813 learned_addr learned my addr 172.21.15.17:0/2426716813 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e67fc640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06e8102450 0x7f06e8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e77fe640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e8102e10 0x7f06e819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 -- 172.21.15.17:0/2426716813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e8102e10 msgr2=0x7f06e819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e8102e10 0x7f06e819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 -- 172.21.15.17:0/2426716813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06e8102450 msgr2=0x7f06e8197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06e8102450 0x7f06e8197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 -- 172.21.15.17:0/2426716813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0009560 con 0x7f06e8101260 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e67fc640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06e8102450 0x7f06e8197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06e6ffd640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8101260 0x7f06e81972d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9c0 tx=0x7f06e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06dffff640 1 -- 172.21.15.17:0/2426716813 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e00096e0 con 0x7f06e8101260 2024-02-14T15:01:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06dffff640 1 -- 172.21.15.17:0/2426716813 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06e000cd80 con 0x7f06e8101260 2024-02-14T15:01:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e819f3d0 con 0x7f06e8101260 2024-02-14T15:01:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06dffff640 1 -- 172.21.15.17:0/2426716813 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0010430 con 0x7f06e8101260 2024-02-14T15:01:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.023+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e819f890 con 0x7f06e8101260 2024-02-14T15:01:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.024+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e8108600 con 0x7f06e8101260 2024-02-14T15:01:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.025+0000 7f06dffff640 1 -- 172.21.15.17:0/2426716813 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f06e0003680 con 0x7f06e8101260 2024-02-14T15:01:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.026+0000 7f06dffff640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06d40a20c0 0x7f06d40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.026+0000 7f06e67fc640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06d40a20c0 0x7f06d40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.026+0000 7f06dffff640 1 -- 172.21.15.17:0/2426716813 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f06e0014070 con 0x7f06e8101260 2024-02-14T15:01:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.026+0000 7f06e67fc640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06d40a20c0 0x7f06d40a4580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f06e81987e0 tx=0x7f06d0005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:04.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.030+0000 7f06dffff640 1 -- 172.21.15.17:0/2426716813 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06e0097140 con 0x7f06e8101260 2024-02-14T15:01:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.211+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06e802ce80 con 0x7f06e8101260 2024-02-14T15:01:04.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.212+0000 7f06dffff640 1 -- 172.21.15.17:0/2426716813 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06e009a800 con 0x7f06e8101260 2024-02-14T15:01:04.214 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06d40a20c0 msgr2=0x7f06d40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06d40a20c0 0x7f06d40a4580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f06e81987e0 tx=0x7f06d0005be0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8101260 msgr2=0x7f06e81972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8101260 0x7f06e81972d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9c0 tx=0x7f06e000ee80 comp rx=0 tx=0).stop 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 shutdown_connections 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f06d40a20c0 0x7f06d40a4580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e8102e10 0x7f06e819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f06e8102450 0x7f06e8197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 --2- 172.21.15.17:0/2426716813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06e8101260 0x7f06e81972d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 >> 172.21.15.17:0/2426716813 conn(0x7f06e80fc7f0 msgr2=0x7f06e81057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 shutdown_connections 2024-02-14T15:01:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.214+0000 7f06ed65c640 1 -- 172.21.15.17:0/2426716813 wait complete. 2024-02-14T15:01:04.226 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 -- 172.21.15.17:0/3349510243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 msgr2=0x7f43781028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 --2- 172.21.15.17:0/3349510243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f43781028b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f436c009900 tx=0x7f436c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 -- 172.21.15.17:0/3349510243 shutdown_connections 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 --2- 172.21.15.17:0/3349510243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f43781028b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 --2- 172.21.15.17:0/3349510243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4378102df0 0x7f4378109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 --2- 172.21.15.17:0/3349510243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4378101240 0x7f4378101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 -- 172.21.15.17:0/3349510243 >> 172.21.15.17:0/3349510243 conn(0x7f43780fc7b0 msgr2=0x7f43780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 -- 172.21.15.17:0/3349510243 shutdown_connections 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.332+0000 7f437caae640 1 -- 172.21.15.17:0/3349510243 wait complete. 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 Processor -- start 2024-02-14T15:01:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 -- start start 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4378101240 0x7f4378197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f4378197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4378102df0 0x7f437819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819ee20 con 0x7f4378102430 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819ef90 con 0x7f4378101240 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437caae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819f150 con 0x7f4378102df0 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f4378197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f4378197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40630/0 (socket says 172.21.15.17:40630) 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 -- 172.21.15.17:0/575297743 learned_addr learned my addr 172.21.15.17:0/575297743 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437659c640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4378101240 0x7f4378197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4376d9d640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4378102df0 0x7f437819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 -- 172.21.15.17:0/575297743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4378102df0 msgr2=0x7f437819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4378102df0 0x7f437819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 -- 172.21.15.17:0/575297743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4378101240 msgr2=0x7f4378197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4378101240 0x7f4378197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 -- 172.21.15.17:0/575297743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c009560 con 0x7f4378102430 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f437659c640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4378101240 0x7f4378197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.333+0000 7f4375d9b640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f4378197870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f436c006010 tx=0x7f436c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.334+0000 7f43677fe640 1 -- 172.21.15.17:0/575297743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0044e0 con 0x7f4378102430 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.334+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437819f3d0 con 0x7f4378102430 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.334+0000 7f43677fe640 1 -- 172.21.15.17:0/575297743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f436c0054e0 con 0x7f4378102430 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.334+0000 7f43677fe640 1 -- 172.21.15.17:0/575297743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c03e450 con 0x7f4378102430 2024-02-14T15:01:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.334+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437819f960 con 0x7f4378102430 2024-02-14T15:01:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.335+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43781085e0 con 0x7f4378102430 2024-02-14T15:01:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.336+0000 7f43677fe640 1 -- 172.21.15.17:0/575297743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f436c0035f0 con 0x7f4378102430 2024-02-14T15:01:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.336+0000 7f43677fe640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43600a20c0 0x7f43600a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.336+0000 7f437659c640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43600a20c0 0x7f43600a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.336+0000 7f43677fe640 1 -- 172.21.15.17:0/575297743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f436c0f08f0 con 0x7f4378102430 2024-02-14T15:01:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.337+0000 7f437659c640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43600a20c0 0x7f43600a4580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43680058c0 tx=0x7f4368005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.340+0000 7f43677fe640 1 -- 172.21.15.17:0/575297743 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f436c0b96d0 con 0x7f4378102430 2024-02-14T15:01:04.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.519+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f437810ce20 con 0x7f4378102430 2024-02-14T15:01:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.520+0000 7f43677fe640 1 -- 172.21.15.17:0/575297743 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f437810ce20 con 0x7f4378102430 2024-02-14T15:01:04.522 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:01:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43600a20c0 msgr2=0x7f43600a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43600a20c0 0x7f43600a4580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43680058c0 tx=0x7f4368005850 comp rx=0 tx=0).stop 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 msgr2=0x7f4378197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f4378197870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f436c006010 tx=0x7f436c0042f0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 shutdown_connections 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43600a20c0 0x7f43600a4580 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4378102430 0x7f4378197870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4378102df0 0x7f437819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 --2- 172.21.15.17:0/575297743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4378101240 0x7f4378197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 >> 172.21.15.17:0/575297743 conn(0x7f43780fc7b0 msgr2=0x7f43781057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 shutdown_connections 2024-02-14T15:01:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.522+0000 7f437caae640 1 -- 172.21.15.17:0/575297743 wait complete. 2024-02-14T15:01:04.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3368171272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac100140 msgr2=0x7f01ac100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3368171272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac100140 0x7f01ac100560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f019c009900 tx=0x7f019c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3368171272 shutdown_connections 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3368171272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01ac101cf0 0x7f01ac108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3368171272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01ac101330 0x7f01ac1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3368171272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac100140 0x7f01ac100560 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3368171272 >> 172.21.15.17:0/3368171272 conn(0x7f01ac0fb690 msgr2=0x7f01ac0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3368171272 shutdown_connections 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3368171272 wait complete. 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 Processor -- start 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 -- start start 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01ac100140 0x7f01ac06a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.638+0000 7f01b1a0f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac101330 0x7f01ac19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b1a0f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01ac101cf0 0x7f01ac19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b1a0f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac19ee50 con 0x7f01ac101330 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b1a0f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac19efc0 con 0x7f01ac100140 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b1a0f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac19f180 con 0x7f01ac101cf0 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac101330 0x7f01ac19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac101330 0x7f01ac19c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40644/0 (socket says 172.21.15.17:40644) 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 -- 172.21.15.17:0/3259597930 learned_addr learned my addr 172.21.15.17:0/3259597930 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aaffd640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01ac100140 0x7f01ac06a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01ab7fe640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01ac101cf0 0x7f01ac19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 -- 172.21.15.17:0/3259597930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01ac101cf0 msgr2=0x7f01ac19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01ac101cf0 0x7f01ac19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 -- 172.21.15.17:0/3259597930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01ac100140 msgr2=0x7f01ac06a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01ac100140 0x7f01ac06a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 -- 172.21.15.17:0/3259597930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c009560 con 0x7f01ac101330 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01aa7fc640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac101330 0x7f01ac19c050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01a000d950 tx=0x7f01a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b0a0d640 1 -- 172.21.15.17:0/3259597930 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0009c30 con 0x7f01ac101330 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b0a0d640 1 -- 172.21.15.17:0/3259597930 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01a0009d90 con 0x7f01ac101330 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b0a0d640 1 -- 172.21.15.17:0/3259597930 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000f7e0 con 0x7f01ac101330 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac19f460 con 0x7f01ac101330 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.639+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac19f9f0 con 0x7f01ac101330 2024-02-14T15:01:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.640+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac107460 con 0x7f01ac101330 2024-02-14T15:01:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.641+0000 7f01b0a0d640 1 -- 172.21.15.17:0/3259597930 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f01a0010ad0 con 0x7f01ac101330 2024-02-14T15:01:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.642+0000 7f01b0a0d640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f01940a20c0 0x7f01940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.642+0000 7f01aaffd640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f01940a20c0 0x7f01940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.642+0000 7f01b0a0d640 1 -- 172.21.15.17:0/3259597930 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f01a00cdb60 con 0x7f01ac101330 2024-02-14T15:01:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.642+0000 7f01aaffd640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f01940a20c0 0x7f01940a4580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f019c005ab0 tx=0x7f019c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.646+0000 7f01b0a0d640 1 -- 172.21.15.17:0/3259597930 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01a00968e0 con 0x7f01ac101330 2024-02-14T15:01:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.840+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f01ac02cde0 con 0x7f01ac101330 2024-02-14T15:01:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.841+0000 7f01b0a0d640 1 -- 172.21.15.17:0/3259597930 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+592 (secure 0 0 0) 0x7f01a0017260 con 0x7f01ac101330 2024-02-14T15:01:04.842 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3179868836 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3756063081 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/1044675107 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1117796267 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3241397077 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/2869822133 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/2869822133 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3371941300 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/1044675107 2024-02-15T14:53:42.569888+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2319986884 2024-02-15T14:53:15.465600+0000 2024-02-14T15:01:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:listed 10 entries 2024-02-14T15:01:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.842+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f01940a20c0 msgr2=0x7f01940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.842+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f01940a20c0 0x7f01940a4580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f019c005ab0 tx=0x7f019c005fd0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac101330 msgr2=0x7f01ac19c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac101330 0x7f01ac19c050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01a000d950 tx=0x7f01a000de10 comp rx=0 tx=0).stop 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 shutdown_connections 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f01940a20c0 0x7f01940a4580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01ac101330 0x7f01ac19c050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01ac101cf0 0x7f01ac19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 --2- 172.21.15.17:0/3259597930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01ac100140 0x7f01ac06a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 >> 172.21.15.17:0/3259597930 conn(0x7f01ac0fb690 msgr2=0x7f01ac104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 shutdown_connections 2024-02-14T15:01:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:04.843+0000 7f01b1a0f640 1 -- 172.21.15.17:0/3259597930 wait complete. 2024-02-14T15:01:04.854 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4279215633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.854 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3848292403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.854 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2426716813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.854 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/575297743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.855 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2024-02-14T15:01:04.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:04.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-02-14T15:01:04.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[76306]: fuse finished with error 0 and tester_r 0 2024-02-14T15:01:04.882 INFO:teuthology.orchestra.run:waiting for 300 2024-02-14T15:01:04.889 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4279215633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.889 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3848292403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.889 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2426716813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.889 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/575297743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4279215633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3848292403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2426716813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:01:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/575297743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:01:05.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:05 smithi050 ceph-mon[36512]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 304 B/s wr, 0 op/s 2024-02-14T15:01:05.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:05 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3259597930' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:01:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:05 smithi076 ceph-mon[37057]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 304 B/s wr, 0 op/s 2024-02-14T15:01:05.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:05 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3259597930' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:01:06.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:05 smithi017 ceph-mon[32549]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 304 B/s wr, 0 op/s 2024-02-14T15:01:06.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:05 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3259597930' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:01:07.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:07 smithi076 ceph-mon[37057]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-14T15:01:08.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:07 smithi017 ceph-mon[32549]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-14T15:01:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:07 smithi050 ceph-mon[36512]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-14T15:01:09.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:09 smithi076 ceph-mon[37057]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:10.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:09 smithi017 ceph-mon[32549]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:10.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:09 smithi050 ceph-mon[36512]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:10.983 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:01:10.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:01:10.983 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-02-14T15:01:11.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- 172.21.15.17:0/1842779541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 msgr2=0x7f6fb4101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 --2- 172.21.15.17:0/1842779541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb4101680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009900 tx=0x7f6fa802bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- 172.21.15.17:0/1842779541 shutdown_connections 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 --2- 172.21.15.17:0/1842779541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb4102e10 0x7f6fb41096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 --2- 172.21.15.17:0/1842779541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb4102450 0x7f6fb41028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 --2- 172.21.15.17:0/1842779541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb4101680 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- 172.21.15.17:0/1842779541 >> 172.21.15.17:0/1842779541 conn(0x7f6fb40fc810 msgr2=0x7f6fb40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- 172.21.15.17:0/1842779541 shutdown_connections 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- 172.21.15.17:0/1842779541 wait complete. 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 Processor -- start 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- start start 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb4102450 0x7f6fb4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb4102e10 0x7f6fb419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb419edc0 con 0x7f6fb4101260 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb419ef30 con 0x7f6fb4102450 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.162+0000 7f6fbac50640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb419f0f0 con 0x7f6fb4102e10 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55100/0 (socket says 172.21.15.17:55100) 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 -- 172.21.15.17:0/500891834 learned_addr learned my addr 172.21.15.17:0/500891834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb91ed640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb4102e10 0x7f6fb419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 -- 172.21.15.17:0/500891834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb4102e10 msgr2=0x7f6fb419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb4102e10 0x7f6fb419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 -- 172.21.15.17:0/500891834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb4102450 msgr2=0x7f6fb4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb3fff640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb4102450 0x7f6fb4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb4102450 0x7f6fb4197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 -- 172.21.15.17:0/500891834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009560 con 0x7f6fb4101260 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb91ed640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb4102e10 0x7f6fb419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb89ec640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb41972d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000c00 tx=0x7f6fa8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb1ffb640 1 -- 172.21.15.17:0/500891834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004df0 con 0x7f6fb4101260 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb1ffb640 1 -- 172.21.15.17:0/500891834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fa8005430 con 0x7f6fb4101260 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb419f370 con 0x7f6fb4101260 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fb1ffb640 1 -- 172.21.15.17:0/500891834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa803e450 con 0x7f6fb4101260 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.163+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb419f8d0 con 0x7f6fb4101260 2024-02-14T15:01:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.165+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb41085d0 con 0x7f6fb4101260 2024-02-14T15:01:11.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.165+0000 7f6fb1ffb640 1 -- 172.21.15.17:0/500891834 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6fa8003680 con 0x7f6fb4101260 2024-02-14T15:01:11.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.166+0000 7f6fb1ffb640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6fa40a2140 0x7f6fa40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.166+0000 7f6fb3fff640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6fa40a2140 0x7f6fa40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.166+0000 7f6fb1ffb640 1 -- 172.21.15.17:0/500891834 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6fa80f03b0 con 0x7f6fb4101260 2024-02-14T15:01:11.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.166+0000 7f6fb3fff640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6fa40a2140 0x7f6fa40a4600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0005fd0 tx=0x7f6fa0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:11.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.171+0000 7f6fb1ffb640 1 -- 172.21.15.17:0/500891834 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6fa80b9190 con 0x7f6fb4101260 2024-02-14T15:01:11.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.353+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fb402cdb0 con 0x7f6fb4101260 2024-02-14T15:01:11.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.355+0000 7f6fb1ffb640 1 -- 172.21.15.17:0/500891834 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7f6fa80bc850 con 0x7f6fb4101260 2024-02-14T15:01:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:11.357 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:01:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6fa40a2140 msgr2=0x7f6fa40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6fa40a2140 0x7f6fa40a4600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0005fd0 tx=0x7f6fa0005850 comp rx=0 tx=0).stop 2024-02-14T15:01:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 msgr2=0x7f6fb41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb41972d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000c00 tx=0x7f6fa8004270 comp rx=0 tx=0).stop 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 shutdown_connections 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6fa40a2140 0x7f6fa40a4600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb4102e10 0x7f6fb419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6fb4102450 0x7f6fb4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 --2- 172.21.15.17:0/500891834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb4101260 0x7f6fb41972d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 >> 172.21.15.17:0/500891834 conn(0x7f6fb40fc810 msgr2=0x7f6fb41057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 shutdown_connections 2024-02-14T15:01:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.357+0000 7f6fbac50640 1 -- 172.21.15.17:0/500891834 wait complete. 2024-02-14T15:01:11.370 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.471+0000 7f45f8bd6640 1 -- 172.21.15.17:0/3042522685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4104610 msgr2=0x7f45f4106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.471+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/3042522685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4104610 0x7f45f4106a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009900 tx=0x7f45e802bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 -- 172.21.15.17:0/3042522685 shutdown_connections 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/3042522685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4104610 0x7f45f4106a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/3042522685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f4106f40 0x7f45f4109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/3042522685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45f4101ce0 0x7f45f41040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 -- 172.21.15.17:0/3042522685 >> 172.21.15.17:0/3042522685 conn(0x7f45f40fb6d0 msgr2=0x7f45f40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:11.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 -- 172.21.15.17:0/3042522685 shutdown_connections 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 -- 172.21.15.17:0/3042522685 wait complete. 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 Processor -- start 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.472+0000 7f45f8bd6640 1 -- start start 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4101ce0 0x7f45f4101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45f4104610 0x7f45f40ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f4106f40 0x7f45f41001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4100930 con 0x7f45f4101ce0 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4100aa0 con 0x7f45f4104610 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f41a1330 con 0x7f45f4106f40 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4101ce0 0x7f45f4101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4101ce0 0x7f45f4101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55112/0 (socket says 172.21.15.17:55112) 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 -- 172.21.15.17:0/2901342234 learned_addr learned my addr 172.21.15.17:0/2901342234 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f1d9b640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45f4104610 0x7f45f40ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f2d9d640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f4106f40 0x7f45f41001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 -- 172.21.15.17:0/2901342234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f4106f40 msgr2=0x7f45f41001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f4106f40 0x7f45f41001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 -- 172.21.15.17:0/2901342234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45f4104610 msgr2=0x7f45f40ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45f4104610 0x7f45f40ffc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 -- 172.21.15.17:0/2901342234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8009560 con 0x7f45f4101ce0 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f2d9d640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f4106f40 0x7f45f41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f259c640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4101ce0 0x7f45f4101650 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00ea20 tx=0x7f45ec00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45e37fe640 1 -- 172.21.15.17:0/2901342234 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec009700 con 0x7f45f4101ce0 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f41a15e0 con 0x7f45f4101ce0 2024-02-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45e37fe640 1 -- 172.21.15.17:0/2901342234 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45ec00cd80 con 0x7f45f4101ce0 2024-02-14T15:01:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45e37fe640 1 -- 172.21.15.17:0/2901342234 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec018840 con 0x7f45f4101ce0 2024-02-14T15:01:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.473+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f41a1b10 con 0x7f45f4101ce0 2024-02-14T15:01:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.475+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f40fd1c0 con 0x7f45f4101ce0 2024-02-14T15:01:11.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.476+0000 7f45e37fe640 1 -- 172.21.15.17:0/2901342234 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f45ec010a80 con 0x7f45f4101ce0 2024-02-14T15:01:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.476+0000 7f45e37fe640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f45dc0a20c0 0x7f45dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.476+0000 7f45f1d9b640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f45dc0a20c0 0x7f45dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.476+0000 7f45e37fe640 1 -- 172.21.15.17:0/2901342234 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45ec014070 con 0x7f45f4101ce0 2024-02-14T15:01:11.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.477+0000 7f45f1d9b640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f45dc0a20c0 0x7f45dc0a4580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45e8005ab0 tx=0x7f45e8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:11.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.480+0000 7f45e37fe640 1 -- 172.21.15.17:0/2901342234 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f45ec052d20 con 0x7f45f4101ce0 2024-02-14T15:01:11.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.664+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45f40fcc30 con 0x7f45f4101ce0 2024-02-14T15:01:11.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.666+0000 7f45e37fe640 1 -- 172.21.15.17:0/2901342234 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7f45ec097000 con 0x7f45f4101ce0 2024-02-14T15:01:11.668 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:11.668 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f45dc0a20c0 msgr2=0x7f45dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f45dc0a20c0 0x7f45dc0a4580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45e8005ab0 tx=0x7f45e8005fd0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4101ce0 msgr2=0x7f45f4101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4101ce0 0x7f45f4101650 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00ea20 tx=0x7f45ec00eee0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 shutdown_connections 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f45dc0a20c0 0x7f45dc0a4580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f4106f40 0x7f45f41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45f4104610 0x7f45f40ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 --2- 172.21.15.17:0/2901342234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f4101ce0 0x7f45f4101650 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 >> 172.21.15.17:0/2901342234 conn(0x7f45f40fb6d0 msgr2=0x7f45f41027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:11.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 shutdown_connections 2024-02-14T15:01:11.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.668+0000 7f45f8bd6640 1 -- 172.21.15.17:0/2901342234 wait complete. 2024-02-14T15:01:11.679 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:11 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/500891834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:11.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-14T15:01:11.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 -- 172.21.15.17:0/1148107040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8102eb0 msgr2=0x7f9cf8109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/1148107040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8102eb0 0x7f9cf8109740 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009ca0 tx=0x7f9cf002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 -- 172.21.15.17:0/1148107040 shutdown_connections 2024-02-14T15:01:11.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/1148107040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8102eb0 0x7f9cf8109740 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/1148107040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cf81024f0 0x7f9cf8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/1148107040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf80ff520 0x7f9cf80ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 -- 172.21.15.17:0/1148107040 >> 172.21.15.17:0/1148107040 conn(0x7f9cf80fae50 msgr2=0x7f9cf80fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 -- 172.21.15.17:0/1148107040 shutdown_connections 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.785+0000 7f9cffc8f640 1 -- 172.21.15.17:0/1148107040 wait complete. 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 Processor -- start 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 -- start start 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff520 0x7f9cf8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cf81024f0 0x7f9cf8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf8102eb0 0x7f9cf819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf819ed20 con 0x7f9cf80ff520 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf819ee90 con 0x7f9cf81024f0 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cffc8f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf819f050 con 0x7f9cf8102eb0 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff520 0x7f9cf8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff520 0x7f9cf8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55132/0 (socket says 172.21.15.17:55132) 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 -- 172.21.15.17:0/633746887 learned_addr learned my addr 172.21.15.17:0/633746887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfd22a640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cf81024f0 0x7f9cf8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfe22c640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf8102eb0 0x7f9cf819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 -- 172.21.15.17:0/633746887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf8102eb0 msgr2=0x7f9cf819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf8102eb0 0x7f9cf819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 -- 172.21.15.17:0/633746887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cf81024f0 msgr2=0x7f9cf8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cf81024f0 0x7f9cf8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 -- 172.21.15.17:0/633746887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009900 con 0x7f9cf80ff520 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfe22c640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf8102eb0 0x7f9cf819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.786+0000 7f9cfda2b640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff520 0x7f9cf8197230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e9c0 tx=0x7f9cf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.787+0000 7f9ceeffd640 1 -- 172.21.15.17:0/633746887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40096e0 con 0x7f9cf80ff520 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.787+0000 7f9ceeffd640 1 -- 172.21.15.17:0/633746887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cf400cd80 con 0x7f9cf80ff520 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.787+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf819f330 con 0x7f9cf80ff520 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.787+0000 7f9ceeffd640 1 -- 172.21.15.17:0/633746887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4010430 con 0x7f9cf80ff520 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.787+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf819f840 con 0x7f9cf80ff520 2024-02-14T15:01:11.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.788+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf81085c0 con 0x7f9cf80ff520 2024-02-14T15:01:11.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.789+0000 7f9ceeffd640 1 -- 172.21.15.17:0/633746887 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9cf4003680 con 0x7f9cf80ff520 2024-02-14T15:01:11.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.789+0000 7f9ceeffd640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ce40a2140 0x7f9ce40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:11.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.789+0000 7f9cfd22a640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ce40a2140 0x7f9ce40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:11.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.789+0000 7f9ceeffd640 1 -- 172.21.15.17:0/633746887 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9cf4014070 con 0x7f9cf80ff520 2024-02-14T15:01:11.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.790+0000 7f9cfd22a640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ce40a2140 0x7f9ce40a4600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8198740 tx=0x7f9ce800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:11.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:11.793+0000 7f9ceeffd640 1 -- 172.21.15.17:0/633746887 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9cf40970d0 con 0x7f9cf80ff520 2024-02-14T15:01:11.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:11 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/500891834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:11.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:11 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/500891834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:12.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.013+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9cf810ce00 con 0x7f9cf80ff520 2024-02-14T15:01:12.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.556+0000 7f9ceeffd640 1 -- 172.21.15.17:0/633746887 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9cf409a790 con 0x7f9cf80ff520 2024-02-14T15:01:12.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ce40a2140 msgr2=0x7f9ce40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:12.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ce40a2140 0x7f9ce40a4600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8198740 tx=0x7f9ce800b390 comp rx=0 tx=0).stop 2024-02-14T15:01:12.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff520 msgr2=0x7f9cf8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:12.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff520 0x7f9cf8197230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e9c0 tx=0x7f9cf400ee80 comp rx=0 tx=0).stop 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 shutdown_connections 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ce40a2140 0x7f9ce40a4600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf8102eb0 0x7f9cf819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cf81024f0 0x7f9cf8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 --2- 172.21.15.17:0/633746887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff520 0x7f9cf8197230 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 >> 172.21.15.17:0/633746887 conn(0x7f9cf80fae50 msgr2=0x7f9cf8105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 shutdown_connections 2024-02-14T15:01:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.558+0000 7f9cffc8f640 1 -- 172.21.15.17:0/633746887 wait complete. 2024-02-14T15:01:12.573 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-02-14T15:01:12.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-02-14T15:01:12.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.682+0000 7f7ad3513640 1 -- 172.21.15.17:0/2357746552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 msgr2=0x7f7acc109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.682+0000 7f7ad3513640 1 --2- 172.21.15.17:0/2357746552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc109820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009ca0 tx=0x7f7ac402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- 172.21.15.17:0/2357746552 shutdown_connections 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 --2- 172.21.15.17:0/2357746552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc109820 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 --2- 172.21.15.17:0/2357746552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7acc0ff2b0 0x7f7acc0ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 --2- 172.21.15.17:0/2357746552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7acc102840 0x7f7acc102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- 172.21.15.17:0/2357746552 >> 172.21.15.17:0/2357746552 conn(0x7f7acc0fae50 msgr2=0x7f7acc0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- 172.21.15.17:0/2357746552 shutdown_connections 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- 172.21.15.17:0/2357746552 wait complete. 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 Processor -- start 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- start start 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7acc0ff2b0 0x7f7acc197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7acc102840 0x7f7acc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19ed20 con 0x7f7acc0ffc70 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19ee90 con 0x7f7acc102840 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad3513640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19f050 con 0x7f7acc0ff2b0 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.683+0000 7f7ad0aae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55150/0 (socket says 172.21.15.17:55150) 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 -- 172.21.15.17:0/1597961547 learned_addr learned my addr 172.21.15.17:0/1597961547 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad12af640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7acc0ff2b0 0x7f7acc197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 -- 172.21.15.17:0/1597961547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7acc0ff2b0 msgr2=0x7f7acc197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7acc0ff2b0 0x7f7acc197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 -- 172.21.15.17:0/1597961547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7acc102840 msgr2=0x7f7acc19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7acc102840 0x7f7acc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 -- 172.21.15.17:0/1597961547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac4009900 con 0x7f7acc0ffc70 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad0aae640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc197770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c920 tx=0x7f7abc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ac27fc640 1 -- 172.21.15.17:0/1597961547 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc007c70 con 0x7f7acc0ffc70 2024-02-14T15:01:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ac27fc640 1 -- 172.21.15.17:0/1597961547 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7abc007dd0 con 0x7f7acc0ffc70 2024-02-14T15:01:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ac27fc640 1 -- 172.21.15.17:0/1597961547 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc018860 con 0x7f7acc0ffc70 2024-02-14T15:01:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc19f330 con 0x7f7acc0ffc70 2024-02-14T15:01:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.684+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc19f7f0 con 0x7f7acc0ffc70 2024-02-14T15:01:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.685+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc108630 con 0x7f7acc0ffc70 2024-02-14T15:01:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.686+0000 7f7ac27fc640 1 -- 172.21.15.17:0/1597961547 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7abc00fac0 con 0x7f7acc0ffc70 2024-02-14T15:01:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.687+0000 7f7ac27fc640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7ab80a20c0 0x7f7ab80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.687+0000 7f7ad12af640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7ab80a20c0 0x7f7ab80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.687+0000 7f7ac27fc640 1 -- 172.21.15.17:0/1597961547 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7abc0cdfe0 con 0x7f7acc0ffc70 2024-02-14T15:01:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.687+0000 7f7ad12af640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7ab80a20c0 0x7f7ab80a4580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8005fd0 tx=0x7f7ac800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:12.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.691+0000 7f7ac27fc640 1 -- 172.21.15.17:0/1597961547 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7abc096dc0 con 0x7f7acc0ffc70 2024-02-14T15:01:12.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:12 smithi050 ceph-mon[36512]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:12.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:12 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2901342234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:12.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:12 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/633746887' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-14T15:01:12.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:12.915+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f7acc02cf30 con 0x7f7acc0ffc70 2024-02-14T15:01:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:12 smithi076 ceph-mon[37057]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:12 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2901342234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:12 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/633746887' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-14T15:01:13.009 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:12 smithi017 ceph-mon[32549]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:13.009 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:12 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2901342234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:13.009 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:12 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/633746887' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-14T15:01:13.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.566+0000 7f7ac27fc640 1 -- 172.21.15.17:0/1597961547 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7abc09a480 con 0x7f7acc0ffc70 2024-02-14T15:01:13.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7ab80a20c0 msgr2=0x7f7ab80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:13.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7ab80a20c0 0x7f7ab80a4580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8005fd0 tx=0x7f7ac800b390 comp rx=0 tx=0).stop 2024-02-14T15:01:13.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 msgr2=0x7f7acc197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:13.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc197770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c920 tx=0x7f7abc00cde0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 shutdown_connections 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7ab80a20c0 0x7f7ab80a4580 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7acc0ffc70 0x7f7acc197770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.568+0000 7f7ad3513640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7acc102840 0x7f7acc19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.569+0000 7f7ad3513640 1 --2- 172.21.15.17:0/1597961547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7acc0ff2b0 0x7f7acc197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.569+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 >> 172.21.15.17:0/1597961547 conn(0x7f7acc0fae50 msgr2=0x7f7acc105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.569+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 shutdown_connections 2024-02-14T15:01:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.569+0000 7f7ad3513640 1 -- 172.21.15.17:0/1597961547 wait complete. 2024-02-14T15:01:13.585 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 -- 172.21.15.17:0/2359471785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484102430 msgr2=0x7f44841028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 --2- 172.21.15.17:0/2359471785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484102430 0x7f44841028b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4480009900 tx=0x7f448002bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 -- 172.21.15.17:0/2359471785 shutdown_connections 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 --2- 172.21.15.17:0/2359471785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484102430 0x7f44841028b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 --2- 172.21.15.17:0/2359471785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4484102df0 0x7f4484109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 --2- 172.21.15.17:0/2359471785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4484101240 0x7f4484101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 -- 172.21.15.17:0/2359471785 >> 172.21.15.17:0/2359471785 conn(0x7f44840fc7b0 msgr2=0x7f44840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 -- 172.21.15.17:0/2359471785 shutdown_connections 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 -- 172.21.15.17:0/2359471785 wait complete. 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 Processor -- start 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.692+0000 7f448bf93640 1 -- start start 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484101240 0x7f44841972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4484102430 0x7f4484197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4484102df0 0x7f448419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419edc0 con 0x7f4484101240 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419ef30 con 0x7f4484102430 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419f0f0 con 0x7f4484102df0 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484101240 0x7f44841972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484101240 0x7f44841972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55158/0 (socket says 172.21.15.17:55158) 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 -- 172.21.15.17:0/4116809978 learned_addr learned my addr 172.21.15.17:0/4116809978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448952e640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4484102430 0x7f4484197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448a530640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4484102df0 0x7f448419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 -- 172.21.15.17:0/4116809978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4484102df0 msgr2=0x7f448419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4484102df0 0x7f448419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 -- 172.21.15.17:0/4116809978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4484102430 msgr2=0x7f4484197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4484102430 0x7f4484197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 -- 172.21.15.17:0/4116809978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009560 con 0x7f4484101240 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448952e640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4484102430 0x7f4484197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4489d2f640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484101240 0x7f44841972d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f447800c970 tx=0x7f447800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4476ffd640 1 -- 172.21.15.17:0/4116809978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478004220 con 0x7f4484101240 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4476ffd640 1 -- 172.21.15.17:0/4116809978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4478007e90 con 0x7f4484101240 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448419f3d0 con 0x7f4484101240 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f4476ffd640 1 -- 172.21.15.17:0/4116809978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478004a00 con 0x7f4484101240 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.693+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448419f8e0 con 0x7f4484101240 2024-02-14T15:01:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.694+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44841085e0 con 0x7f4484101240 2024-02-14T15:01:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.695+0000 7f4476ffd640 1 -- 172.21.15.17:0/4116809978 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f44780079b0 con 0x7f4484101240 2024-02-14T15:01:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.696+0000 7f4476ffd640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f44700a2140 0x7f44700a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.696+0000 7f448952e640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f44700a2140 0x7f44700a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.696+0000 7f4476ffd640 1 -- 172.21.15.17:0/4116809978 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f44780cdf40 con 0x7f4484101240 2024-02-14T15:01:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.697+0000 7f448952e640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f44700a2140 0x7f44700a4600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44841987e0 tx=0x7f4480005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.701+0000 7f4476ffd640 1 -- 172.21.15.17:0/4116809978 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4478096d20 con 0x7f4484101240 2024-02-14T15:01:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/633746887' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-14T15:01:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:13 smithi050 ceph-mon[36512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:01:13.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1597961547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-14T15:01:13.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:13.919+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f448402ce50 con 0x7f4484101240 2024-02-14T15:01:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/633746887' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-14T15:01:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:13 smithi076 ceph-mon[37057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:01:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1597961547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-14T15:01:14.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/633746887' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-14T15:01:14.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:13 smithi017 ceph-mon[32549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:01:14.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1597961547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-14T15:01:14.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.572+0000 7f4476ffd640 1 -- 172.21.15.17:0/4116809978 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f447801d020 con 0x7f4484101240 2024-02-14T15:01:14.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f44700a2140 msgr2=0x7f44700a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:14.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f44700a2140 0x7f44700a4600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44841987e0 tx=0x7f4480005be0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484101240 msgr2=0x7f44841972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:14.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484101240 0x7f44841972d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f447800c970 tx=0x7f447800ce30 comp rx=0 tx=0).stop 2024-02-14T15:01:14.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 shutdown_connections 2024-02-14T15:01:14.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f44700a2140 0x7f44700a4600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4484102df0 0x7f448419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4484102430 0x7f4484197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 --2- 172.21.15.17:0/4116809978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4484101240 0x7f44841972d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 >> 172.21.15.17:0/4116809978 conn(0x7f44840fc7b0 msgr2=0x7f44841057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 shutdown_connections 2024-02-14T15:01:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.575+0000 7f448bf93640 1 -- 172.21.15.17:0/4116809978 wait complete. 2024-02-14T15:01:14.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-14T15:01:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.705+0000 7ff629859640 1 -- 172.21.15.17:0/504382122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624100150 msgr2=0x7ff624100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.705+0000 7ff629859640 1 --2- 172.21.15.17:0/504382122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624100150 0x7ff624100570 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff614009900 tx=0x7ff61402bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- 172.21.15.17:0/504382122 shutdown_connections 2024-02-14T15:01:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 --2- 172.21.15.17:0/504382122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff624101d00 0x7ff624108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 --2- 172.21.15.17:0/504382122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff624101340 0x7ff6241017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 --2- 172.21.15.17:0/504382122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624100150 0x7ff624100570 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- 172.21.15.17:0/504382122 >> 172.21.15.17:0/504382122 conn(0x7ff6240fb6c0 msgr2=0x7ff6240fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- 172.21.15.17:0/504382122 shutdown_connections 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- 172.21.15.17:0/504382122 wait complete. 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 Processor -- start 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- start start 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff624100150 0x7ff6241040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff624101340 0x7ff624102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624101d00 0x7ff624102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241033c0 con 0x7ff624101d00 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624103530 con 0x7ff624101340 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff629859640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241a1390 con 0x7ff624100150 2024-02-14T15:01:14.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff6237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624101d00 0x7ff624102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.706+0000 7ff6237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624101d00 0x7ff624102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55190/0 (socket says 172.21.15.17:55190) 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff6237fe640 1 -- 172.21.15.17:0/54112178 learned_addr learned my addr 172.21.15.17:0/54112178 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff622ffd640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff624100150 0x7ff6241040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff61bfff640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff624101340 0x7ff624102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff6237fe640 1 -- 172.21.15.17:0/54112178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff624100150 msgr2=0x7ff6241040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff6237fe640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff624100150 0x7ff6241040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff6237fe640 1 -- 172.21.15.17:0/54112178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff624101340 msgr2=0x7ff624102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff6237fe640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff624101340 0x7ff624102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff6237fe640 1 -- 172.21.15.17:0/54112178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614009560 con 0x7ff624101d00 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff6237fe640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624101d00 0x7ff624102c60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00ead0 tx=0x7ff61c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff628857640 1 -- 172.21.15.17:0/54112178 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00cde0 con 0x7ff624101d00 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff628857640 1 -- 172.21.15.17:0/54112178 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff61c005020 con 0x7ff624101d00 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6241a1640 con 0x7ff624101d00 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff628857640 1 -- 172.21.15.17:0/54112178 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c0116b0 con 0x7ff624101d00 2024-02-14T15:01:14.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.707+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6241a1b50 con 0x7ff624101d00 2024-02-14T15:01:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.708+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62410bb60 con 0x7ff624101d00 2024-02-14T15:01:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.709+0000 7ff628857640 1 -- 172.21.15.17:0/54112178 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff61c005520 con 0x7ff624101d00 2024-02-14T15:01:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.710+0000 7ff628857640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff6100a20c0 0x7ff6100a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.710+0000 7ff622ffd640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff6100a20c0 0x7ff6100a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.710+0000 7ff628857640 1 -- 172.21.15.17:0/54112178 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff61c0ce170 con 0x7ff624101d00 2024-02-14T15:01:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.710+0000 7ff622ffd640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff6100a20c0 0x7ff6100a4580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff614006010 tx=0x7ff61402c740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.714+0000 7ff628857640 1 -- 172.21.15.17:0/54112178 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff61c096ef0 con 0x7ff624101d00 2024-02-14T15:01:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-14T15:01:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1597961547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-14T15:01:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-14T15:01:14.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:01:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-14T15:01:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-14T15:01:14.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:14 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4116809978' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-14T15:01:14.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:14.936+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7ff6241a2050 con 0x7ff624101d00 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1597961547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-14T15:01:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:14 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4116809978' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1597961547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-14T15:01:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:14 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4116809978' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-14T15:01:15.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.551+0000 7ff628857640 1 -- 172.21.15.17:0/54112178 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff61c01f0b0 con 0x7ff624101d00 2024-02-14T15:01:15.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff6100a20c0 msgr2=0x7ff6100a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:15.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff6100a20c0 0x7ff6100a4580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff614006010 tx=0x7ff61402c740 comp rx=0 tx=0).stop 2024-02-14T15:01:15.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624101d00 msgr2=0x7ff624102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:15.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624101d00 0x7ff624102c60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00ead0 tx=0x7ff61c00ef90 comp rx=0 tx=0).stop 2024-02-14T15:01:15.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 shutdown_connections 2024-02-14T15:01:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff6100a20c0 0x7ff6100a4580 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff624101d00 0x7ff624102c60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff624101340 0x7ff624102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 --2- 172.21.15.17:0/54112178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff624100150 0x7ff6241040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 >> 172.21.15.17:0/54112178 conn(0x7ff6240fb6c0 msgr2=0x7ff6240fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 shutdown_connections 2024-02-14T15:01:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.554+0000 7ff629859640 1 -- 172.21.15.17:0/54112178 wait complete. 2024-02-14T15:01:15.571 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-02-14T15:01:15.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-14T15:01:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.681+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2945253693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe08100c40 msgr2=0x7fbe081014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.681+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2945253693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe08100c40 0x7fbe081014c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009ca0 tx=0x7fbdf802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2945253693 shutdown_connections 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2945253693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe08100c40 0x7fbe081014c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2945253693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe08100280 0x7fbe08100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2945253693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe080ff520 0x7fbe080ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2945253693 >> 172.21.15.17:0/2945253693 conn(0x7fbe080fae50 msgr2=0x7fbe080fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2945253693 shutdown_connections 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2945253693 wait complete. 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 Processor -- start 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- start start 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe080ff520 0x7fbe08199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe08100280 0x7fbe08199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe08100c40 0x7fbe081a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081a0f30 con 0x7fbe080ff520 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081a10a0 con 0x7fbe08100c40 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0f71e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081a1260 con 0x7fbe08100280 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.682+0000 7fbe0d4ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe080ff520 0x7fbe08199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe080ff520 0x7fbe08199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55218/0 (socket says 172.21.15.17:55218) 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 -- 172.21.15.17:0/2435602593 learned_addr learned my addr 172.21.15.17:0/2435602593 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0ccb9640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe08100280 0x7fbe08199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 -- 172.21.15.17:0/2435602593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe08100280 msgr2=0x7fbe08199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0dcbb640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe08100c40 0x7fbe081a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe08100280 0x7fbe08199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 -- 172.21.15.17:0/2435602593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe08100c40 msgr2=0x7fbe081a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe08100c40 0x7fbe081a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 -- 172.21.15.17:0/2435602593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8009900 con 0x7fbe080ff520 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0dcbb640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe08100c40 0x7fbe081a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0d4ba640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe080ff520 0x7fbe08199440 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b4a0 tx=0x7fbe0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:15.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe067fc640 1 -- 172.21.15.17:0/2435602593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00010610 con 0x7fbe080ff520 2024-02-14T15:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe067fc640 1 -- 172.21.15.17:0/2435602593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe00010c50 con 0x7fbe080ff520 2024-02-14T15:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe067fc640 1 -- 172.21.15.17:0/2435602593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00017e20 con 0x7fbe080ff520 2024-02-14T15:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe081a1540 con 0x7fbe080ff520 2024-02-14T15:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.683+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe081a1a00 con 0x7fbe080ff520 2024-02-14T15:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.684+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0810b100 con 0x7fbe080ff520 2024-02-14T15:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.685+0000 7fbe067fc640 1 -- 172.21.15.17:0/2435602593 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbe00017480 con 0x7fbe080ff520 2024-02-14T15:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.686+0000 7fbe067fc640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbdf00a20c0 0x7fbdf00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.686+0000 7fbe067fc640 1 -- 172.21.15.17:0/2435602593 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbe000cd8f0 con 0x7fbe080ff520 2024-02-14T15:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.686+0000 7fbe0ccb9640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbdf00a20c0 0x7fbdf00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.686+0000 7fbe0ccb9640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbdf00a20c0 0x7fbdf00a4580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0058c0 tx=0x7fbdfc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.690+0000 7fbe067fc640 1 -- 172.21.15.17:0/2435602593 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe000966d0 con 0x7fbe080ff520 2024-02-14T15:01:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4116809978' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-14T15:01:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] up:active 2024-02-14T15:01:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-14T15:01:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-14T15:01:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: Cluster is now healthy 2024-02-14T15:01:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-14T15:01:15.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-14T15:01:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-14T15:01:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/54112178' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-14T15:01:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/54112178' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-14T15:01:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:active 2024-02-14T15:01:15.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:15 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:01:15.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.897+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe0802cec0 con 0x7fbdf00a20c0 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4116809978' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] up:active 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: Cluster is now healthy 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-14T15:01:15.905 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/54112178' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-14T15:01:15.906 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/54112178' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-14T15:01:15.906 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:active 2024-02-14T15:01:15.906 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:01:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.920+0000 7fbe067fc640 1 -- 172.21.15.17:0/2435602593 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbe0802cec0 con 0x7fbdf00a20c0 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbdf00a20c0 msgr2=0x7fbdf00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbdf00a20c0 0x7fbdf00a4580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0058c0 tx=0x7fbdfc005850 comp rx=0 tx=0).stop 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe080ff520 msgr2=0x7fbe08199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe080ff520 0x7fbe08199440 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b4a0 tx=0x7fbe0000b960 comp rx=0 tx=0).stop 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 shutdown_connections 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbdf00a20c0 0x7fbdf00a4580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe08100c40 0x7fbe081a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe08100280 0x7fbe08199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 --2- 172.21.15.17:0/2435602593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe080ff520 0x7fbe08199440 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 >> 172.21.15.17:0/2435602593 conn(0x7fbe080fae50 msgr2=0x7fbe080fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 shutdown_connections 2024-02-14T15:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:15.922+0000 7fbe0f71e640 1 -- 172.21.15.17:0/2435602593 wait complete. 2024-02-14T15:01:15.934 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4116809978' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] up:active 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: Cluster is now healthy 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/54112178' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-14T15:01:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/54112178' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-14T15:01:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:active 2024-02-14T15:01:15.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:15 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 -- 172.21.15.17:0/2779149459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c101d30 msgr2=0x7fef3c104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 --2- 172.21.15.17:0/2779149459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c101d30 0x7fef3c104120 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fef34009900 tx=0x7fef3402bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 -- 172.21.15.17:0/2779149459 shutdown_connections 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 --2- 172.21.15.17:0/2779149459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3c106f90 0x7fef3c109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 --2- 172.21.15.17:0/2779149459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef3c104660 0x7fef3c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 --2- 172.21.15.17:0/2779149459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c101d30 0x7fef3c104120 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 -- 172.21.15.17:0/2779149459 >> 172.21.15.17:0/2779149459 conn(0x7fef3c0fb6a0 msgr2=0x7fef3c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 -- 172.21.15.17:0/2779149459 shutdown_connections 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.040+0000 7fef447f4640 1 -- 172.21.15.17:0/2779149459 wait complete. 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 Processor -- start 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 -- start start 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3c101d30 0x7fef3c1016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef3c104660 0x7fef3c0ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c106f90 0x7fef3c100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c100980 con 0x7fef3c106f90 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c100af0 con 0x7fef3c101d30 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef447f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c1a1330 con 0x7fef3c104660 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42d91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c106f90 0x7fef3c100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42d91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c106f90 0x7fef3c100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55248/0 (socket says 172.21.15.17:55248) 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42d91640 1 -- 172.21.15.17:0/180793960 learned_addr learned my addr 172.21.15.17:0/180793960 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef41d8f640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef3c104660 0x7fef3c0ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42590640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3c101d30 0x7fef3c1016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42d91640 1 -- 172.21.15.17:0/180793960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef3c104660 msgr2=0x7fef3c0ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42d91640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef3c104660 0x7fef3c0ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42d91640 1 -- 172.21.15.17:0/180793960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3c101d30 msgr2=0x7fef3c1016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.041+0000 7fef42d91640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3c101d30 0x7fef3c1016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.042+0000 7fef42d91640 1 -- 172.21.15.17:0/180793960 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34009560 con 0x7fef3c106f90 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.042+0000 7fef42d91640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c106f90 0x7fef3c100220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00cb00 tx=0x7fef2c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.042+0000 7fef3b7fe640 1 -- 172.21.15.17:0/180793960 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00ae90 con 0x7fef3c106f90 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.042+0000 7fef3b7fe640 1 -- 172.21.15.17:0/180793960 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef2c005020 con 0x7fef3c106f90 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.042+0000 7fef3b7fe640 1 -- 172.21.15.17:0/180793960 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00f6b0 con 0x7fef3c106f90 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.042+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c1a15e0 con 0x7fef3c106f90 2024-02-14T15:01:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.042+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c1a1aa0 con 0x7fef3c106f90 2024-02-14T15:01:16.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.043+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c0fd190 con 0x7fef3c106f90 2024-02-14T15:01:16.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.044+0000 7fef3b7fe640 1 -- 172.21.15.17:0/180793960 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fef2c005520 con 0x7fef3c106f90 2024-02-14T15:01:16.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.044+0000 7fef3b7fe640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef240a20c0 0x7fef240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.045+0000 7fef42590640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef240a20c0 0x7fef240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.045+0000 7fef3b7fe640 1 -- 172.21.15.17:0/180793960 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fef2c0cdea0 con 0x7fef3c106f90 2024-02-14T15:01:16.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.045+0000 7fef42590640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef240a20c0 0x7fef240a4580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef3402c6a0 tx=0x7fef34005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.050+0000 7fef3b7fe640 1 -- 172.21.15.17:0/180793960 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef2c096c80 con 0x7fef3c106f90 2024-02-14T15:01:16.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.257+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fef3c02cec0 con 0x7fef240a20c0 2024-02-14T15:01:16.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:01:15.902+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:01:16.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:01:15.902+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:01:16.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:01:15.902+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:01:16.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:01:15.902+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:01:16.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:01:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:01:15.902+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:01:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.281+0000 7fef3b7fe640 1 -- 172.21.15.17:0/180793960 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fef3c02cec0 con 0x7fef240a20c0 2024-02-14T15:01:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef240a20c0 msgr2=0x7fef240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef240a20c0 0x7fef240a4580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef3402c6a0 tx=0x7fef34005fd0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c106f90 msgr2=0x7fef3c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c106f90 0x7fef3c100220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00cb00 tx=0x7fef2c00a250 comp rx=0 tx=0).stop 2024-02-14T15:01:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 shutdown_connections 2024-02-14T15:01:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef240a20c0 0x7fef240a4580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c106f90 0x7fef3c100220 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef3c104660 0x7fef3c0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 --2- 172.21.15.17:0/180793960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef3c101d30 0x7fef3c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 >> 172.21.15.17:0/180793960 conn(0x7fef3c0fb6a0 msgr2=0x7fef3c1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 shutdown_connections 2024-02-14T15:01:16.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.284+0000 7fef447f4640 1 -- 172.21.15.17:0/180793960 wait complete. 2024-02-14T15:01:16.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-14T15:01:16.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 -- 172.21.15.17:0/1648788649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101d00 msgr2=0x7fcaf8108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1648788649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101d00 0x7fcaf8108590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009ca0 tx=0x7fcaf002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 -- 172.21.15.17:0/1648788649 shutdown_connections 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1648788649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101d00 0x7fcaf8108590 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1648788649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf8101340 0x7fcaf81017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1648788649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaf8100150 0x7fcaf8100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 -- 172.21.15.17:0/1648788649 >> 172.21.15.17:0/1648788649 conn(0x7fcaf80fb6e0 msgr2=0x7fcaf80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 -- 172.21.15.17:0/1648788649 shutdown_connections 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 -- 172.21.15.17:0/1648788649 wait complete. 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 Processor -- start 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.402+0000 7fcafffa0640 1 -- start start 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaf8100150 0x7fcaf8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101340 0x7fcaf81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf8101d00 0x7fcaf819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf819ed70 con 0x7fcaf8101340 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf819eee0 con 0x7fcaf8100150 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf819f0a0 con 0x7fcaf8101d00 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101340 0x7fcaf81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101340 0x7fcaf81977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55262/0 (socket says 172.21.15.17:55262) 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 -- 172.21.15.17:0/1724655560 learned_addr learned my addr 172.21.15.17:0/1724655560 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafe53d640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf8101d00 0x7fcaf819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafdd3c640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaf8100150 0x7fcaf8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 -- 172.21.15.17:0/1724655560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf8101d00 msgr2=0x7fcaf819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf8101d00 0x7fcaf819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 -- 172.21.15.17:0/1724655560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaf8100150 msgr2=0x7fcaf8197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaf8100150 0x7fcaf8197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 -- 172.21.15.17:0/1724655560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0009900 con 0x7fcaf8101340 2024-02-14T15:01:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafd53b640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101340 0x7fcaf81977c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9f0 tx=0x7fcae800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcaeeffd640 1 -- 172.21.15.17:0/1724655560 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8009720 con 0x7fcaf8101340 2024-02-14T15:01:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcaeeffd640 1 -- 172.21.15.17:0/1724655560 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae800cd80 con 0x7fcaf8101340 2024-02-14T15:01:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf819f380 con 0x7fcaf8101340 2024-02-14T15:01:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcaeeffd640 1 -- 172.21.15.17:0/1724655560 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8010690 con 0x7fcaf8101340 2024-02-14T15:01:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.403+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf819f840 con 0x7fcaf8101340 2024-02-14T15:01:16.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.404+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf810bb60 con 0x7fcaf8101340 2024-02-14T15:01:16.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.405+0000 7fcaeeffd640 1 -- 172.21.15.17:0/1724655560 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcae80046c0 con 0x7fcaf8101340 2024-02-14T15:01:16.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.406+0000 7fcaeeffd640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcae40a20c0 0x7fcae40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.406+0000 7fcafdd3c640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcae40a20c0 0x7fcae40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.406+0000 7fcaeeffd640 1 -- 172.21.15.17:0/1724655560 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcae8014070 con 0x7fcaf8101340 2024-02-14T15:01:16.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.407+0000 7fcafdd3c640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcae40a20c0 0x7fcae40a4580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005fd0 tx=0x7fcaf4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:16.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.411+0000 7fcaeeffd640 1 -- 172.21.15.17:0/1724655560 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcae8097130 con 0x7fcaf8101340 2024-02-14T15:01:16.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.621+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaf80fd850 con 0x7fcae40a20c0 2024-02-14T15:01:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.625+0000 7fcaeeffd640 1 -- 172.21.15.17:0/1724655560 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fcaf80fd850 con 0x7fcae40a20c0 2024-02-14T15:01:16.627 INFO:teuthology.orchestra.run.smithi017.stdout:/volumes/qa/sv_0/73ed429c-56eb-46b0-9009-eb6e3c8e1a14 2024-02-14T15:01:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcae40a20c0 msgr2=0x7fcae40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcae40a20c0 0x7fcae40a4580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005fd0 tx=0x7fcaf4005850 comp rx=0 tx=0).stop 2024-02-14T15:01:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101340 msgr2=0x7fcaf81977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101340 0x7fcaf81977c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9f0 tx=0x7fcae800edf0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 shutdown_connections 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcae40a20c0 0x7fcae40a4580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf8101340 0x7fcaf81977c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf8101d00 0x7fcaf819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 --2- 172.21.15.17:0/1724655560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcaf8100150 0x7fcaf8197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 >> 172.21.15.17:0/1724655560 conn(0x7fcaf80fb6e0 msgr2=0x7fcaf80fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 shutdown_connections 2024-02-14T15:01:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.627+0000 7fcafffa0640 1 -- 172.21.15.17:0/1724655560 wait complete. 2024-02-14T15:01:16.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-14T15:01:16.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 -- 172.21.15.17:0/865520510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284102de0 msgr2=0x7fb284109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/865520510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284102de0 0x7fb284109670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009ca0 tx=0x7fb27c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 -- 172.21.15.17:0/865520510 shutdown_connections 2024-02-14T15:01:16.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/865520510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284102de0 0x7fb284109670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/865520510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb284102420 0x7fb2841028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/865520510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb284101230 0x7fb284101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 -- 172.21.15.17:0/865520510 >> 172.21.15.17:0/865520510 conn(0x7fb2840fc760 msgr2=0x7fb2840febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 -- 172.21.15.17:0/865520510 shutdown_connections 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.747+0000 7fb28bc5e640 1 -- 172.21.15.17:0/865520510 wait complete. 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 Processor -- start 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 -- start start 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284101230 0x7fb284197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb284102420 0x7fb284197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb284102de0 0x7fb28419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28419ed30 con 0x7fb284101230 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28419eea0 con 0x7fb284102de0 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28bc5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28419f060 con 0x7fb284102420 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284101230 0x7fb284197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284101230 0x7fb284197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55282/0 (socket says 172.21.15.17:55282) 2024-02-14T15:01:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 -- 172.21.15.17:0/3201639665 learned_addr learned my addr 172.21.15.17:0/3201639665 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb28a1fb640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb284102de0 0x7fb28419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2891f9640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb284102420 0x7fb284197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 -- 172.21.15.17:0/3201639665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb284102420 msgr2=0x7fb284197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb284102420 0x7fb284197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 -- 172.21.15.17:0/3201639665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb284102de0 msgr2=0x7fb28419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb284102de0 0x7fb28419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 -- 172.21.15.17:0/3201639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c009900 con 0x7fb284101230 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.748+0000 7fb2899fa640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284101230 0x7fb284197240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb28000d950 tx=0x7fb28000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.749+0000 7fb27affd640 1 -- 172.21.15.17:0/3201639665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280009c30 con 0x7fb284101230 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.749+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28419f340 con 0x7fb284101230 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.749+0000 7fb27affd640 1 -- 172.21.15.17:0/3201639665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb280009d90 con 0x7fb284101230 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.749+0000 7fb27affd640 1 -- 172.21.15.17:0/3201639665 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28000f720 con 0x7fb284101230 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.749+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28419f850 con 0x7fb284101230 2024-02-14T15:01:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.750+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb284108480 con 0x7fb284101230 2024-02-14T15:01:16.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.751+0000 7fb27affd640 1 -- 172.21.15.17:0/3201639665 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb280010a60 con 0x7fb284101230 2024-02-14T15:01:16.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.751+0000 7fb27affd640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2700a20c0 0x7fb2700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:16.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.751+0000 7fb2891f9640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2700a20c0 0x7fb2700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:16.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.751+0000 7fb27affd640 1 -- 172.21.15.17:0/3201639665 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb2800cdc40 con 0x7fb284101230 2024-02-14T15:01:16.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.752+0000 7fb2891f9640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2700a20c0 0x7fb2700a4580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb284198750 tx=0x7fb274005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:16.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.755+0000 7fb27affd640 1 -- 172.21.15.17:0/3201639665 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb280096a20 con 0x7fb284101230 2024-02-14T15:01:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:16 smithi050 ceph-mon[36512]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-02-14T15:01:16.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.964+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb28402ce20 con 0x7fb2700a20c0 2024-02-14T15:01:16.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:16 smithi076 ceph-mon[37057]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-02-14T15:01:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.990+0000 7fb27affd640 1 -- 172.21.15.17:0/3201639665 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb28402ce20 con 0x7fb2700a20c0 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2700a20c0 msgr2=0x7fb2700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2700a20c0 0x7fb2700a4580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb284198750 tx=0x7fb274005f90 comp rx=0 tx=0).stop 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284101230 msgr2=0x7fb284197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284101230 0x7fb284197240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb28000d950 tx=0x7fb28000de10 comp rx=0 tx=0).stop 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 shutdown_connections 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2700a20c0 0x7fb2700a4580 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb284102de0 0x7fb28419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb284102420 0x7fb284197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 --2- 172.21.15.17:0/3201639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284101230 0x7fb284197240 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:16.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 >> 172.21.15.17:0/3201639665 conn(0x7fb2840fc760 msgr2=0x7fb284105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:16.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 shutdown_connections 2024-02-14T15:01:16.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:16.992+0000 7fb28bc5e640 1 -- 172.21.15.17:0/3201639665 wait complete. 2024-02-14T15:01:17.002 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:16 smithi017 ceph-mon[32549]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-02-14T15:01:17.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 -- 172.21.15.17:0/3868015113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64102df0 msgr2=0x7fac64109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3868015113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64102df0 0x7fac64109680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009ca0 tx=0x7fac5c02c100 comp rx=0 tx=0).stop 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 -- 172.21.15.17:0/3868015113 shutdown_connections 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3868015113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64102df0 0x7fac64109680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3868015113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac64102430 0x7fac641028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3868015113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64101240 0x7fac64101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 -- 172.21.15.17:0/3868015113 >> 172.21.15.17:0/3868015113 conn(0x7fac640fc7b0 msgr2=0x7fac640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 -- 172.21.15.17:0/3868015113 shutdown_connections 2024-02-14T15:01:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.111+0000 7fac68c2a640 1 -- 172.21.15.17:0/3868015113 wait complete. 2024-02-14T15:01:17.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 Processor -- start 2024-02-14T15:01:17.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 -- start start 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64101240 0x7fac64197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64102430 0x7fac64197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac64102df0 0x7fac6419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6419ec40 con 0x7fac64101240 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6419edb0 con 0x7fac64102430 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac68c2a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6419ef70 con 0x7fac64102df0 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64101240 0x7fac64197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64101240 0x7fac64197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55306/0 (socket says 172.21.15.17:55306) 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 -- 172.21.15.17:0/3106994144 learned_addr learned my addr 172.21.15.17:0/3106994144 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac62d9d640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac64102df0 0x7fac6419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac5bfff640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64102430 0x7fac64197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 -- 172.21.15.17:0/3106994144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac64102df0 msgr2=0x7fac6419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac64102df0 0x7fac6419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 -- 172.21.15.17:0/3106994144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64102430 msgr2=0x7fac64197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64102430 0x7fac64197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 -- 172.21.15.17:0/3106994144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c009900 con 0x7fac64101240 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac62d9d640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac64102df0 0x7fac6419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.112+0000 7fac6259c640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64101240 0x7fac64197150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c970 tx=0x7fac5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.113+0000 7fac5b7fe640 1 -- 172.21.15.17:0/3106994144 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54007c10 con 0x7fac64101240 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.113+0000 7fac5b7fe640 1 -- 172.21.15.17:0/3106994144 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac54007d70 con 0x7fac64101240 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.113+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6419f250 con 0x7fac64101240 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.113+0000 7fac5b7fe640 1 -- 172.21.15.17:0/3106994144 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5400f8a0 con 0x7fac64101240 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.113+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6419f7b0 con 0x7fac64101240 2024-02-14T15:01:17.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.114+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac64108490 con 0x7fac64101240 2024-02-14T15:01:17.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.115+0000 7fac5b7fe640 1 -- 172.21.15.17:0/3106994144 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fac54018af0 con 0x7fac64101240 2024-02-14T15:01:17.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.115+0000 7fac5b7fe640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac500a2140 0x7fac500a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.115+0000 7fac5bfff640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac500a2140 0x7fac500a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.115+0000 7fac5b7fe640 1 -- 172.21.15.17:0/3106994144 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fac54013070 con 0x7fac64101240 2024-02-14T15:01:17.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.116+0000 7fac5bfff640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac500a2140 0x7fac500a4600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac4c005e10 tx=0x7fac4c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:17.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.120+0000 7fac5b7fe640 1 -- 172.21.15.17:0/3106994144 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac54097050 con 0x7fac64101240 2024-02-14T15:01:17.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.332+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fac6402cec0 con 0x7fac500a2140 2024-02-14T15:01:17.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.335+0000 7fac5b7fe640 1 -- 172.21.15.17:0/3106994144 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fac6402cec0 con 0x7fac500a2140 2024-02-14T15:01:17.336 INFO:teuthology.orchestra.run.smithi017.stdout:/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f 2024-02-14T15:01:17.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac500a2140 msgr2=0x7fac500a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac500a2140 0x7fac500a4600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac4c005e10 tx=0x7fac4c005da0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64101240 msgr2=0x7fac64197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64101240 0x7fac64197150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c970 tx=0x7fac5400ce30 comp rx=0 tx=0).stop 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 shutdown_connections 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac500a2140 0x7fac500a4600 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac64102df0 0x7fac6419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac64102430 0x7fac64197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 --2- 172.21.15.17:0/3106994144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac64101240 0x7fac64197150 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 >> 172.21.15.17:0/3106994144 conn(0x7fac640fc7b0 msgr2=0x7fac64105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 shutdown_connections 2024-02-14T15:01:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.337+0000 7fac68c2a640 1 -- 172.21.15.17:0/3106994144 wait complete. 2024-02-14T15:01:17.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 -- 172.21.15.17:0/2249533914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 msgr2=0x7fbbd4102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/2249533914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4102930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc002bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 -- 172.21.15.17:0/2249533914 shutdown_connections 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/2249533914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4102930 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/2249533914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbd4102e70 0x7fbbd4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/2249533914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbd41012c0 0x7fbbd41016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 -- 172.21.15.17:0/2249533914 >> 172.21.15.17:0/2249533914 conn(0x7fbbd40fc850 msgr2=0x7fbbd40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 -- 172.21.15.17:0/2249533914 shutdown_connections 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 -- 172.21.15.17:0/2249533914 wait complete. 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 Processor -- start 2024-02-14T15:01:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.451+0000 7fbbdaafb640 1 -- start start 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbdaafb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbd41012c0 0x7fbbd41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbdaafb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbdaafb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbd4102e70 0x7fbbd419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbdaafb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd419edd0 con 0x7fbbd41024b0 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbdaafb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd419ef40 con 0x7fbbd4102e70 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbdaafb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd419f100 con 0x7fbbd41012c0 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55330/0 (socket says 172.21.15.17:55330) 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 -- 172.21.15.17:0/1213254447 learned_addr learned my addr 172.21.15.17:0/1213254447 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd8897640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbd41012c0 0x7fbbd41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd9098640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbd4102e70 0x7fbbd419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 -- 172.21.15.17:0/1213254447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbd41012c0 msgr2=0x7fbbd41972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbd41012c0 0x7fbbd41972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 -- 172.21.15.17:0/1213254447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbd4102e70 msgr2=0x7fbbd419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbd4102e70 0x7fbbd419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 -- 172.21.15.17:0/1213254447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0009560 con 0x7fbbd41024b0 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd8897640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbd41012c0 0x7fbbd41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd9098640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbd4102e70 0x7fbbd419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd3fff640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4197820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbbc002c840 tx=0x7fbbc0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0004a80 con 0x7fbbd41024b0 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbc00052d0 con 0x7fbbd41024b0 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0003e20 con 0x7fbbd41024b0 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.452+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd419f380 con 0x7fbbd41024b0 2024-02-14T15:01:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.453+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd419f860 con 0x7fbbd41024b0 2024-02-14T15:01:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.454+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd4108580 con 0x7fbbd41024b0 2024-02-14T15:01:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.455+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbbc00035f0 con 0x7fbbd41024b0 2024-02-14T15:01:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.455+0000 7fbbd1ffb640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbc40a20c0 0x7fbbc40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.455+0000 7fbbd8897640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbc40a20c0 0x7fbbc40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.455+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbbc00f03d0 con 0x7fbbd41024b0 2024-02-14T15:01:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.456+0000 7fbbd8897640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbc40a20c0 0x7fbbc40a4580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80058c0 tx=0x7fbbc8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:17.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.460+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbc00b91b0 con 0x7fbbd41024b0 2024-02-14T15:01:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.596+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fbbc00b9580 con 0x7fbbd41024b0 2024-02-14T15:01:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.687+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbd402cdb0 con 0x7fbbd41024b0 2024-02-14T15:01:17.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.688+0000 7fbbd1ffb640 1 -- 172.21.15.17:0/1213254447 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11317 (secure 0 0 0) 0x7fbbc00a8130 con 0x7fbbd41024b0 2024-02-14T15:01:17.689 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:17.689 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:01:15.551062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:01:17.689 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbc40a20c0 msgr2=0x7fbbc40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbc40a20c0 0x7fbbc40a4580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80058c0 tx=0x7fbbc8005850 comp rx=0 tx=0).stop 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 msgr2=0x7fbbd4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4197820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbbc002c840 tx=0x7fbbc0004270 comp rx=0 tx=0).stop 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 shutdown_connections 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbbc40a20c0 0x7fbbc40a4580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbd41024b0 0x7fbbd4197820 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbbd4102e70 0x7fbbd419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 --2- 172.21.15.17:0/1213254447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbd41012c0 0x7fbbd41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 >> 172.21.15.17:0/1213254447 conn(0x7fbbd40fc850 msgr2=0x7fbbd4105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 shutdown_connections 2024-02-14T15:01:17.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.690+0000 7fbbdaafb640 1 -- 172.21.15.17:0/1213254447 wait complete. 2024-02-14T15:01:17.700 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:17 smithi017 ceph-mon[32549]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:17.701 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:17 smithi017 ceph-mon[32549]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:17.702 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:01:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.803+0000 7fadfa90c640 1 -- 172.21.15.17:0/2544822902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101350 msgr2=0x7fadf41017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.803+0000 7fadfa90c640 1 --2- 172.21.15.17:0/2544822902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101350 0x7fadf41017d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fade8009900 tx=0x7fade802bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- 172.21.15.17:0/2544822902 shutdown_connections 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 --2- 172.21.15.17:0/2544822902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101350 0x7fadf41017d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 --2- 172.21.15.17:0/2544822902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadf4101d10 0x7fadf41085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 --2- 172.21.15.17:0/2544822902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf4100160 0x7fadf4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- 172.21.15.17:0/2544822902 >> 172.21.15.17:0/2544822902 conn(0x7fadf40fb6f0 msgr2=0x7fadf40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- 172.21.15.17:0/2544822902 shutdown_connections 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- 172.21.15.17:0/2544822902 wait complete. 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 Processor -- start 2024-02-14T15:01:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- start start 2024-02-14T15:01:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadf4100160 0x7fadf406a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf4101350 0x7fadf419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101d10 0x7fadf419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf419ee10 con 0x7fadf4101d10 2024-02-14T15:01:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf419ef80 con 0x7fadf4100160 2024-02-14T15:01:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.804+0000 7fadfa90c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf419f140 con 0x7fadf4101350 2024-02-14T15:01:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101d10 0x7fadf419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101d10 0x7fadf419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55352/0 (socket says 172.21.15.17:55352) 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 -- 172.21.15.17:0/3161930851 learned_addr learned my addr 172.21.15.17:0/3161930851 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf3fff640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadf4100160 0x7fadf406a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf37fe640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf4101350 0x7fadf419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 -- 172.21.15.17:0/3161930851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf4101350 msgr2=0x7fadf419c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf4101350 0x7fadf419c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 -- 172.21.15.17:0/3161930851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadf4100160 msgr2=0x7fadf406a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadf4100160 0x7fadf406a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 -- 172.21.15.17:0/3161930851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade8009560 con 0x7fadf4101d10 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf8ea9640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101d10 0x7fadf419e680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fadec009fd0 tx=0x7fadec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf17fa640 1 -- 172.21.15.17:0/3161930851 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec00bea0 con 0x7fadf4101d10 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf419f420 con 0x7fadf4101d10 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf17fa640 1 -- 172.21.15.17:0/3161930851 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadec009940 con 0x7fadf4101d10 2024-02-14T15:01:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadf17fa640 1 -- 172.21.15.17:0/3161930851 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec0106c0 con 0x7fadf4101d10 2024-02-14T15:01:17.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.805+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf419f8b0 con 0x7fadf4101d10 2024-02-14T15:01:17.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.806+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf4107480 con 0x7fadf4101d10 2024-02-14T15:01:17.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.807+0000 7fadf17fa640 1 -- 172.21.15.17:0/3161930851 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadec004240 con 0x7fadf4101d10 2024-02-14T15:01:17.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.808+0000 7fadf17fa640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faddc0a2150 0x7faddc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:17.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.808+0000 7fadf3fff640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faddc0a2150 0x7faddc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:17.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.808+0000 7fadf17fa640 1 -- 172.21.15.17:0/3161930851 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fadec0cdca0 con 0x7fadf4101d10 2024-02-14T15:01:17.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.808+0000 7fadf3fff640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faddc0a2150 0x7faddc0a4610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fadf41011b0 tx=0x7fade4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:17.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.812+0000 7fadf17fa640 1 -- 172.21.15.17:0/3161930851 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadec0969d0 con 0x7fadf4101d10 2024-02-14T15:01:17.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:17 smithi050 ceph-mon[36512]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:17.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:17 smithi050 ceph-mon[36512]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:17 smithi076 ceph-mon[37057]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:17 smithi076 ceph-mon[37057]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:17.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.996+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadf410bb70 con 0x7fadf4101d10 2024-02-14T15:01:17.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.997+0000 7fadf17fa640 1 -- 172.21.15.17:0/3161930851 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7fadec09a090 con 0x7fadf4101d10 2024-02-14T15:01:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:17.998 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:01:18.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.999+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faddc0a2150 msgr2=0x7faddc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.999+0000 7fadfa90c640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faddc0a2150 0x7faddc0a4610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fadf41011b0 tx=0x7fade4005f90 comp rx=0 tx=0).stop 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.999+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101d10 msgr2=0x7fadf419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:17.999+0000 7fadfa90c640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101d10 0x7fadf419e680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fadec009fd0 tx=0x7fadec00b260 comp rx=0 tx=0).stop 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 shutdown_connections 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faddc0a2150 0x7faddc0a4610 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadf4101d10 0x7fadf419e680 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf4101350 0x7fadf419c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 --2- 172.21.15.17:0/3161930851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadf4100160 0x7fadf406a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 >> 172.21.15.17:0/3161930851 conn(0x7fadf40fb6f0 msgr2=0x7fadf4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 shutdown_connections 2024-02-14T15:01:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.000+0000 7fadfa90c640 1 -- 172.21.15.17:0/3161930851 wait complete. 2024-02-14T15:01:18.012 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-14T15:01:18.012 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 -- 172.21.15.17:0/3064760294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 msgr2=0x7fa2781040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 --2- 172.21.15.17:0/3064760294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781040d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa268009900 tx=0x7fa26802bc30 comp rx=0 tx=0).stop 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 -- 172.21.15.17:0/3064760294 shutdown_connections 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 --2- 172.21.15.17:0/3064760294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa278106f40 0x7fa278109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 --2- 172.21.15.17:0/3064760294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa278104610 0x7fa278106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 --2- 172.21.15.17:0/3064760294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781040d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 -- 172.21.15.17:0/3064760294 >> 172.21.15.17:0/3064760294 conn(0x7fa2780fb6f0 msgr2=0x7fa2780fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 -- 172.21.15.17:0/3064760294 shutdown_connections 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 -- 172.21.15.17:0/3064760294 wait complete. 2024-02-14T15:01:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.114+0000 7fa27db25640 1 Processor -- start 2024-02-14T15:01:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa27db25640 1 -- start start 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa27db25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa27db25640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa278104610 0x7fa278197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa27db25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa278106f40 0x7fa27819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa27db25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819edb0 con 0x7fa278101ce0 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa27db25640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819ef20 con 0x7fa278104610 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa27db25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819f0e0 con 0x7fa278106f40 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55368/0 (socket says 172.21.15.17:55368) 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 -- 172.21.15.17:0/1760808276 learned_addr learned my addr 172.21.15.17:0/1760808276 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa276ffd640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa278104610 0x7fa278197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa277fff640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa278106f40 0x7fa27819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 -- 172.21.15.17:0/1760808276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa278106f40 msgr2=0x7fa27819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa278106f40 0x7fa27819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 -- 172.21.15.17:0/1760808276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa278104610 msgr2=0x7fa278197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa278104610 0x7fa278197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa2777fe640 1 -- 172.21.15.17:0/1760808276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268009560 con 0x7fa278101ce0 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.115+0000 7fa277fff640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa278106f40 0x7fa27819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.116+0000 7fa2777fe640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781972c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa268006010 tx=0x7fa2680040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.116+0000 7fa274ff9640 1 -- 172.21.15.17:0/1760808276 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2680044c0 con 0x7fa278101ce0 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.116+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27819f360 con 0x7fa278101ce0 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.116+0000 7fa274ff9640 1 -- 172.21.15.17:0/1760808276 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2680052d0 con 0x7fa278101ce0 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.116+0000 7fa274ff9640 1 -- 172.21.15.17:0/1760808276 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268003e10 con 0x7fa278101ce0 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.116+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27819f870 con 0x7fa278101ce0 2024-02-14T15:01:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.117+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2780fd1e0 con 0x7fa278101ce0 2024-02-14T15:01:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.118+0000 7fa274ff9640 1 -- 172.21.15.17:0/1760808276 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa268005440 con 0x7fa278101ce0 2024-02-14T15:01:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.118+0000 7fa274ff9640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa2600a2150 0x7fa2600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.118+0000 7fa274ff9640 1 -- 172.21.15.17:0/1760808276 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa2680f08d0 con 0x7fa278101ce0 2024-02-14T15:01:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.118+0000 7fa276ffd640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa2600a2150 0x7fa2600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.119+0000 7fa276ffd640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa2600a2150 0x7fa2600a4610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2781987d0 tx=0x7fa26c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:18.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.122+0000 7fa274ff9640 1 -- 172.21.15.17:0/1760808276 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2680b9600 con 0x7fa278101ce0 2024-02-14T15:01:18.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.302+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa27802ce50 con 0x7fa2600a2150 2024-02-14T15:01:18.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.315+0000 7fa274ff9640 1 -- 172.21.15.17:0/1760808276 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423360 (secure 0 0 0) 0x7fa27802ce50 con 0x7fa2600a2150 2024-02-14T15:01:18.323 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:18.328 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-02-14T15:01:18.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa2600a2150 msgr2=0x7fa2600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa2600a2150 0x7fa2600a4610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2781987d0 tx=0x7fa26c005f90 comp rx=0 tx=0).stop 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 msgr2=0x7fa2781972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781972c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa268006010 tx=0x7fa2680040c0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 shutdown_connections 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa2600a2150 0x7fa2600a4610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa278106f40 0x7fa27819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa278104610 0x7fa278197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 --2- 172.21.15.17:0/1760808276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa278101ce0 0x7fa2781972c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 >> 172.21.15.17:0/1760808276 conn(0x7fa2780fb6f0 msgr2=0x7fa278102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 shutdown_connections 2024-02-14T15:01:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.319+0000 7fa27db25640 1 -- 172.21.15.17:0/1760808276 wait complete. 2024-02-14T15:01:18.332 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-02-14T15:01:16.659573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597238,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":141,"num_read_kb":124,"num_write":289,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387504,"kb_used_data":13380,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554320,"statfs":{"total":1151940427776,"available":1151543623680,"internally_reserved":0,"allocated":13701120,"data_stored":7508130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2371,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":61,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003238"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670461+0000","last_change":"2024-02-14T15:00:49.281057+0000","last_active":"2024-02-14T15:00:54.670461+0000","last_peered":"2024-02-14T15:00:54.670461+0000","last_clean":"2024-02-14T15:00:54.670461+0000","last_became_active":"2024-02-14T15:00:49.280642+0000","last_became_peered":"2024-02-14T15:00:49.280642+0000","last_unstale":"2024-02-14T15:00:54.670461+0000","last_undegraded":"2024-02-14T15:00:54.670461+0000","last_fullsized":"2024-02-14T15:00:54.670461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:03.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569726+0000","last_change":"2024-02-14T15:00:49.282794+0000","last_active":"2024-02-14T15:00:54.569726+0000","last_peered":"2024-02-14T15:00:54.569726+0000","last_clean":"2024-02-14T15:00:54.569726+0000","last_became_active":"2024-02-14T15:00:49.281441+0000","last_became_peered":"2024-02-14T15:00:49.281441+0000","last_unstale":"2024-02-14T15:00:54.569726+0000","last_undegraded":"2024-02-14T15:00:54.569726+0000","last_fullsized":"2024-02-14T15:00:54.569726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:57:36.577281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569612+0000","last_change":"2024-02-14T15:00:49.286253+0000","last_active":"2024-02-14T15:00:54.569612+0000","last_peered":"2024-02-14T15:00:54.569612+0000","last_clean":"2024-02-14T15:00:54.569612+0000","last_became_active":"2024-02-14T15:00:49.285950+0000","last_became_peered":"2024-02-14T15:00:49.285950+0000","last_unstale":"2024-02-14T15:00:54.569612+0000","last_undegraded":"2024-02-14T15:00:54.569612+0000","last_fullsized":"2024-02-14T15:00:54.569612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:53.395339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670936+0000","last_change":"2024-02-14T15:00:49.272764+0000","last_active":"2024-02-14T15:00:54.670936+0000","last_peered":"2024-02-14T15:00:54.670936+0000","last_clean":"2024-02-14T15:00:54.670936+0000","last_became_active":"2024-02-14T15:00:49.272430+0000","last_became_peered":"2024-02-14T15:00:49.272430+0000","last_unstale":"2024-02-14T15:00:54.670936+0000","last_undegraded":"2024-02-14T15:00:54.670936+0000","last_fullsized":"2024-02-14T15:00:54.670936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:36.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669582+0000","last_change":"2024-02-14T15:00:49.275028+0000","last_active":"2024-02-14T15:00:54.669582+0000","last_peered":"2024-02-14T15:00:54.669582+0000","last_clean":"2024-02-14T15:00:54.669582+0000","last_became_active":"2024-02-14T15:00:49.274686+0000","last_became_peered":"2024-02-14T15:00:49.274686+0000","last_unstale":"2024-02-14T15:00:54.669582+0000","last_undegraded":"2024-02-14T15:00:54.669582+0000","last_fullsized":"2024-02-14T15:00:54.669582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:21:23.465321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669806+0000","last_change":"2024-02-14T15:00:49.287144+0000","last_active":"2024-02-14T15:00:54.669806+0000","last_peered":"2024-02-14T15:00:54.669806+0000","last_clean":"2024-02-14T15:00:54.669806+0000","last_became_active":"2024-02-14T15:00:49.286767+0000","last_became_peered":"2024-02-14T15:00:49.286767+0000","last_unstale":"2024-02-14T15:00:54.669806+0000","last_undegraded":"2024-02-14T15:00:54.669806+0000","last_fullsized":"2024-02-14T15:00:54.669806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:45:09.391595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566171+0000","last_change":"2024-02-14T15:00:49.274442+0000","last_active":"2024-02-14T15:00:54.566171+0000","last_peered":"2024-02-14T15:00:54.566171+0000","last_clean":"2024-02-14T15:00:54.566171+0000","last_became_active":"2024-02-14T15:00:49.274122+0000","last_became_peered":"2024-02-14T15:00:49.274122+0000","last_unstale":"2024-02-14T15:00:54.566171+0000","last_undegraded":"2024-02-14T15:00:54.566171+0000","last_fullsized":"2024-02-14T15:00:54.566171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:55.974889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566077+0000","last_change":"2024-02-14T15:00:49.285432+0000","last_active":"2024-02-14T15:00:54.566077+0000","last_peered":"2024-02-14T15:00:54.566077+0000","last_clean":"2024-02-14T15:00:54.566077+0000","last_became_active":"2024-02-14T15:00:49.283703+0000","last_became_peered":"2024-02-14T15:00:49.283703+0000","last_unstale":"2024-02-14T15:00:54.566077+0000","last_undegraded":"2024-02-14T15:00:54.566077+0000","last_fullsized":"2024-02-14T15:00:54.566077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:14.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670013+0000","last_change":"2024-02-14T15:00:49.283908+0000","last_active":"2024-02-14T15:00:54.670013+0000","last_peered":"2024-02-14T15:00:54.670013+0000","last_clean":"2024-02-14T15:00:54.670013+0000","last_became_active":"2024-02-14T15:00:49.283131+0000","last_became_peered":"2024-02-14T15:00:49.283131+0000","last_unstale":"2024-02-14T15:00:54.670013+0000","last_undegraded":"2024-02-14T15:00:54.670013+0000","last_fullsized":"2024-02-14T15:00:54.670013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:42.247624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670908+0000","last_change":"2024-02-14T15:00:49.287229+0000","last_active":"2024-02-14T15:00:54.670908+0000","last_peered":"2024-02-14T15:00:54.670908+0000","last_clean":"2024-02-14T15:00:54.670908+0000","last_became_active":"2024-02-14T15:00:49.286791+0000","last_became_peered":"2024-02-14T15:00:49.286791+0000","last_unstale":"2024-02-14T15:00:54.670908+0000","last_undegraded":"2024-02-14T15:00:54.670908+0000","last_fullsized":"2024-02-14T15:00:54.670908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:33:01.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566653+0000","last_change":"2024-02-14T15:00:49.281669+0000","last_active":"2024-02-14T15:00:54.566653+0000","last_peered":"2024-02-14T15:00:54.566653+0000","last_clean":"2024-02-14T15:00:54.566653+0000","last_became_active":"2024-02-14T15:00:49.281298+0000","last_became_peered":"2024-02-14T15:00:49.281298+0000","last_unstale":"2024-02-14T15:00:54.566653+0000","last_undegraded":"2024-02-14T15:00:54.566653+0000","last_fullsized":"2024-02-14T15:00:54.566653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:54:40.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670845+0000","last_change":"2024-02-14T15:00:49.289679+0000","last_active":"2024-02-14T15:00:54.670845+0000","last_peered":"2024-02-14T15:00:54.670845+0000","last_clean":"2024-02-14T15:00:54.670845+0000","last_became_active":"2024-02-14T15:00:49.288767+0000","last_became_peered":"2024-02-14T15:00:49.288767+0000","last_unstale":"2024-02-14T15:00:54.670845+0000","last_undegraded":"2024-02-14T15:00:54.670845+0000","last_fullsized":"2024-02-14T15:00:54.670845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:16:20.181517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669768+0000","last_change":"2024-02-14T15:00:49.257471+0000","last_active":"2024-02-14T15:00:54.669768+0000","last_peered":"2024-02-14T15:00:54.669768+0000","last_clean":"2024-02-14T15:00:54.669768+0000","last_became_active":"2024-02-14T15:00:49.256766+0000","last_became_peered":"2024-02-14T15:00:49.256766+0000","last_unstale":"2024-02-14T15:00:54.669768+0000","last_undegraded":"2024-02-14T15:00:54.669768+0000","last_fullsized":"2024-02-14T15:00:54.669768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:14:00.525621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569150+0000","last_change":"2024-02-14T15:00:49.278462+0000","last_active":"2024-02-14T15:00:54.569150+0000","last_peered":"2024-02-14T15:00:54.569150+0000","last_clean":"2024-02-14T15:00:54.569150+0000","last_became_active":"2024-02-14T15:00:49.278114+0000","last_became_peered":"2024-02-14T15:00:49.278114+0000","last_unstale":"2024-02-14T15:00:54.569150+0000","last_undegraded":"2024-02-14T15:00:54.569150+0000","last_fullsized":"2024-02-14T15:00:54.569150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:02:16.601359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569089+0000","last_change":"2024-02-14T15:00:49.270653+0000","last_active":"2024-02-14T15:00:54.569089+0000","last_peered":"2024-02-14T15:00:54.569089+0000","last_clean":"2024-02-14T15:00:54.569089+0000","last_became_active":"2024-02-14T15:00:49.270334+0000","last_became_peered":"2024-02-14T15:00:49.270334+0000","last_unstale":"2024-02-14T15:00:54.569089+0000","last_undegraded":"2024-02-14T15:00:54.569089+0000","last_fullsized":"2024-02-14T15:00:54.569089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:13:53.840909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.668977+0000","last_change":"2024-02-14T15:00:49.280597+0000","last_active":"2024-02-14T15:00:54.668977+0000","last_peered":"2024-02-14T15:00:54.668977+0000","last_clean":"2024-02-14T15:00:54.668977+0000","last_became_active":"2024-02-14T15:00:49.279612+0000","last_became_peered":"2024-02-14T15:00:49.279612+0000","last_unstale":"2024-02-14T15:00:54.668977+0000","last_undegraded":"2024-02-14T15:00:54.668977+0000","last_fullsized":"2024-02-14T15:00:54.668977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:11.676608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670351+0000","last_change":"2024-02-14T15:00:49.281422+0000","last_active":"2024-02-14T15:00:54.670351+0000","last_peered":"2024-02-14T15:00:54.670351+0000","last_clean":"2024-02-14T15:00:54.670351+0000","last_became_active":"2024-02-14T15:00:49.281048+0000","last_became_peered":"2024-02-14T15:00:49.281048+0000","last_unstale":"2024-02-14T15:00:54.670351+0000","last_undegraded":"2024-02-14T15:00:54.670351+0000","last_fullsized":"2024-02-14T15:00:54.670351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:04.994390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669975+0000","last_change":"2024-02-14T15:00:49.288245+0000","last_active":"2024-02-14T15:00:54.669975+0000","last_peered":"2024-02-14T15:00:54.669975+0000","last_clean":"2024-02-14T15:00:54.669975+0000","last_became_active":"2024-02-14T15:00:49.287908+0000","last_became_peered":"2024-02-14T15:00:49.287908+0000","last_unstale":"2024-02-14T15:00:54.669975+0000","last_undegraded":"2024-02-14T15:00:54.669975+0000","last_fullsized":"2024-02-14T15:00:54.669975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:52.397826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567037+0000","last_change":"2024-02-14T15:00:49.275428+0000","last_active":"2024-02-14T15:00:54.567037+0000","last_peered":"2024-02-14T15:00:54.567037+0000","last_clean":"2024-02-14T15:00:54.567037+0000","last_became_active":"2024-02-14T15:00:49.273464+0000","last_became_peered":"2024-02-14T15:00:49.273464+0000","last_unstale":"2024-02-14T15:00:54.567037+0000","last_undegraded":"2024-02-14T15:00:54.567037+0000","last_fullsized":"2024-02-14T15:00:54.567037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:34.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566596+0000","last_change":"2024-02-14T15:00:53.587219+0000","last_active":"2024-02-14T15:00:54.566596+0000","last_peered":"2024-02-14T15:00:54.566596+0000","last_clean":"2024-02-14T15:00:54.566596+0000","last_became_active":"2024-02-14T15:00:53.586917+0000","last_became_peered":"2024-02-14T15:00:53.586917+0000","last_unstale":"2024-02-14T15:00:54.566596+0000","last_undegraded":"2024-02-14T15:00:54.566596+0000","last_fullsized":"2024-02-14T15:00:54.566596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:39:38.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669893+0000","last_change":"2024-02-14T15:00:48.269243+0000","last_active":"2024-02-14T15:00:54.669893+0000","last_peered":"2024-02-14T15:00:54.669893+0000","last_clean":"2024-02-14T15:00:54.669893+0000","last_became_active":"2024-02-14T15:00:48.268883+0000","last_became_peered":"2024-02-14T15:00:48.268883+0000","last_unstale":"2024-02-14T15:00:54.669893+0000","last_undegraded":"2024-02-14T15:00:54.669893+0000","last_fullsized":"2024-02-14T15:00:54.669893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:08.530092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567087+0000","last_change":"2024-02-14T15:00:49.282458+0000","last_active":"2024-02-14T15:00:54.567087+0000","last_peered":"2024-02-14T15:00:54.567087+0000","last_clean":"2024-02-14T15:00:54.567087+0000","last_became_active":"2024-02-14T15:00:49.281982+0000","last_became_peered":"2024-02-14T15:00:49.281982+0000","last_unstale":"2024-02-14T15:00:54.567087+0000","last_undegraded":"2024-02-14T15:00:54.567087+0000","last_fullsized":"2024-02-14T15:00:54.567087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:51:33.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669555+0000","last_change":"2024-02-14T15:00:53.586578+0000","last_active":"2024-02-14T15:00:54.669555+0000","last_peered":"2024-02-14T15:00:54.669555+0000","last_clean":"2024-02-14T15:00:54.669555+0000","last_became_active":"2024-02-14T15:00:53.586109+0000","last_became_peered":"2024-02-14T15:00:53.586109+0000","last_unstale":"2024-02-14T15:00:54.669555+0000","last_undegraded":"2024-02-14T15:00:54.669555+0000","last_fullsized":"2024-02-14T15:00:54.669555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:07.332435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567928+0000","last_change":"2024-02-14T15:00:48.281953+0000","last_active":"2024-02-14T15:00:54.567928+0000","last_peered":"2024-02-14T15:00:54.567928+0000","last_clean":"2024-02-14T15:00:54.567928+0000","last_became_active":"2024-02-14T15:00:48.281590+0000","last_became_peered":"2024-02-14T15:00:48.281590+0000","last_unstale":"2024-02-14T15:00:54.567928+0000","last_undegraded":"2024-02-14T15:00:54.567928+0000","last_fullsized":"2024-02-14T15:00:54.567928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:22.991037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569845+0000","last_change":"2024-02-14T15:00:49.284729+0000","last_active":"2024-02-14T15:00:54.569845+0000","last_peered":"2024-02-14T15:00:54.569845+0000","last_clean":"2024-02-14T15:00:54.569845+0000","last_became_active":"2024-02-14T15:00:49.284013+0000","last_became_peered":"2024-02-14T15:00:49.284013+0000","last_unstale":"2024-02-14T15:00:54.569845+0000","last_undegraded":"2024-02-14T15:00:54.569845+0000","last_fullsized":"2024-02-14T15:00:54.569845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:03:31.313032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669267+0000","last_change":"2024-02-14T15:00:53.589825+0000","last_active":"2024-02-14T15:00:54.669267+0000","last_peered":"2024-02-14T15:00:54.669267+0000","last_clean":"2024-02-14T15:00:54.669267+0000","last_became_active":"2024-02-14T15:00:53.589424+0000","last_became_peered":"2024-02-14T15:00:53.589424+0000","last_unstale":"2024-02-14T15:00:54.669267+0000","last_undegraded":"2024-02-14T15:00:54.669267+0000","last_fullsized":"2024-02-14T15:00:54.669267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:24.522152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.588182+0000","last_change":"2024-02-14T15:00:48.276884+0000","last_active":"2024-02-14T15:01:14.588182+0000","last_peered":"2024-02-14T15:01:14.588182+0000","last_clean":"2024-02-14T15:01:14.588182+0000","last_became_active":"2024-02-14T15:00:48.276576+0000","last_became_peered":"2024-02-14T15:00:48.276576+0000","last_unstale":"2024-02-14T15:01:14.588182+0000","last_undegraded":"2024-02-14T15:01:14.588182+0000","last_fullsized":"2024-02-14T15:01:14.588182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:25.793368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670022+0000","last_change":"2024-02-14T15:00:49.277602+0000","last_active":"2024-02-14T15:00:54.670022+0000","last_peered":"2024-02-14T15:00:54.670022+0000","last_clean":"2024-02-14T15:00:54.670022+0000","last_became_active":"2024-02-14T15:00:49.277249+0000","last_became_peered":"2024-02-14T15:00:49.277249+0000","last_unstale":"2024-02-14T15:00:54.670022+0000","last_undegraded":"2024-02-14T15:00:54.670022+0000","last_fullsized":"2024-02-14T15:00:54.670022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.347962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670918+0000","last_change":"2024-02-14T15:00:53.587838+0000","last_active":"2024-02-14T15:00:54.670918+0000","last_peered":"2024-02-14T15:00:54.670918+0000","last_clean":"2024-02-14T15:00:54.670918+0000","last_became_active":"2024-02-14T15:00:53.587012+0000","last_became_peered":"2024-02-14T15:00:53.587012+0000","last_unstale":"2024-02-14T15:00:54.670918+0000","last_undegraded":"2024-02-14T15:00:54.670918+0000","last_fullsized":"2024-02-14T15:00:54.670918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:32:12.276578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670317+0000","last_change":"2024-02-14T15:00:48.273529+0000","last_active":"2024-02-14T15:00:54.670317+0000","last_peered":"2024-02-14T15:00:54.670317+0000","last_clean":"2024-02-14T15:00:54.670317+0000","last_became_active":"2024-02-14T15:00:48.272555+0000","last_became_peered":"2024-02-14T15:00:48.272555+0000","last_unstale":"2024-02-14T15:00:54.670317+0000","last_undegraded":"2024-02-14T15:00:54.670317+0000","last_fullsized":"2024-02-14T15:00:54.670317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:19.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670995+0000","last_change":"2024-02-14T15:00:49.278834+0000","last_active":"2024-02-14T15:00:54.670995+0000","last_peered":"2024-02-14T15:00:54.670995+0000","last_clean":"2024-02-14T15:00:54.670995+0000","last_became_active":"2024-02-14T15:00:49.277095+0000","last_became_peered":"2024-02-14T15:00:49.277095+0000","last_unstale":"2024-02-14T15:00:54.670995+0000","last_undegraded":"2024-02-14T15:00:54.670995+0000","last_fullsized":"2024-02-14T15:00:54.670995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:27.291974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566954+0000","last_change":"2024-02-14T15:00:53.577162+0000","last_active":"2024-02-14T15:00:54.566954+0000","last_peered":"2024-02-14T15:00:54.566954+0000","last_clean":"2024-02-14T15:00:54.566954+0000","last_became_active":"2024-02-14T15:00:53.576809+0000","last_became_peered":"2024-02-14T15:00:53.576809+0000","last_unstale":"2024-02-14T15:00:54.566954+0000","last_undegraded":"2024-02-14T15:00:54.566954+0000","last_fullsized":"2024-02-14T15:00:54.566954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:37.698842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.584429+0000","last_change":"2024-02-14T15:00:48.276146+0000","last_active":"2024-02-14T15:01:13.584429+0000","last_peered":"2024-02-14T15:01:13.584429+0000","last_clean":"2024-02-14T15:01:13.584429+0000","last_became_active":"2024-02-14T15:00:48.275833+0000","last_became_peered":"2024-02-14T15:00:48.275833+0000","last_unstale":"2024-02-14T15:01:13.584429+0000","last_undegraded":"2024-02-14T15:01:13.584429+0000","last_fullsized":"2024-02-14T15:01:13.584429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:46.649486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669866+0000","last_change":"2024-02-14T15:00:49.267285+0000","last_active":"2024-02-14T15:00:54.669866+0000","last_peered":"2024-02-14T15:00:54.669866+0000","last_clean":"2024-02-14T15:00:54.669866+0000","last_became_active":"2024-02-14T15:00:49.265592+0000","last_became_peered":"2024-02-14T15:00:49.265592+0000","last_unstale":"2024-02-14T15:00:54.669866+0000","last_undegraded":"2024-02-14T15:00:54.669866+0000","last_fullsized":"2024-02-14T15:00:54.669866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:18.469835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566800+0000","last_change":"2024-02-14T15:00:53.582085+0000","last_active":"2024-02-14T15:00:54.566800+0000","last_peered":"2024-02-14T15:00:54.566800+0000","last_clean":"2024-02-14T15:00:54.566800+0000","last_became_active":"2024-02-14T15:00:53.581706+0000","last_became_peered":"2024-02-14T15:00:53.581706+0000","last_unstale":"2024-02-14T15:00:54.566800+0000","last_undegraded":"2024-02-14T15:00:54.566800+0000","last_fullsized":"2024-02-14T15:00:54.566800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:32.481142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669760+0000","last_change":"2024-02-14T15:00:48.268975+0000","last_active":"2024-02-14T15:00:54.669760+0000","last_peered":"2024-02-14T15:00:54.669760+0000","last_clean":"2024-02-14T15:00:54.669760+0000","last_became_active":"2024-02-14T15:00:48.268547+0000","last_became_peered":"2024-02-14T15:00:48.268547+0000","last_unstale":"2024-02-14T15:00:54.669760+0000","last_undegraded":"2024-02-14T15:00:54.669760+0000","last_fullsized":"2024-02-14T15:00:54.669760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:13.937917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670470+0000","last_change":"2024-02-14T15:00:49.276909+0000","last_active":"2024-02-14T15:00:54.670470+0000","last_peered":"2024-02-14T15:00:54.670470+0000","last_clean":"2024-02-14T15:00:54.670470+0000","last_became_active":"2024-02-14T15:00:49.276533+0000","last_became_peered":"2024-02-14T15:00:49.276533+0000","last_unstale":"2024-02-14T15:00:54.670470+0000","last_undegraded":"2024-02-14T15:00:54.670470+0000","last_fullsized":"2024-02-14T15:00:54.670470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:35:31.321455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670402+0000","last_change":"2024-02-14T15:00:53.592223+0000","last_active":"2024-02-14T15:00:54.670402+0000","last_peered":"2024-02-14T15:00:54.670402+0000","last_clean":"2024-02-14T15:00:54.670402+0000","last_became_active":"2024-02-14T15:00:53.591883+0000","last_became_peered":"2024-02-14T15:00:53.591883+0000","last_unstale":"2024-02-14T15:00:54.670402+0000","last_undegraded":"2024-02-14T15:00:54.670402+0000","last_fullsized":"2024-02-14T15:00:54.670402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:10.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.590374+0000","last_change":"2024-02-14T15:00:48.274485+0000","last_active":"2024-02-14T15:01:14.590374+0000","last_peered":"2024-02-14T15:01:14.590374+0000","last_clean":"2024-02-14T15:01:14.590374+0000","last_became_active":"2024-02-14T15:00:48.274160+0000","last_became_peered":"2024-02-14T15:00:48.274160+0000","last_unstale":"2024-02-14T15:01:14.590374+0000","last_undegraded":"2024-02-14T15:01:14.590374+0000","last_fullsized":"2024-02-14T15:01:14.590374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:00:47.105069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566302+0000","last_change":"2024-02-14T15:00:49.272577+0000","last_active":"2024-02-14T15:00:54.566302+0000","last_peered":"2024-02-14T15:00:54.566302+0000","last_clean":"2024-02-14T15:00:54.566302+0000","last_became_active":"2024-02-14T15:00:49.271670+0000","last_became_peered":"2024-02-14T15:00:49.271670+0000","last_unstale":"2024-02-14T15:00:54.566302+0000","last_undegraded":"2024-02-14T15:00:54.566302+0000","last_fullsized":"2024-02-14T15:00:54.566302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:19.968389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569872+0000","last_change":"2024-02-14T15:00:53.581212+0000","last_active":"2024-02-14T15:00:54.569872+0000","last_peered":"2024-02-14T15:00:54.569872+0000","last_clean":"2024-02-14T15:00:54.569872+0000","last_became_active":"2024-02-14T15:00:53.580782+0000","last_became_peered":"2024-02-14T15:00:53.580782+0000","last_unstale":"2024-02-14T15:00:54.569872+0000","last_undegraded":"2024-02-14T15:00:54.569872+0000","last_fullsized":"2024-02-14T15:00:54.569872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:14.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669988+0000","last_change":"2024-02-14T15:00:48.273572+0000","last_active":"2024-02-14T15:00:54.669988+0000","last_peered":"2024-02-14T15:00:54.669988+0000","last_clean":"2024-02-14T15:00:54.669988+0000","last_became_active":"2024-02-14T15:00:48.273138+0000","last_became_peered":"2024-02-14T15:00:48.273138+0000","last_unstale":"2024-02-14T15:00:54.669988+0000","last_undegraded":"2024-02-14T15:00:54.669988+0000","last_fullsized":"2024-02-14T15:00:54.669988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:04.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670295+0000","last_change":"2024-02-14T15:00:49.288226+0000","last_active":"2024-02-14T15:00:54.670295+0000","last_peered":"2024-02-14T15:00:54.670295+0000","last_clean":"2024-02-14T15:00:54.670295+0000","last_became_active":"2024-02-14T15:00:49.287874+0000","last_became_peered":"2024-02-14T15:00:49.287874+0000","last_unstale":"2024-02-14T15:00:54.670295+0000","last_undegraded":"2024-02-14T15:00:54.670295+0000","last_fullsized":"2024-02-14T15:00:54.670295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:24:02.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566490+0000","last_change":"2024-02-14T15:00:53.590543+0000","last_active":"2024-02-14T15:00:54.566490+0000","last_peered":"2024-02-14T15:00:54.566490+0000","last_clean":"2024-02-14T15:00:54.566490+0000","last_became_active":"2024-02-14T15:00:53.589072+0000","last_became_peered":"2024-02-14T15:00:53.589072+0000","last_unstale":"2024-02-14T15:00:54.566490+0000","last_undegraded":"2024-02-14T15:00:54.566490+0000","last_fullsized":"2024-02-14T15:00:54.566490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:13.685655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.582408+0000","last_change":"2024-02-14T15:00:48.272027+0000","last_active":"2024-02-14T15:01:13.582408+0000","last_peered":"2024-02-14T15:01:13.582408+0000","last_clean":"2024-02-14T15:01:13.582408+0000","last_became_active":"2024-02-14T15:00:48.271587+0000","last_became_peered":"2024-02-14T15:00:48.271587+0000","last_unstale":"2024-02-14T15:01:13.582408+0000","last_undegraded":"2024-02-14T15:01:13.582408+0000","last_fullsized":"2024-02-14T15:01:13.582408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:33:03.566189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669727+0000","last_change":"2024-02-14T15:00:49.278936+0000","last_active":"2024-02-14T15:00:54.669727+0000","last_peered":"2024-02-14T15:00:54.669727+0000","last_clean":"2024-02-14T15:00:54.669727+0000","last_became_active":"2024-02-14T15:00:49.278318+0000","last_became_peered":"2024-02-14T15:00:49.278318+0000","last_unstale":"2024-02-14T15:00:54.669727+0000","last_undegraded":"2024-02-14T15:00:54.669727+0000","last_fullsized":"2024-02-14T15:00:54.669727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:01.327303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669815+0000","last_change":"2024-02-14T15:00:53.580340+0000","last_active":"2024-02-14T15:00:54.669815+0000","last_peered":"2024-02-14T15:00:54.669815+0000","last_clean":"2024-02-14T15:00:54.669815+0000","last_became_active":"2024-02-14T15:00:53.579742+0000","last_became_peered":"2024-02-14T15:00:53.579742+0000","last_unstale":"2024-02-14T15:00:54.669815+0000","last_undegraded":"2024-02-14T15:00:54.669815+0000","last_fullsized":"2024-02-14T15:00:54.669815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:38:33.182328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583025+0000","last_change":"2024-02-14T15:00:48.271891+0000","last_active":"2024-02-14T15:01:13.583025+0000","last_peered":"2024-02-14T15:01:13.583025+0000","last_clean":"2024-02-14T15:01:13.583025+0000","last_became_active":"2024-02-14T15:00:48.271419+0000","last_became_peered":"2024-02-14T15:00:48.271419+0000","last_unstale":"2024-02-14T15:01:13.583025+0000","last_undegraded":"2024-02-14T15:01:13.583025+0000","last_fullsized":"2024-02-14T15:01:13.583025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:03:04.725647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569792+0000","last_change":"2024-02-14T15:00:49.277025+0000","last_active":"2024-02-14T15:00:54.569792+0000","last_peered":"2024-02-14T15:00:54.569792+0000","last_clean":"2024-02-14T15:00:54.569792+0000","last_became_active":"2024-02-14T15:00:49.276694+0000","last_became_peered":"2024-02-14T15:00:49.276694+0000","last_unstale":"2024-02-14T15:00:54.569792+0000","last_undegraded":"2024-02-14T15:00:54.569792+0000","last_fullsized":"2024-02-14T15:00:54.569792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:43:07.249407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669532+0000","last_change":"2024-02-14T15:00:53.588208+0000","last_active":"2024-02-14T15:00:54.669532+0000","last_peered":"2024-02-14T15:00:54.669532+0000","last_clean":"2024-02-14T15:00:54.669532+0000","last_became_active":"2024-02-14T15:00:53.587153+0000","last_became_peered":"2024-02-14T15:00:53.587153+0000","last_unstale":"2024-02-14T15:00:54.669532+0000","last_undegraded":"2024-02-14T15:00:54.669532+0000","last_fullsized":"2024-02-14T15:00:54.669532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:14:02.707991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670939+0000","last_change":"2024-02-14T15:00:48.272866+0000","last_active":"2024-02-14T15:00:54.670939+0000","last_peered":"2024-02-14T15:00:54.670939+0000","last_clean":"2024-02-14T15:00:54.670939+0000","last_became_active":"2024-02-14T15:00:48.272520+0000","last_became_peered":"2024-02-14T15:00:48.272520+0000","last_unstale":"2024-02-14T15:00:54.670939+0000","last_undegraded":"2024-02-14T15:00:54.670939+0000","last_fullsized":"2024-02-14T15:00:54.670939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:39.434577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670182+0000","last_change":"2024-02-14T15:00:49.280994+0000","last_active":"2024-02-14T15:00:54.670182+0000","last_peered":"2024-02-14T15:00:54.670182+0000","last_clean":"2024-02-14T15:00:54.670182+0000","last_became_active":"2024-02-14T15:00:49.280205+0000","last_became_peered":"2024-02-14T15:00:49.280205+0000","last_unstale":"2024-02-14T15:00:54.670182+0000","last_undegraded":"2024-02-14T15:00:54.670182+0000","last_fullsized":"2024-02-14T15:00:54.670182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:53:48.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670214+0000","last_change":"2024-02-14T15:00:53.582599+0000","last_active":"2024-02-14T15:00:54.670214+0000","last_peered":"2024-02-14T15:00:54.670214+0000","last_clean":"2024-02-14T15:00:54.670214+0000","last_became_active":"2024-02-14T15:00:53.582231+0000","last_became_peered":"2024-02-14T15:00:53.582231+0000","last_unstale":"2024-02-14T15:00:54.670214+0000","last_undegraded":"2024-02-14T15:00:54.670214+0000","last_fullsized":"2024-02-14T15:00:54.670214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:19.126264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583342+0000","last_change":"2024-02-14T15:00:48.274178+0000","last_active":"2024-02-14T15:01:13.583342+0000","last_peered":"2024-02-14T15:01:13.583342+0000","last_clean":"2024-02-14T15:01:13.583342+0000","last_became_active":"2024-02-14T15:00:48.273533+0000","last_became_peered":"2024-02-14T15:00:48.273533+0000","last_unstale":"2024-02-14T15:01:13.583342+0000","last_undegraded":"2024-02-14T15:01:13.583342+0000","last_fullsized":"2024-02-14T15:01:13.583342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:33:08.621601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670865+0000","last_change":"2024-02-14T15:00:49.270144+0000","last_active":"2024-02-14T15:00:54.670865+0000","last_peered":"2024-02-14T15:00:54.670865+0000","last_clean":"2024-02-14T15:00:54.670865+0000","last_became_active":"2024-02-14T15:00:49.269806+0000","last_became_peered":"2024-02-14T15:00:49.269806+0000","last_unstale":"2024-02-14T15:00:54.670865+0000","last_undegraded":"2024-02-14T15:00:54.670865+0000","last_fullsized":"2024-02-14T15:00:54.670865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:42:18.559226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.671366+0000","last_change":"2024-02-14T15:00:53.582755+0000","last_active":"2024-02-14T15:00:54.671366+0000","last_peered":"2024-02-14T15:00:54.671366+0000","last_clean":"2024-02-14T15:00:54.671366+0000","last_became_active":"2024-02-14T15:00:53.582375+0000","last_became_peered":"2024-02-14T15:00:53.582375+0000","last_unstale":"2024-02-14T15:00:54.671366+0000","last_undegraded":"2024-02-14T15:00:54.671366+0000","last_fullsized":"2024-02-14T15:00:54.671366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:11:49.115136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583242+0000","last_change":"2024-02-14T15:00:48.268982+0000","last_active":"2024-02-14T15:01:13.583242+0000","last_peered":"2024-02-14T15:01:13.583242+0000","last_clean":"2024-02-14T15:01:13.583242+0000","last_became_active":"2024-02-14T15:00:48.267137+0000","last_became_peered":"2024-02-14T15:00:48.267137+0000","last_unstale":"2024-02-14T15:01:13.583242+0000","last_undegraded":"2024-02-14T15:01:13.583242+0000","last_fullsized":"2024-02-14T15:01:13.583242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:44.351561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670062+0000","last_change":"2024-02-14T15:00:49.267493+0000","last_active":"2024-02-14T15:00:54.670062+0000","last_peered":"2024-02-14T15:00:54.670062+0000","last_clean":"2024-02-14T15:00:54.670062+0000","last_became_active":"2024-02-14T15:00:49.267161+0000","last_became_peered":"2024-02-14T15:00:49.267161+0000","last_unstale":"2024-02-14T15:00:54.670062+0000","last_undegraded":"2024-02-14T15:00:54.670062+0000","last_fullsized":"2024-02-14T15:00:54.670062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:35:22.997999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670040+0000","last_change":"2024-02-14T15:00:53.591201+0000","last_active":"2024-02-14T15:00:54.670040+0000","last_peered":"2024-02-14T15:00:54.670040+0000","last_clean":"2024-02-14T15:00:54.670040+0000","last_became_active":"2024-02-14T15:00:53.590769+0000","last_became_peered":"2024-02-14T15:00:53.590769+0000","last_unstale":"2024-02-14T15:00:54.670040+0000","last_undegraded":"2024-02-14T15:00:54.670040+0000","last_fullsized":"2024-02-14T15:00:54.670040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.881271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583767+0000","last_change":"2024-02-14T15:00:48.282085+0000","last_active":"2024-02-14T15:01:13.583767+0000","last_peered":"2024-02-14T15:01:13.583767+0000","last_clean":"2024-02-14T15:01:13.583767+0000","last_became_active":"2024-02-14T15:00:48.281836+0000","last_became_peered":"2024-02-14T15:00:48.281836+0000","last_unstale":"2024-02-14T15:01:13.583767+0000","last_undegraded":"2024-02-14T15:01:13.583767+0000","last_fullsized":"2024-02-14T15:01:13.583767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:07:28.145074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566527+0000","last_change":"2024-02-14T15:00:49.280013+0000","last_active":"2024-02-14T15:00:54.566527+0000","last_peered":"2024-02-14T15:00:54.566527+0000","last_clean":"2024-02-14T15:00:54.566527+0000","last_became_active":"2024-02-14T15:00:49.279667+0000","last_became_peered":"2024-02-14T15:00:49.279667+0000","last_unstale":"2024-02-14T15:00:54.566527+0000","last_undegraded":"2024-02-14T15:00:54.566527+0000","last_fullsized":"2024-02-14T15:00:54.566527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:03:16.559295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669511+0000","last_change":"2024-02-14T15:00:53.580269+0000","last_active":"2024-02-14T15:00:54.669511+0000","last_peered":"2024-02-14T15:00:54.669511+0000","last_clean":"2024-02-14T15:00:54.669511+0000","last_became_active":"2024-02-14T15:00:53.579863+0000","last_became_peered":"2024-02-14T15:00:53.579863+0000","last_unstale":"2024-02-14T15:00:54.669511+0000","last_undegraded":"2024-02-14T15:00:54.669511+0000","last_fullsized":"2024-02-14T15:00:54.669511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:37:19.275654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.588316+0000","last_change":"2024-02-14T15:00:48.267310+0000","last_active":"2024-02-14T15:01:14.588316+0000","last_peered":"2024-02-14T15:01:14.588316+0000","last_clean":"2024-02-14T15:01:14.588316+0000","last_became_active":"2024-02-14T15:00:48.266864+0000","last_became_peered":"2024-02-14T15:00:48.266864+0000","last_unstale":"2024-02-14T15:01:14.588316+0000","last_undegraded":"2024-02-14T15:01:14.588316+0000","last_fullsized":"2024-02-14T15:01:14.588316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:07.703549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566387+0000","last_change":"2024-02-14T15:00:49.288568+0000","last_active":"2024-02-14T15:00:54.566387+0000","last_peered":"2024-02-14T15:00:54.566387+0000","last_clean":"2024-02-14T15:00:54.566387+0000","last_became_active":"2024-02-14T15:00:49.288222+0000","last_became_peered":"2024-02-14T15:00:49.288222+0000","last_unstale":"2024-02-14T15:00:54.566387+0000","last_undegraded":"2024-02-14T15:00:54.566387+0000","last_fullsized":"2024-02-14T15:00:54.566387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:38:38.909769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669939+0000","last_change":"2024-02-14T15:00:53.585673+0000","last_active":"2024-02-14T15:00:54.669939+0000","last_peered":"2024-02-14T15:00:54.669939+0000","last_clean":"2024-02-14T15:00:54.669939+0000","last_became_active":"2024-02-14T15:00:53.585238+0000","last_became_peered":"2024-02-14T15:00:53.585238+0000","last_unstale":"2024-02-14T15:00:54.669939+0000","last_undegraded":"2024-02-14T15:00:54.669939+0000","last_fullsized":"2024-02-14T15:00:54.669939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:48:11.845921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.590807+0000","last_change":"2024-02-14T15:00:48.275833+0000","last_active":"2024-02-14T15:01:14.590807+0000","last_peered":"2024-02-14T15:01:14.590807+0000","last_clean":"2024-02-14T15:01:14.590807+0000","last_became_active":"2024-02-14T15:00:48.275499+0000","last_became_peered":"2024-02-14T15:00:48.275499+0000","last_unstale":"2024-02-14T15:01:14.590807+0000","last_undegraded":"2024-02-14T15:01:14.590807+0000","last_fullsized":"2024-02-14T15:01:14.590807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:35:10.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670195+0000","last_change":"2024-02-14T15:00:49.276231+0000","last_active":"2024-02-14T15:00:54.670195+0000","last_peered":"2024-02-14T15:00:54.670195+0000","last_clean":"2024-02-14T15:00:54.670195+0000","last_became_active":"2024-02-14T15:00:49.275678+0000","last_became_peered":"2024-02-14T15:00:49.275678+0000","last_unstale":"2024-02-14T15:00:54.670195+0000","last_undegraded":"2024-02-14T15:00:54.670195+0000","last_fullsized":"2024-02-14T15:00:54.670195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:53:46.402577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669981+0000","last_change":"2024-02-14T15:00:53.586406+0000","last_active":"2024-02-14T15:00:54.669981+0000","last_peered":"2024-02-14T15:00:54.669981+0000","last_clean":"2024-02-14T15:00:54.669981+0000","last_became_active":"2024-02-14T15:00:53.586034+0000","last_became_peered":"2024-02-14T15:00:53.586034+0000","last_unstale":"2024-02-14T15:00:54.669981+0000","last_undegraded":"2024-02-14T15:00:54.669981+0000","last_fullsized":"2024-02-14T15:00:54.669981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:46.886154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670762+0000","last_change":"2024-02-14T15:00:48.274305+0000","last_active":"2024-02-14T15:00:54.670762+0000","last_peered":"2024-02-14T15:00:54.670762+0000","last_clean":"2024-02-14T15:00:54.670762+0000","last_became_active":"2024-02-14T15:00:48.273966+0000","last_became_peered":"2024-02-14T15:00:48.273966+0000","last_unstale":"2024-02-14T15:00:54.670762+0000","last_undegraded":"2024-02-14T15:00:54.670762+0000","last_fullsized":"2024-02-14T15:00:54.670762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:49.198099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670637+0000","last_change":"2024-02-14T15:00:49.278473+0000","last_active":"2024-02-14T15:00:54.670637+0000","last_peered":"2024-02-14T15:00:54.670637+0000","last_clean":"2024-02-14T15:00:54.670637+0000","last_became_active":"2024-02-14T15:00:49.278145+0000","last_became_peered":"2024-02-14T15:00:49.278145+0000","last_unstale":"2024-02-14T15:00:54.670637+0000","last_undegraded":"2024-02-14T15:00:54.670637+0000","last_fullsized":"2024-02-14T15:00:54.670637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:49:11.718035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669811+0000","last_change":"2024-02-14T15:00:53.581164+0000","last_active":"2024-02-14T15:00:54.669811+0000","last_peered":"2024-02-14T15:00:54.669811+0000","last_clean":"2024-02-14T15:00:54.669811+0000","last_became_active":"2024-02-14T15:00:53.580796+0000","last_became_peered":"2024-02-14T15:00:53.580796+0000","last_unstale":"2024-02-14T15:00:54.669811+0000","last_undegraded":"2024-02-14T15:00:54.669811+0000","last_fullsized":"2024-02-14T15:00:54.669811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:37.245035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670142+0000","last_change":"2024-02-14T15:00:48.271427+0000","last_active":"2024-02-14T15:00:54.670142+0000","last_peered":"2024-02-14T15:00:54.670142+0000","last_clean":"2024-02-14T15:00:54.670142+0000","last_became_active":"2024-02-14T15:00:48.271115+0000","last_became_peered":"2024-02-14T15:00:48.271115+0000","last_unstale":"2024-02-14T15:00:54.670142+0000","last_undegraded":"2024-02-14T15:00:54.670142+0000","last_fullsized":"2024-02-14T15:00:54.670142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:56:01.008058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670429+0000","last_change":"2024-02-14T15:00:49.269964+0000","last_active":"2024-02-14T15:00:54.670429+0000","last_peered":"2024-02-14T15:00:54.670429+0000","last_clean":"2024-02-14T15:00:54.670429+0000","last_became_active":"2024-02-14T15:00:49.269628+0000","last_became_peered":"2024-02-14T15:00:49.269628+0000","last_unstale":"2024-02-14T15:00:54.670429+0000","last_undegraded":"2024-02-14T15:00:54.670429+0000","last_fullsized":"2024-02-14T15:00:54.670429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:04:26.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670615+0000","last_change":"2024-02-14T15:00:53.579843+0000","last_active":"2024-02-14T15:00:54.670615+0000","last_peered":"2024-02-14T15:00:54.670615+0000","last_clean":"2024-02-14T15:00:54.670615+0000","last_became_active":"2024-02-14T15:00:53.579457+0000","last_became_peered":"2024-02-14T15:00:53.579457+0000","last_unstale":"2024-02-14T15:00:54.670615+0000","last_undegraded":"2024-02-14T15:00:54.670615+0000","last_fullsized":"2024-02-14T15:00:54.670615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:34:49.438317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566622+0000","last_change":"2024-02-14T15:00:48.275016+0000","last_active":"2024-02-14T15:00:54.566622+0000","last_peered":"2024-02-14T15:00:54.566622+0000","last_clean":"2024-02-14T15:00:54.566622+0000","last_became_active":"2024-02-14T15:00:48.274636+0000","last_became_peered":"2024-02-14T15:00:48.274636+0000","last_unstale":"2024-02-14T15:00:54.566622+0000","last_undegraded":"2024-02-14T15:00:54.566622+0000","last_fullsized":"2024-02-14T15:00:54.566622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:12:12.961276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670061+0000","last_change":"2024-02-14T15:00:49.292025+0000","last_active":"2024-02-14T15:00:54.670061+0000","last_peered":"2024-02-14T15:00:54.670061+0000","last_clean":"2024-02-14T15:00:54.670061+0000","last_became_active":"2024-02-14T15:00:49.291130+0000","last_became_peered":"2024-02-14T15:00:49.291130+0000","last_unstale":"2024-02-14T15:00:54.670061+0000","last_undegraded":"2024-02-14T15:00:54.670061+0000","last_fullsized":"2024-02-14T15:00:54.670061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:37.150827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669702+0000","last_change":"2024-02-14T15:00:53.584068+0000","last_active":"2024-02-14T15:00:54.669702+0000","last_peered":"2024-02-14T15:00:54.669702+0000","last_clean":"2024-02-14T15:00:54.669702+0000","last_became_active":"2024-02-14T15:00:53.583263+0000","last_became_peered":"2024-02-14T15:00:53.583263+0000","last_unstale":"2024-02-14T15:00:54.669702+0000","last_undegraded":"2024-02-14T15:00:54.669702+0000","last_fullsized":"2024-02-14T15:00:54.669702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:15.726442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670125+0000","last_change":"2024-02-14T15:00:49.271265+0000","last_active":"2024-02-14T15:00:54.670125+0000","last_peered":"2024-02-14T15:00:54.670125+0000","last_clean":"2024-02-14T15:00:54.670125+0000","last_became_active":"2024-02-14T15:00:49.270931+0000","last_became_peered":"2024-02-14T15:00:49.270931+0000","last_unstale":"2024-02-14T15:00:54.670125+0000","last_undegraded":"2024-02-14T15:00:54.670125+0000","last_fullsized":"2024-02-14T15:00:54.670125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:15:40.570730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.590133+0000","last_change":"2024-02-14T15:00:48.286669+0000","last_active":"2024-02-14T15:01:14.590133+0000","last_peered":"2024-02-14T15:01:14.590133+0000","last_clean":"2024-02-14T15:01:14.590133+0000","last_became_active":"2024-02-14T15:00:48.286252+0000","last_became_peered":"2024-02-14T15:00:48.286252+0000","last_unstale":"2024-02-14T15:01:14.590133+0000","last_undegraded":"2024-02-14T15:01:14.590133+0000","last_fullsized":"2024-02-14T15:01:14.590133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:40.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566567+0000","last_change":"2024-02-14T15:00:53.583353+0000","last_active":"2024-02-14T15:00:54.566567+0000","last_peered":"2024-02-14T15:00:54.566567+0000","last_clean":"2024-02-14T15:00:54.566567+0000","last_became_active":"2024-02-14T15:00:53.582449+0000","last_became_peered":"2024-02-14T15:00:53.582449+0000","last_unstale":"2024-02-14T15:00:54.566567+0000","last_undegraded":"2024-02-14T15:00:54.566567+0000","last_fullsized":"2024-02-14T15:00:54.566567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:09:01.514603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566701+0000","last_change":"2024-02-14T15:00:49.285579+0000","last_active":"2024-02-14T15:00:54.566701+0000","last_peered":"2024-02-14T15:00:54.566701+0000","last_clean":"2024-02-14T15:00:54.566701+0000","last_became_active":"2024-02-14T15:00:49.283741+0000","last_became_peered":"2024-02-14T15:00:49.283741+0000","last_unstale":"2024-02-14T15:00:54.566701+0000","last_undegraded":"2024-02-14T15:00:54.566701+0000","last_fullsized":"2024-02-14T15:00:54.566701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:05.813218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.582185+0000","last_change":"2024-02-14T15:00:48.264434+0000","last_active":"2024-02-14T15:01:13.582185+0000","last_peered":"2024-02-14T15:01:13.582185+0000","last_clean":"2024-02-14T15:01:13.582185+0000","last_became_active":"2024-02-14T15:00:48.264048+0000","last_became_peered":"2024-02-14T15:00:48.264048+0000","last_unstale":"2024-02-14T15:01:13.582185+0000","last_undegraded":"2024-02-14T15:01:13.582185+0000","last_fullsized":"2024-02-14T15:01:13.582185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:52:32.498128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566155+0000","last_change":"2024-02-14T14:59:36.539524+0000","last_active":"2024-02-14T15:00:54.566155+0000","last_peered":"2024-02-14T15:00:54.566155+0000","last_clean":"2024-02-14T15:00:54.566155+0000","last_became_active":"2024-02-14T14:59:36.538922+0000","last_became_peered":"2024-02-14T14:59:36.538922+0000","last_unstale":"2024-02-14T15:00:54.566155+0000","last_undegraded":"2024-02-14T15:00:54.566155+0000","last_fullsized":"2024-02-14T15:00:54.566155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_clean_scrub_stamp":"2024-02-14T14:56:09.567049+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:58.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670878+0000","last_change":"2024-02-14T15:00:53.585747+0000","last_active":"2024-02-14T15:00:54.670878+0000","last_peered":"2024-02-14T15:00:54.670878+0000","last_clean":"2024-02-14T15:00:54.670878+0000","last_became_active":"2024-02-14T15:00:53.583817+0000","last_became_peered":"2024-02-14T15:00:53.583817+0000","last_unstale":"2024-02-14T15:00:54.670878+0000","last_undegraded":"2024-02-14T15:00:54.670878+0000","last_fullsized":"2024-02-14T15:00:54.670878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:56:35.081078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669181+0000","last_change":"2024-02-14T15:00:49.272692+0000","last_active":"2024-02-14T15:00:54.669181+0000","last_peered":"2024-02-14T15:00:54.669181+0000","last_clean":"2024-02-14T15:00:54.669181+0000","last_became_active":"2024-02-14T15:00:49.272353+0000","last_became_peered":"2024-02-14T15:00:49.272353+0000","last_unstale":"2024-02-14T15:00:54.669181+0000","last_undegraded":"2024-02-14T15:00:54.669181+0000","last_fullsized":"2024-02-14T15:00:54.669181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:47.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.588917+0000","last_change":"2024-02-14T15:00:48.272470+0000","last_active":"2024-02-14T15:01:14.588917+0000","last_peered":"2024-02-14T15:01:14.588917+0000","last_clean":"2024-02-14T15:01:14.588917+0000","last_became_active":"2024-02-14T15:00:48.272126+0000","last_became_peered":"2024-02-14T15:00:48.272126+0000","last_unstale":"2024-02-14T15:01:14.588917+0000","last_undegraded":"2024-02-14T15:01:14.588917+0000","last_fullsized":"2024-02-14T15:01:14.588917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:21:02.601411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669473+0000","last_change":"2024-02-14T15:00:53.585950+0000","last_active":"2024-02-14T15:00:54.669473+0000","last_peered":"2024-02-14T15:00:54.669473+0000","last_clean":"2024-02-14T15:00:54.669473+0000","last_became_active":"2024-02-14T15:00:53.585547+0000","last_became_peered":"2024-02-14T15:00:53.585547+0000","last_unstale":"2024-02-14T15:00:54.669473+0000","last_undegraded":"2024-02-14T15:00:54.669473+0000","last_fullsized":"2024-02-14T15:00:54.669473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:22.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583167+0000","last_change":"2024-02-14T15:00:48.274650+0000","last_active":"2024-02-14T15:01:13.583167+0000","last_peered":"2024-02-14T15:01:13.583167+0000","last_clean":"2024-02-14T15:01:13.583167+0000","last_became_active":"2024-02-14T15:00:48.274085+0000","last_became_peered":"2024-02-14T15:00:48.274085+0000","last_unstale":"2024-02-14T15:01:13.583167+0000","last_undegraded":"2024-02-14T15:01:13.583167+0000","last_fullsized":"2024-02-14T15:01:13.583167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:53.307596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670651+0000","last_change":"2024-02-14T15:00:49.268419+0000","last_active":"2024-02-14T15:00:54.670651+0000","last_peered":"2024-02-14T15:00:54.670651+0000","last_clean":"2024-02-14T15:00:54.670651+0000","last_became_active":"2024-02-14T15:00:49.268120+0000","last_became_peered":"2024-02-14T15:00:49.268120+0000","last_unstale":"2024-02-14T15:00:54.670651+0000","last_undegraded":"2024-02-14T15:00:54.670651+0000","last_fullsized":"2024-02-14T15:00:54.670651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:10:36.319337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566309+0000","last_change":"2024-02-14T15:00:53.586151+0000","last_active":"2024-02-14T15:00:54.566309+0000","last_peered":"2024-02-14T15:00:54.566309+0000","last_clean":"2024-02-14T15:00:54.566309+0000","last_became_active":"2024-02-14T15:00:53.585727+0000","last_became_peered":"2024-02-14T15:00:53.585727+0000","last_unstale":"2024-02-14T15:00:54.566309+0000","last_undegraded":"2024-02-14T15:00:54.566309+0000","last_fullsized":"2024-02-14T15:00:54.566309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:40:03.741063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670079+0000","last_change":"2024-02-14T15:00:48.269837+0000","last_active":"2024-02-14T15:00:54.670079+0000","last_peered":"2024-02-14T15:00:54.670079+0000","last_clean":"2024-02-14T15:00:54.670079+0000","last_became_active":"2024-02-14T15:00:48.268699+0000","last_became_peered":"2024-02-14T15:00:48.268699+0000","last_unstale":"2024-02-14T15:00:54.670079+0000","last_undegraded":"2024-02-14T15:00:54.670079+0000","last_fullsized":"2024-02-14T15:00:54.670079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:55:03.013888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566821+0000","last_change":"2024-02-14T15:00:49.285726+0000","last_active":"2024-02-14T15:00:54.566821+0000","last_peered":"2024-02-14T15:00:54.566821+0000","last_clean":"2024-02-14T15:00:54.566821+0000","last_became_active":"2024-02-14T15:00:49.283728+0000","last_became_peered":"2024-02-14T15:00:49.283728+0000","last_unstale":"2024-02-14T15:00:54.566821+0000","last_undegraded":"2024-02-14T15:00:54.566821+0000","last_fullsized":"2024-02-14T15:00:54.566821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.764048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566421+0000","last_change":"2024-02-14T15:00:53.583644+0000","last_active":"2024-02-14T15:00:54.566421+0000","last_peered":"2024-02-14T15:00:54.566421+0000","last_clean":"2024-02-14T15:00:54.566421+0000","last_became_active":"2024-02-14T15:00:53.582801+0000","last_became_peered":"2024-02-14T15:00:53.582801+0000","last_unstale":"2024-02-14T15:00:54.566421+0000","last_undegraded":"2024-02-14T15:00:54.566421+0000","last_fullsized":"2024-02-14T15:00:54.566421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:41:40.622091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.584078+0000","last_change":"2024-02-14T15:00:48.270231+0000","last_active":"2024-02-14T15:01:13.584078+0000","last_peered":"2024-02-14T15:01:13.584078+0000","last_clean":"2024-02-14T15:01:13.584078+0000","last_became_active":"2024-02-14T15:00:48.269880+0000","last_became_peered":"2024-02-14T15:00:48.269880+0000","last_unstale":"2024-02-14T15:01:13.584078+0000","last_undegraded":"2024-02-14T15:01:13.584078+0000","last_fullsized":"2024-02-14T15:01:13.584078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:08:47.376417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569865+0000","last_change":"2024-02-14T15:00:49.277227+0000","last_active":"2024-02-14T15:00:54.569865+0000","last_peered":"2024-02-14T15:00:54.569865+0000","last_clean":"2024-02-14T15:00:54.569865+0000","last_became_active":"2024-02-14T15:00:49.276904+0000","last_became_peered":"2024-02-14T15:00:49.276904+0000","last_unstale":"2024-02-14T15:00:54.569865+0000","last_undegraded":"2024-02-14T15:00:54.569865+0000","last_fullsized":"2024-02-14T15:00:54.569865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:01.091327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669924+0000","last_change":"2024-02-14T15:00:53.587855+0000","last_active":"2024-02-14T15:00:54.669924+0000","last_peered":"2024-02-14T15:00:54.669924+0000","last_clean":"2024-02-14T15:00:54.669924+0000","last_became_active":"2024-02-14T15:00:53.586313+0000","last_became_peered":"2024-02-14T15:00:53.586313+0000","last_unstale":"2024-02-14T15:00:54.669924+0000","last_undegraded":"2024-02-14T15:00:54.669924+0000","last_fullsized":"2024-02-14T15:00:54.669924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:36:41.279582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583364+0000","last_change":"2024-02-14T15:00:48.267385+0000","last_active":"2024-02-14T15:01:13.583364+0000","last_peered":"2024-02-14T15:01:13.583364+0000","last_clean":"2024-02-14T15:01:13.583364+0000","last_became_active":"2024-02-14T15:00:48.267036+0000","last_became_peered":"2024-02-14T15:00:48.267036+0000","last_unstale":"2024-02-14T15:01:13.583364+0000","last_undegraded":"2024-02-14T15:01:13.583364+0000","last_fullsized":"2024-02-14T15:01:13.583364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:29:24.840030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669367+0000","last_change":"2024-02-14T15:00:49.272233+0000","last_active":"2024-02-14T15:00:54.669367+0000","last_peered":"2024-02-14T15:00:54.669367+0000","last_clean":"2024-02-14T15:00:54.669367+0000","last_became_active":"2024-02-14T15:00:49.271813+0000","last_became_peered":"2024-02-14T15:00:49.271813+0000","last_unstale":"2024-02-14T15:00:54.669367+0000","last_undegraded":"2024-02-14T15:00:54.669367+0000","last_fullsized":"2024-02-14T15:00:54.669367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:30:32.521443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669790+0000","last_change":"2024-02-14T15:00:53.583385+0000","last_active":"2024-02-14T15:00:54.669790+0000","last_peered":"2024-02-14T15:00:54.669790+0000","last_clean":"2024-02-14T15:00:54.669790+0000","last_became_active":"2024-02-14T15:00:53.583023+0000","last_became_peered":"2024-02-14T15:00:53.583023+0000","last_unstale":"2024-02-14T15:00:54.669790+0000","last_undegraded":"2024-02-14T15:00:54.669790+0000","last_fullsized":"2024-02-14T15:00:54.669790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:40:53.500939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.584073+0000","last_change":"2024-02-14T15:00:48.261376+0000","last_active":"2024-02-14T15:01:13.584073+0000","last_peered":"2024-02-14T15:01:13.584073+0000","last_clean":"2024-02-14T15:01:13.584073+0000","last_became_active":"2024-02-14T15:00:48.260461+0000","last_became_peered":"2024-02-14T15:00:48.260461+0000","last_unstale":"2024-02-14T15:01:13.584073+0000","last_undegraded":"2024-02-14T15:01:13.584073+0000","last_fullsized":"2024-02-14T15:01:13.584073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:35.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669401+0000","last_change":"2024-02-14T15:00:49.273117+0000","last_active":"2024-02-14T15:00:54.669401+0000","last_peered":"2024-02-14T15:00:54.669401+0000","last_clean":"2024-02-14T15:00:54.669401+0000","last_became_active":"2024-02-14T15:00:49.272810+0000","last_became_peered":"2024-02-14T15:00:49.272810+0000","last_unstale":"2024-02-14T15:00:54.669401+0000","last_undegraded":"2024-02-14T15:00:54.669401+0000","last_fullsized":"2024-02-14T15:00:54.669401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:37:00.992212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670882+0000","last_change":"2024-02-14T15:00:53.583972+0000","last_active":"2024-02-14T15:00:54.670882+0000","last_peered":"2024-02-14T15:00:54.670882+0000","last_clean":"2024-02-14T15:00:54.670882+0000","last_became_active":"2024-02-14T15:00:53.583563+0000","last_became_peered":"2024-02-14T15:00:53.583563+0000","last_unstale":"2024-02-14T15:00:54.670882+0000","last_undegraded":"2024-02-14T15:00:54.670882+0000","last_fullsized":"2024-02-14T15:00:54.670882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:08:55.811082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566490+0000","last_change":"2024-02-14T15:00:48.270363+0000","last_active":"2024-02-14T15:00:54.566490+0000","last_peered":"2024-02-14T15:00:54.566490+0000","last_clean":"2024-02-14T15:00:54.566490+0000","last_became_active":"2024-02-14T15:00:48.270030+0000","last_became_peered":"2024-02-14T15:00:48.270030+0000","last_unstale":"2024-02-14T15:00:54.566490+0000","last_undegraded":"2024-02-14T15:00:54.566490+0000","last_fullsized":"2024-02-14T15:00:54.566490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:28.885451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566482+0000","last_change":"2024-02-14T15:00:49.285647+0000","last_active":"2024-02-14T15:00:54.566482+0000","last_peered":"2024-02-14T15:00:54.566482+0000","last_clean":"2024-02-14T15:00:54.566482+0000","last_became_active":"2024-02-14T15:00:49.285267+0000","last_became_peered":"2024-02-14T15:00:49.285267+0000","last_unstale":"2024-02-14T15:00:54.566482+0000","last_undegraded":"2024-02-14T15:00:54.566482+0000","last_fullsized":"2024-02-14T15:00:54.566482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:03.143177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670946+0000","last_change":"2024-02-14T15:00:53.591029+0000","last_active":"2024-02-14T15:00:54.670946+0000","last_peered":"2024-02-14T15:00:54.670946+0000","last_clean":"2024-02-14T15:00:54.670946+0000","last_became_active":"2024-02-14T15:00:53.590570+0000","last_became_peered":"2024-02-14T15:00:53.590570+0000","last_unstale":"2024-02-14T15:00:54.670946+0000","last_undegraded":"2024-02-14T15:00:54.670946+0000","last_fullsized":"2024-02-14T15:00:54.670946+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:41.809227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583711+0000","last_change":"2024-02-14T15:00:48.277652+0000","last_active":"2024-02-14T15:01:13.583711+0000","last_peered":"2024-02-14T15:01:13.583711+0000","last_clean":"2024-02-14T15:01:13.583711+0000","last_became_active":"2024-02-14T15:00:48.277317+0000","last_became_peered":"2024-02-14T15:00:48.277317+0000","last_unstale":"2024-02-14T15:01:13.583711+0000","last_undegraded":"2024-02-14T15:01:13.583711+0000","last_fullsized":"2024-02-14T15:01:13.583711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:40:22.474551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566391+0000","last_change":"2024-02-14T15:00:49.276346+0000","last_active":"2024-02-14T15:00:54.566391+0000","last_peered":"2024-02-14T15:00:54.566391+0000","last_clean":"2024-02-14T15:00:54.566391+0000","last_became_active":"2024-02-14T15:00:49.276000+0000","last_became_peered":"2024-02-14T15:00:49.276000+0000","last_unstale":"2024-02-14T15:00:54.566391+0000","last_undegraded":"2024-02-14T15:00:54.566391+0000","last_fullsized":"2024-02-14T15:00:54.566391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:14:03.811981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669265+0000","last_change":"2024-02-14T15:00:53.587021+0000","last_active":"2024-02-14T15:00:54.669265+0000","last_peered":"2024-02-14T15:00:54.669265+0000","last_clean":"2024-02-14T15:00:54.669265+0000","last_became_active":"2024-02-14T15:00:53.586492+0000","last_became_peered":"2024-02-14T15:00:53.586492+0000","last_unstale":"2024-02-14T15:00:54.669265+0000","last_undegraded":"2024-02-14T15:00:54.669265+0000","last_fullsized":"2024-02-14T15:00:54.669265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:51:02.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566923+0000","last_change":"2024-02-14T15:00:48.270322+0000","last_active":"2024-02-14T15:00:54.566923+0000","last_peered":"2024-02-14T15:00:54.566923+0000","last_clean":"2024-02-14T15:00:54.566923+0000","last_became_active":"2024-02-14T15:00:48.269806+0000","last_became_peered":"2024-02-14T15:00:48.269806+0000","last_unstale":"2024-02-14T15:00:54.566923+0000","last_undegraded":"2024-02-14T15:00:54.566923+0000","last_fullsized":"2024-02-14T15:00:54.566923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:20:12.010694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566694+0000","last_change":"2024-02-14T15:00:49.276110+0000","last_active":"2024-02-14T15:00:54.566694+0000","last_peered":"2024-02-14T15:00:54.566694+0000","last_clean":"2024-02-14T15:00:54.566694+0000","last_became_active":"2024-02-14T15:00:49.275729+0000","last_became_peered":"2024-02-14T15:00:49.275729+0000","last_unstale":"2024-02-14T15:00:54.566694+0000","last_undegraded":"2024-02-14T15:00:54.566694+0000","last_fullsized":"2024-02-14T15:00:54.566694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:07:29.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569270+0000","last_change":"2024-02-14T15:00:53.582061+0000","last_active":"2024-02-14T15:00:54.569270+0000","last_peered":"2024-02-14T15:00:54.569270+0000","last_clean":"2024-02-14T15:00:54.569270+0000","last_became_active":"2024-02-14T15:00:53.581659+0000","last_became_peered":"2024-02-14T15:00:53.581659+0000","last_unstale":"2024-02-14T15:00:54.569270+0000","last_undegraded":"2024-02-14T15:00:54.569270+0000","last_fullsized":"2024-02-14T15:00:54.569270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:13.814496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.671090+0000","last_change":"2024-02-14T15:00:48.283300+0000","last_active":"2024-02-14T15:00:54.671090+0000","last_peered":"2024-02-14T15:00:54.671090+0000","last_clean":"2024-02-14T15:00:54.671090+0000","last_became_active":"2024-02-14T15:00:48.282939+0000","last_became_peered":"2024-02-14T15:00:48.282939+0000","last_unstale":"2024-02-14T15:00:54.671090+0000","last_undegraded":"2024-02-14T15:00:54.671090+0000","last_fullsized":"2024-02-14T15:00:54.671090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:56:52.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566464+0000","last_change":"2024-02-14T15:00:49.273435+0000","last_active":"2024-02-14T15:00:54.566464+0000","last_peered":"2024-02-14T15:00:54.566464+0000","last_clean":"2024-02-14T15:00:54.566464+0000","last_became_active":"2024-02-14T15:00:49.272980+0000","last_became_peered":"2024-02-14T15:00:49.272980+0000","last_unstale":"2024-02-14T15:00:54.566464+0000","last_undegraded":"2024-02-14T15:00:54.566464+0000","last_fullsized":"2024-02-14T15:00:54.566464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:19:05.294943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669872+0000","last_change":"2024-02-14T15:00:53.583509+0000","last_active":"2024-02-14T15:00:54.669872+0000","last_peered":"2024-02-14T15:00:54.669872+0000","last_clean":"2024-02-14T15:00:54.669872+0000","last_became_active":"2024-02-14T15:00:53.583148+0000","last_became_peered":"2024-02-14T15:00:53.583148+0000","last_unstale":"2024-02-14T15:00:54.669872+0000","last_undegraded":"2024-02-14T15:00:54.669872+0000","last_fullsized":"2024-02-14T15:00:54.669872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:06:47.059643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669695+0000","last_change":"2024-02-14T15:00:48.286707+0000","last_active":"2024-02-14T15:00:54.669695+0000","last_peered":"2024-02-14T15:00:54.669695+0000","last_clean":"2024-02-14T15:00:54.669695+0000","last_became_active":"2024-02-14T15:00:48.286355+0000","last_became_peered":"2024-02-14T15:00:48.286355+0000","last_unstale":"2024-02-14T15:00:54.669695+0000","last_undegraded":"2024-02-14T15:00:54.669695+0000","last_fullsized":"2024-02-14T15:00:54.669695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:48:31.024464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566799+0000","last_change":"2024-02-14T15:00:49.271598+0000","last_active":"2024-02-14T15:00:54.566799+0000","last_peered":"2024-02-14T15:00:54.566799+0000","last_clean":"2024-02-14T15:00:54.566799+0000","last_became_active":"2024-02-14T15:00:49.270572+0000","last_became_peered":"2024-02-14T15:00:49.270572+0000","last_unstale":"2024-02-14T15:00:54.566799+0000","last_undegraded":"2024-02-14T15:00:54.566799+0000","last_fullsized":"2024-02-14T15:00:54.566799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:12.666557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669530+0000","last_change":"2024-02-14T15:00:49.276697+0000","last_active":"2024-02-14T15:00:54.669530+0000","last_peered":"2024-02-14T15:00:54.669530+0000","last_clean":"2024-02-14T15:00:54.669530+0000","last_became_active":"2024-02-14T15:00:49.276279+0000","last_became_peered":"2024-02-14T15:00:49.276279+0000","last_unstale":"2024-02-14T15:00:54.669530+0000","last_undegraded":"2024-02-14T15:00:54.669530+0000","last_fullsized":"2024-02-14T15:00:54.669530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:11:31.824974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566723+0000","last_change":"2024-02-14T15:00:49.270104+0000","last_active":"2024-02-14T15:00:54.566723+0000","last_peered":"2024-02-14T15:00:54.566723+0000","last_clean":"2024-02-14T15:00:54.566723+0000","last_became_active":"2024-02-14T15:00:49.269742+0000","last_became_peered":"2024-02-14T15:00:49.269742+0000","last_unstale":"2024-02-14T15:00:54.566723+0000","last_undegraded":"2024-02-14T15:00:54.566723+0000","last_fullsized":"2024-02-14T15:00:54.566723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:10:07.494478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566445+0000","last_change":"2024-02-14T15:00:49.274219+0000","last_active":"2024-02-14T15:00:54.566445+0000","last_peered":"2024-02-14T15:00:54.566445+0000","last_clean":"2024-02-14T15:00:54.566445+0000","last_became_active":"2024-02-14T15:00:49.273833+0000","last_became_peered":"2024-02-14T15:00:49.273833+0000","last_unstale":"2024-02-14T15:00:54.566445+0000","last_undegraded":"2024-02-14T15:00:54.566445+0000","last_fullsized":"2024-02-14T15:00:54.566445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:06:02.720273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569416+0000","last_change":"2024-02-14T15:00:49.273576+0000","last_active":"2024-02-14T15:00:54.569416+0000","last_peered":"2024-02-14T15:00:54.569416+0000","last_clean":"2024-02-14T15:00:54.569416+0000","last_became_active":"2024-02-14T15:00:49.272920+0000","last_became_peered":"2024-02-14T15:00:49.272920+0000","last_unstale":"2024-02-14T15:00:54.569416+0000","last_undegraded":"2024-02-14T15:00:54.569416+0000","last_fullsized":"2024-02-14T15:00:54.569416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:22:03.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670610+0000","last_change":"2024-02-14T15:00:49.274549+0000","last_active":"2024-02-14T15:00:54.670610+0000","last_peered":"2024-02-14T15:00:54.670610+0000","last_clean":"2024-02-14T15:00:54.670610+0000","last_became_active":"2024-02-14T15:00:49.274097+0000","last_became_peered":"2024-02-14T15:00:49.274097+0000","last_unstale":"2024-02-14T15:00:54.670610+0000","last_undegraded":"2024-02-14T15:00:54.670610+0000","last_fullsized":"2024-02-14T15:00:54.670610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:47:44.017607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669957+0000","last_change":"2024-02-14T15:00:49.275699+0000","last_active":"2024-02-14T15:00:54.669957+0000","last_peered":"2024-02-14T15:00:54.669957+0000","last_clean":"2024-02-14T15:00:54.669957+0000","last_became_active":"2024-02-14T15:00:49.275361+0000","last_became_peered":"2024-02-14T15:00:49.275361+0000","last_unstale":"2024-02-14T15:00:54.669957+0000","last_undegraded":"2024-02-14T15:00:54.669957+0000","last_fullsized":"2024-02-14T15:00:54.669957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:34:02.638758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567302+0000","last_change":"2024-02-14T15:00:49.275024+0000","last_active":"2024-02-14T15:00:54.567302+0000","last_peered":"2024-02-14T15:00:54.567302+0000","last_clean":"2024-02-14T15:00:54.567302+0000","last_became_active":"2024-02-14T15:00:49.274710+0000","last_became_peered":"2024-02-14T15:00:49.274710+0000","last_unstale":"2024-02-14T15:00:54.567302+0000","last_undegraded":"2024-02-14T15:00:54.567302+0000","last_fullsized":"2024-02-14T15:00:54.567302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:42.471112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567055+0000","last_change":"2024-02-14T15:00:49.282638+0000","last_active":"2024-02-14T15:00:54.567055+0000","last_peered":"2024-02-14T15:00:54.567055+0000","last_clean":"2024-02-14T15:00:54.567055+0000","last_became_active":"2024-02-14T15:00:49.280223+0000","last_became_peered":"2024-02-14T15:00:49.280223+0000","last_unstale":"2024-02-14T15:00:54.567055+0000","last_undegraded":"2024-02-14T15:00:54.567055+0000","last_fullsized":"2024-02-14T15:00:54.567055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:40.250242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669972+0000","last_change":"2024-02-14T15:00:49.273025+0000","last_active":"2024-02-14T15:00:54.669972+0000","last_peered":"2024-02-14T15:00:54.669972+0000","last_clean":"2024-02-14T15:00:54.669972+0000","last_became_active":"2024-02-14T15:00:49.272700+0000","last_became_peered":"2024-02-14T15:00:49.272700+0000","last_unstale":"2024-02-14T15:00:54.669972+0000","last_undegraded":"2024-02-14T15:00:54.669972+0000","last_fullsized":"2024-02-14T15:00:54.669972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:08.677428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566252+0000","last_change":"2024-02-14T15:00:49.282263+0000","last_active":"2024-02-14T15:00:54.566252+0000","last_peered":"2024-02-14T15:00:54.566252+0000","last_clean":"2024-02-14T15:00:54.566252+0000","last_became_active":"2024-02-14T15:00:49.279677+0000","last_became_peered":"2024-02-14T15:00:49.279677+0000","last_unstale":"2024-02-14T15:00:54.566252+0000","last_undegraded":"2024-02-14T15:00:54.566252+0000","last_fullsized":"2024-02-14T15:00:54.566252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:48.036316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669507+0000","last_change":"2024-02-14T15:00:49.282178+0000","last_active":"2024-02-14T15:00:54.669507+0000","last_peered":"2024-02-14T15:00:54.669507+0000","last_clean":"2024-02-14T15:00:54.669507+0000","last_became_active":"2024-02-14T15:00:49.281804+0000","last_became_peered":"2024-02-14T15:00:49.281804+0000","last_unstale":"2024-02-14T15:00:54.669507+0000","last_undegraded":"2024-02-14T15:00:54.669507+0000","last_fullsized":"2024-02-14T15:00:54.669507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:58.672808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669881+0000","last_change":"2024-02-14T15:00:49.269882+0000","last_active":"2024-02-14T15:00:54.669881+0000","last_peered":"2024-02-14T15:00:54.669881+0000","last_clean":"2024-02-14T15:00:54.669881+0000","last_became_active":"2024-02-14T15:00:49.269489+0000","last_became_peered":"2024-02-14T15:00:49.269489+0000","last_unstale":"2024-02-14T15:00:54.669881+0000","last_undegraded":"2024-02-14T15:00:54.669881+0000","last_fullsized":"2024-02-14T15:00:54.669881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:57:40.465044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670052+0000","last_change":"2024-02-14T15:00:49.277688+0000","last_active":"2024-02-14T15:00:54.670052+0000","last_peered":"2024-02-14T15:00:54.670052+0000","last_clean":"2024-02-14T15:00:54.670052+0000","last_became_active":"2024-02-14T15:00:49.277348+0000","last_became_peered":"2024-02-14T15:00:49.277348+0000","last_unstale":"2024-02-14T15:00:54.670052+0000","last_undegraded":"2024-02-14T15:00:54.670052+0000","last_fullsized":"2024-02-14T15:00:54.670052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:33.946319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6870,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":108,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":20132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":476575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64300000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.91200000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.752}]},{"osd":7,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.85699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":9,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.871},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.88300000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.90000000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1597440,"data_stored":1070979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.312}]},{"osd":2,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.47699999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.498},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56999999999999995}]},{"osd":9,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.748},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.60399999999999998}]},{"osd":11,"last update":"Wed Feb 14 15:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66700000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":477157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61299999999999999,"5min":1.22,"15min":1.22},"last":0.52700000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88900000000000001,"5min":1.246,"15min":1.246},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":1.069,"15min":1.069},"last":0.78300000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.745,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.57499999999999996,"15min":0.58699999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.875,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.626},{"interface":"front","average":{"1min":0.628,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.141,"15min":1.141},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.6100000000000001,"5min":0.93000000000000005,"15min":0.81699999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":19.152000000000001,"5min":19.152000000000001,"15min":19.152000000000001},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.31},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":0.97699999999999998,"15min":0.875},"min":{"1min":0.504,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":18.734999999999999,"5min":18.734999999999999,"15min":18.734999999999999},"last":0.749},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.92300000000000004,"15min":0.95899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0409999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.73399999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":0.97999999999999998,"15min":0.87},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":19.135999999999999,"5min":19.135999999999999,"15min":19.135999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.83199999999999996,"15min":0.85399999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0229999999999999,"5min":1.23,"15min":1.23},"last":0.97699999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:00:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.87},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.94699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55200000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.746}]},{"osd":2,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42099999999999999,"15min":0.42199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71799999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73399999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66700000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.751,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.627,"5min":1.161,"15min":1.25},"min":{"1min":0.47099999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79100000000000004,"5min":13.618,"15min":13.618},"last":0.68700000000000006}]},{"osd":7,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.87},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.76600000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83099999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68000000000000005,"15min":0.66900000000000004},"min":{"1min":0.56200000000000006,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80300000000000005}]},{"osd":10,"last update":"Wed Feb 14 15:00:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.90000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1068735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.38200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.39900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.502,"5min":0.499,"15min":0.499},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68899999999999995}]},{"osd":4,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63700000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48899999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83299999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84099999999999997}]},{"osd":6,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.56699999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.58299999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":1.0169999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.59999999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.018,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.0029999999999999}]},{"osd":9,"last update":"Wed Feb 14 15:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":1.052},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.84399999999999997,"15min":0.85399999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.998,"5min":1.175,"15min":1.175},"last":0.98399999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.94799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":476665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.434}]},{"osd":1,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78300000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80700000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.755,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.875},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.443},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.309,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94499999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90900000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0369999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.502,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.79000000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.73199999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.85599999999999998}]},{"osd":9,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93500000000000005,"5min":0.996,"15min":0.996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":1.226,"15min":1.226},"last":0.70199999999999996}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53200000000000003,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.376},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53400000000000003,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56799999999999995,"15min":0.52200000000000002},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.77000000000000002,"15min":0.57399999999999995},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":18.332000000000001,"15min":18.332000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Feb 14 15:00:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.72999999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.626,"15min":0.59399999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64600000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:01:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.497,"15min":0.46300000000000002},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.81899999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.35199999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:00:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.56399999999999995},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.751},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.626,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66500000000000004}]},{"osd":6,"last update":"Wed Feb 14 15:00:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.36699999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.65900000000000003,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:00:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.442}]},{"osd":8,"last update":"Wed Feb 14 15:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.73799999999999999,"15min":0.72699999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80400000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90000000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71499999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.70399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":478053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 15:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.502},"min":{"1min":0.31,"5min":0.253,"15min":0.253},"max":{"1min":0.74299999999999999,"5min":1.002,"15min":1.002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52300000000000002,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:00:17 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53800000000000003,"15min":0.502},"min":{"1min":0.379,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.51400000000000001},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.745}]},{"osd":3,"last update":"Wed Feb 14 14:59:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.65500000000000003,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.45000000000000001,"15min":0.42599999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39400000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54700000000000004,"15min":0.51200000000000001},"min":{"1min":0.38600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:00:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56499999999999995,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.69699999999999995}]},{"osd":6,"last update":"Wed Feb 14 15:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:00:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65800000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:00:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82599999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92700000000000005,"5min":1.052,"15min":1.052},"last":0.78100000000000003}]},{"osd":9,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.47299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.75900000000000001}]},{"osd":11,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.79600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48999999999999999,"15min":0.46999999999999997},"min":{"1min":0.31900000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.47499999999999998},"min":{"1min":0.36499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.627}]},{"osd":1,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.505},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40899999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77100000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.42399999999999999},"min":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67400000000000004,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.435,"15min":0.42899999999999999},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.41599999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91400000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.499},{"interface":"front","average":{"1min":0.624,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.746},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81599999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70499999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58999999999999997}]},{"osd":9,"last update":"Wed Feb 14 15:01:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61199999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":476769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.314},"min":{"1min":0.25700000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.372,"15min":0.34799999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.83599999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72899999999999998,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.081,"5min":19.896999999999998,"15min":19.896999999999998},"last":0.504},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.74299999999999999,"15min":0.59299999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86599999999999999,"5min":19.911999999999999,"15min":19.911999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56000000000000005,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57699999999999996,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Feb 14 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83499999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73299999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65600000000000003}]},{"osd":5,"last update":"Wed Feb 14 15:00:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.2549999999999999,"15min":1.542},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89500000000000002,"5min":19.879999999999999,"15min":19.879999999999999},"last":0.628},{"interface":"front","average":{"1min":0.626,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80700000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.439},{"interface":"front","average":{"1min":0.496,"5min":0.5,"15min":0.48499999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40400000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91600000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.875,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55600000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70499999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.66300000000000003,"15min":0.67900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52700000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:00:37 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":476597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.29499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.63700000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.629},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.42899999999999999,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58199999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71599999999999997}]},{"osd":2,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82099999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.58999999999999997},"min":{"1min":0.41299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83199999999999996,"5min":0.997,"15min":0.997},"last":0.76300000000000001}]},{"osd":3,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.39900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Wed Feb 14 15:00:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.42499999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.874},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91500000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83199999999999996}]},{"osd":6,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":0.91100000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:00:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68000000000000005,"15min":0.68799999999999994},"min":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.81000000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92100000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55200000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.042,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.996}]},{"osd":9,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.57299999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.999,"5min":1.04,"15min":1.04},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.052}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.032}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1068419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.373,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.48899999999999999,"15min":0.46800000000000003},"min":{"1min":0.38100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.624}]},{"osd":2,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36199999999999999,"15min":0.34899999999999998},"min":{"1min":0.28699999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27400000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.42899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.68799999999999994}]},{"osd":6,"last update":"Wed Feb 14 15:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.46500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:00:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71799999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60499999999999998}]},{"osd":8,"last update":"Wed Feb 14 15:00:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54800000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:00:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.75600000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T15:01:18.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-14T15:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 -- 172.21.15.17:0/4442043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c106f40 msgr2=0x7fa61c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 --2- 172.21.15.17:0/4442043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c106f40 0x7fa61c109390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009ca0 tx=0x7fa60c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 -- 172.21.15.17:0/4442043 shutdown_connections 2024-02-14T15:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 --2- 172.21.15.17:0/4442043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c106f40 0x7fa61c109390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 --2- 172.21.15.17:0/4442043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa61c104610 0x7fa61c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 --2- 172.21.15.17:0/4442043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c101ce0 0x7fa61c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 -- 172.21.15.17:0/4442043 >> 172.21.15.17:0/4442043 conn(0x7fa61c0fb6f0 msgr2=0x7fa61c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 -- 172.21.15.17:0/4442043 shutdown_connections 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.472+0000 7fa62166b640 1 -- 172.21.15.17:0/4442043 wait complete. 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 Processor -- start 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 -- start start 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c101ce0 0x7fa61c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c104610 0x7fa61c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa61c106f40 0x7fa61c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19ecb0 con 0x7fa61c104610 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19ee20 con 0x7fa61c101ce0 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa62166b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19efe0 con 0x7fa61c106f40 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c104610 0x7fa61c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c104610 0x7fa61c197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55382/0 (socket says 172.21.15.17:55382) 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa61a7fc640 1 -- 172.21.15.17:0/3862436730 learned_addr learned my addr 172.21.15.17:0/3862436730 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.473+0000 7fa61affd640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c101ce0 0x7fa61c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61b7fe640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa61c106f40 0x7fa61c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61a7fc640 1 -- 172.21.15.17:0/3862436730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa61c106f40 msgr2=0x7fa61c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61a7fc640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa61c106f40 0x7fa61c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61a7fc640 1 -- 172.21.15.17:0/3862436730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c101ce0 msgr2=0x7fa61c1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61a7fc640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c101ce0 0x7fa61c1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61a7fc640 1 -- 172.21.15.17:0/3862436730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c009900 con 0x7fa61c104610 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61affd640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c101ce0 0x7fa61c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa61a7fc640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c104610 0x7fa61c197700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d770 tx=0x7fa60400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa613fff640 1 -- 172.21.15.17:0/3862436730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604009c30 con 0x7fa61c104610 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa613fff640 1 -- 172.21.15.17:0/3862436730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa60400de80 con 0x7fa61c104610 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c19f2c0 con 0x7fa61c104610 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa613fff640 1 -- 172.21.15.17:0/3862436730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60400f6b0 con 0x7fa61c104610 2024-02-14T15:01:18.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.474+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c19f7d0 con 0x7fa61c104610 2024-02-14T15:01:18.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.475+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c0fd1e0 con 0x7fa61c104610 2024-02-14T15:01:18.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.477+0000 7fa613fff640 1 -- 172.21.15.17:0/3862436730 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa604005030 con 0x7fa61c104610 2024-02-14T15:01:18.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.477+0000 7fa613fff640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6080a2150 0x7fa6080a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:18.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.477+0000 7fa61affd640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6080a2150 0x7fa6080a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:18.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.477+0000 7fa613fff640 1 -- 172.21.15.17:0/3862436730 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa6040cde60 con 0x7fa61c104610 2024-02-14T15:01:18.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.478+0000 7fa61affd640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6080a2150 0x7fa6080a4610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa61400abe0 tx=0x7fa61400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:18.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.481+0000 7fa613fff640 1 -- 172.21.15.17:0/3862436730 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa604096ae0 con 0x7fa61c104610 2024-02-14T15:01:18.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.661+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa61c02cec0 con 0x7fa6080a2150 2024-02-14T15:01:18.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.674+0000 7fa613fff640 1 -- 172.21.15.17:0/3862436730 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423460 (secure 0 0 0) 0x7fa61c02cec0 con 0x7fa6080a2150 2024-02-14T15:01:18.680 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:18.682 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6080a2150 msgr2=0x7fa6080a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6080a2150 0x7fa6080a4610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa61400abe0 tx=0x7fa61400a5c0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c104610 msgr2=0x7fa61c197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c104610 0x7fa61c197700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d770 tx=0x7fa60400dc30 comp rx=0 tx=0).stop 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 shutdown_connections 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6080a2150 0x7fa6080a4610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61c104610 0x7fa61c197700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa61c106f40 0x7fa61c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 --2- 172.21.15.17:0/3862436730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa61c101ce0 0x7fa61c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.677+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 >> 172.21.15.17:0/3862436730 conn(0x7fa61c0fb6f0 msgr2=0x7fa61c102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:18.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.678+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 shutdown_connections 2024-02-14T15:01:18.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:18.678+0000 7fa62166b640 1 -- 172.21.15.17:0/3862436730 wait complete. 2024-02-14T15:01:18.689 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-02-14T15:01:18.660120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639067,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":159,"num_read_kb":180,"num_write":317,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387572,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554252,"statfs":{"total":1151940427776,"available":1151543554048,"internally_reserved":0,"allocated":13783040,"data_stored":7554962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44200,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":23,"num_read_kb":61,"num_write":89,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003301"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670461+0000","last_change":"2024-02-14T15:00:49.281057+0000","last_active":"2024-02-14T15:00:54.670461+0000","last_peered":"2024-02-14T15:00:54.670461+0000","last_clean":"2024-02-14T15:00:54.670461+0000","last_became_active":"2024-02-14T15:00:49.280642+0000","last_became_peered":"2024-02-14T15:00:49.280642+0000","last_unstale":"2024-02-14T15:00:54.670461+0000","last_undegraded":"2024-02-14T15:00:54.670461+0000","last_fullsized":"2024-02-14T15:00:54.670461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:03.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569726+0000","last_change":"2024-02-14T15:00:49.282794+0000","last_active":"2024-02-14T15:00:54.569726+0000","last_peered":"2024-02-14T15:00:54.569726+0000","last_clean":"2024-02-14T15:00:54.569726+0000","last_became_active":"2024-02-14T15:00:49.281441+0000","last_became_peered":"2024-02-14T15:00:49.281441+0000","last_unstale":"2024-02-14T15:00:54.569726+0000","last_undegraded":"2024-02-14T15:00:54.569726+0000","last_fullsized":"2024-02-14T15:00:54.569726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:57:36.577281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569612+0000","last_change":"2024-02-14T15:00:49.286253+0000","last_active":"2024-02-14T15:00:54.569612+0000","last_peered":"2024-02-14T15:00:54.569612+0000","last_clean":"2024-02-14T15:00:54.569612+0000","last_became_active":"2024-02-14T15:00:49.285950+0000","last_became_peered":"2024-02-14T15:00:49.285950+0000","last_unstale":"2024-02-14T15:00:54.569612+0000","last_undegraded":"2024-02-14T15:00:54.569612+0000","last_fullsized":"2024-02-14T15:00:54.569612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:53.395339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670936+0000","last_change":"2024-02-14T15:00:49.272764+0000","last_active":"2024-02-14T15:00:54.670936+0000","last_peered":"2024-02-14T15:00:54.670936+0000","last_clean":"2024-02-14T15:00:54.670936+0000","last_became_active":"2024-02-14T15:00:49.272430+0000","last_became_peered":"2024-02-14T15:00:49.272430+0000","last_unstale":"2024-02-14T15:00:54.670936+0000","last_undegraded":"2024-02-14T15:00:54.670936+0000","last_fullsized":"2024-02-14T15:00:54.670936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:36.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669582+0000","last_change":"2024-02-14T15:00:49.275028+0000","last_active":"2024-02-14T15:00:54.669582+0000","last_peered":"2024-02-14T15:00:54.669582+0000","last_clean":"2024-02-14T15:00:54.669582+0000","last_became_active":"2024-02-14T15:00:49.274686+0000","last_became_peered":"2024-02-14T15:00:49.274686+0000","last_unstale":"2024-02-14T15:00:54.669582+0000","last_undegraded":"2024-02-14T15:00:54.669582+0000","last_fullsized":"2024-02-14T15:00:54.669582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:21:23.465321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669806+0000","last_change":"2024-02-14T15:00:49.287144+0000","last_active":"2024-02-14T15:00:54.669806+0000","last_peered":"2024-02-14T15:00:54.669806+0000","last_clean":"2024-02-14T15:00:54.669806+0000","last_became_active":"2024-02-14T15:00:49.286767+0000","last_became_peered":"2024-02-14T15:00:49.286767+0000","last_unstale":"2024-02-14T15:00:54.669806+0000","last_undegraded":"2024-02-14T15:00:54.669806+0000","last_fullsized":"2024-02-14T15:00:54.669806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:45:09.391595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566171+0000","last_change":"2024-02-14T15:00:49.274442+0000","last_active":"2024-02-14T15:00:54.566171+0000","last_peered":"2024-02-14T15:00:54.566171+0000","last_clean":"2024-02-14T15:00:54.566171+0000","last_became_active":"2024-02-14T15:00:49.274122+0000","last_became_peered":"2024-02-14T15:00:49.274122+0000","last_unstale":"2024-02-14T15:00:54.566171+0000","last_undegraded":"2024-02-14T15:00:54.566171+0000","last_fullsized":"2024-02-14T15:00:54.566171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:55.974889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566077+0000","last_change":"2024-02-14T15:00:49.285432+0000","last_active":"2024-02-14T15:00:54.566077+0000","last_peered":"2024-02-14T15:00:54.566077+0000","last_clean":"2024-02-14T15:00:54.566077+0000","last_became_active":"2024-02-14T15:00:49.283703+0000","last_became_peered":"2024-02-14T15:00:49.283703+0000","last_unstale":"2024-02-14T15:00:54.566077+0000","last_undegraded":"2024-02-14T15:00:54.566077+0000","last_fullsized":"2024-02-14T15:00:54.566077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:14.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670013+0000","last_change":"2024-02-14T15:00:49.283908+0000","last_active":"2024-02-14T15:00:54.670013+0000","last_peered":"2024-02-14T15:00:54.670013+0000","last_clean":"2024-02-14T15:00:54.670013+0000","last_became_active":"2024-02-14T15:00:49.283131+0000","last_became_peered":"2024-02-14T15:00:49.283131+0000","last_unstale":"2024-02-14T15:00:54.670013+0000","last_undegraded":"2024-02-14T15:00:54.670013+0000","last_fullsized":"2024-02-14T15:00:54.670013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:42.247624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670908+0000","last_change":"2024-02-14T15:00:49.287229+0000","last_active":"2024-02-14T15:00:54.670908+0000","last_peered":"2024-02-14T15:00:54.670908+0000","last_clean":"2024-02-14T15:00:54.670908+0000","last_became_active":"2024-02-14T15:00:49.286791+0000","last_became_peered":"2024-02-14T15:00:49.286791+0000","last_unstale":"2024-02-14T15:00:54.670908+0000","last_undegraded":"2024-02-14T15:00:54.670908+0000","last_fullsized":"2024-02-14T15:00:54.670908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:33:01.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566653+0000","last_change":"2024-02-14T15:00:49.281669+0000","last_active":"2024-02-14T15:00:54.566653+0000","last_peered":"2024-02-14T15:00:54.566653+0000","last_clean":"2024-02-14T15:00:54.566653+0000","last_became_active":"2024-02-14T15:00:49.281298+0000","last_became_peered":"2024-02-14T15:00:49.281298+0000","last_unstale":"2024-02-14T15:00:54.566653+0000","last_undegraded":"2024-02-14T15:00:54.566653+0000","last_fullsized":"2024-02-14T15:00:54.566653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:54:40.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670845+0000","last_change":"2024-02-14T15:00:49.289679+0000","last_active":"2024-02-14T15:00:54.670845+0000","last_peered":"2024-02-14T15:00:54.670845+0000","last_clean":"2024-02-14T15:00:54.670845+0000","last_became_active":"2024-02-14T15:00:49.288767+0000","last_became_peered":"2024-02-14T15:00:49.288767+0000","last_unstale":"2024-02-14T15:00:54.670845+0000","last_undegraded":"2024-02-14T15:00:54.670845+0000","last_fullsized":"2024-02-14T15:00:54.670845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:16:20.181517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669768+0000","last_change":"2024-02-14T15:00:49.257471+0000","last_active":"2024-02-14T15:00:54.669768+0000","last_peered":"2024-02-14T15:00:54.669768+0000","last_clean":"2024-02-14T15:00:54.669768+0000","last_became_active":"2024-02-14T15:00:49.256766+0000","last_became_peered":"2024-02-14T15:00:49.256766+0000","last_unstale":"2024-02-14T15:00:54.669768+0000","last_undegraded":"2024-02-14T15:00:54.669768+0000","last_fullsized":"2024-02-14T15:00:54.669768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:14:00.525621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569150+0000","last_change":"2024-02-14T15:00:49.278462+0000","last_active":"2024-02-14T15:00:54.569150+0000","last_peered":"2024-02-14T15:00:54.569150+0000","last_clean":"2024-02-14T15:00:54.569150+0000","last_became_active":"2024-02-14T15:00:49.278114+0000","last_became_peered":"2024-02-14T15:00:49.278114+0000","last_unstale":"2024-02-14T15:00:54.569150+0000","last_undegraded":"2024-02-14T15:00:54.569150+0000","last_fullsized":"2024-02-14T15:00:54.569150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:02:16.601359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569089+0000","last_change":"2024-02-14T15:00:49.270653+0000","last_active":"2024-02-14T15:00:54.569089+0000","last_peered":"2024-02-14T15:00:54.569089+0000","last_clean":"2024-02-14T15:00:54.569089+0000","last_became_active":"2024-02-14T15:00:49.270334+0000","last_became_peered":"2024-02-14T15:00:49.270334+0000","last_unstale":"2024-02-14T15:00:54.569089+0000","last_undegraded":"2024-02-14T15:00:54.569089+0000","last_fullsized":"2024-02-14T15:00:54.569089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:13:53.840909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.668977+0000","last_change":"2024-02-14T15:00:49.280597+0000","last_active":"2024-02-14T15:00:54.668977+0000","last_peered":"2024-02-14T15:00:54.668977+0000","last_clean":"2024-02-14T15:00:54.668977+0000","last_became_active":"2024-02-14T15:00:49.279612+0000","last_became_peered":"2024-02-14T15:00:49.279612+0000","last_unstale":"2024-02-14T15:00:54.668977+0000","last_undegraded":"2024-02-14T15:00:54.668977+0000","last_fullsized":"2024-02-14T15:00:54.668977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:11.676608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670351+0000","last_change":"2024-02-14T15:00:49.281422+0000","last_active":"2024-02-14T15:00:54.670351+0000","last_peered":"2024-02-14T15:00:54.670351+0000","last_clean":"2024-02-14T15:00:54.670351+0000","last_became_active":"2024-02-14T15:00:49.281048+0000","last_became_peered":"2024-02-14T15:00:49.281048+0000","last_unstale":"2024-02-14T15:00:54.670351+0000","last_undegraded":"2024-02-14T15:00:54.670351+0000","last_fullsized":"2024-02-14T15:00:54.670351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:04.994390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669975+0000","last_change":"2024-02-14T15:00:49.288245+0000","last_active":"2024-02-14T15:00:54.669975+0000","last_peered":"2024-02-14T15:00:54.669975+0000","last_clean":"2024-02-14T15:00:54.669975+0000","last_became_active":"2024-02-14T15:00:49.287908+0000","last_became_peered":"2024-02-14T15:00:49.287908+0000","last_unstale":"2024-02-14T15:00:54.669975+0000","last_undegraded":"2024-02-14T15:00:54.669975+0000","last_fullsized":"2024-02-14T15:00:54.669975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:52.397826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567037+0000","last_change":"2024-02-14T15:00:49.275428+0000","last_active":"2024-02-14T15:00:54.567037+0000","last_peered":"2024-02-14T15:00:54.567037+0000","last_clean":"2024-02-14T15:00:54.567037+0000","last_became_active":"2024-02-14T15:00:49.273464+0000","last_became_peered":"2024-02-14T15:00:49.273464+0000","last_unstale":"2024-02-14T15:00:54.567037+0000","last_undegraded":"2024-02-14T15:00:54.567037+0000","last_fullsized":"2024-02-14T15:00:54.567037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:34.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566596+0000","last_change":"2024-02-14T15:00:53.587219+0000","last_active":"2024-02-14T15:00:54.566596+0000","last_peered":"2024-02-14T15:00:54.566596+0000","last_clean":"2024-02-14T15:00:54.566596+0000","last_became_active":"2024-02-14T15:00:53.586917+0000","last_became_peered":"2024-02-14T15:00:53.586917+0000","last_unstale":"2024-02-14T15:00:54.566596+0000","last_undegraded":"2024-02-14T15:00:54.566596+0000","last_fullsized":"2024-02-14T15:00:54.566596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:39:38.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.590343+0000","last_change":"2024-02-14T15:00:48.269243+0000","last_active":"2024-02-14T15:01:14.590343+0000","last_peered":"2024-02-14T15:01:14.590343+0000","last_clean":"2024-02-14T15:01:14.590343+0000","last_became_active":"2024-02-14T15:00:48.268883+0000","last_became_peered":"2024-02-14T15:00:48.268883+0000","last_unstale":"2024-02-14T15:01:14.590343+0000","last_undegraded":"2024-02-14T15:01:14.590343+0000","last_fullsized":"2024-02-14T15:01:14.590343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:08.530092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567087+0000","last_change":"2024-02-14T15:00:49.282458+0000","last_active":"2024-02-14T15:00:54.567087+0000","last_peered":"2024-02-14T15:00:54.567087+0000","last_clean":"2024-02-14T15:00:54.567087+0000","last_became_active":"2024-02-14T15:00:49.281982+0000","last_became_peered":"2024-02-14T15:00:49.281982+0000","last_unstale":"2024-02-14T15:00:54.567087+0000","last_undegraded":"2024-02-14T15:00:54.567087+0000","last_fullsized":"2024-02-14T15:00:54.567087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:51:33.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669555+0000","last_change":"2024-02-14T15:00:53.586578+0000","last_active":"2024-02-14T15:00:54.669555+0000","last_peered":"2024-02-14T15:00:54.669555+0000","last_clean":"2024-02-14T15:00:54.669555+0000","last_became_active":"2024-02-14T15:00:53.586109+0000","last_became_peered":"2024-02-14T15:00:53.586109+0000","last_unstale":"2024-02-14T15:00:54.669555+0000","last_undegraded":"2024-02-14T15:00:54.669555+0000","last_fullsized":"2024-02-14T15:00:54.669555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:07.332435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567928+0000","last_change":"2024-02-14T15:00:48.281953+0000","last_active":"2024-02-14T15:00:54.567928+0000","last_peered":"2024-02-14T15:00:54.567928+0000","last_clean":"2024-02-14T15:00:54.567928+0000","last_became_active":"2024-02-14T15:00:48.281590+0000","last_became_peered":"2024-02-14T15:00:48.281590+0000","last_unstale":"2024-02-14T15:00:54.567928+0000","last_undegraded":"2024-02-14T15:00:54.567928+0000","last_fullsized":"2024-02-14T15:00:54.567928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:22.991037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569845+0000","last_change":"2024-02-14T15:00:49.284729+0000","last_active":"2024-02-14T15:00:54.569845+0000","last_peered":"2024-02-14T15:00:54.569845+0000","last_clean":"2024-02-14T15:00:54.569845+0000","last_became_active":"2024-02-14T15:00:49.284013+0000","last_became_peered":"2024-02-14T15:00:49.284013+0000","last_unstale":"2024-02-14T15:00:54.569845+0000","last_undegraded":"2024-02-14T15:00:54.569845+0000","last_fullsized":"2024-02-14T15:00:54.569845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:03:31.313032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669267+0000","last_change":"2024-02-14T15:00:53.589825+0000","last_active":"2024-02-14T15:00:54.669267+0000","last_peered":"2024-02-14T15:00:54.669267+0000","last_clean":"2024-02-14T15:00:54.669267+0000","last_became_active":"2024-02-14T15:00:53.589424+0000","last_became_peered":"2024-02-14T15:00:53.589424+0000","last_unstale":"2024-02-14T15:00:54.669267+0000","last_undegraded":"2024-02-14T15:00:54.669267+0000","last_fullsized":"2024-02-14T15:00:54.669267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:24.522152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.588182+0000","last_change":"2024-02-14T15:00:48.276884+0000","last_active":"2024-02-14T15:01:14.588182+0000","last_peered":"2024-02-14T15:01:14.588182+0000","last_clean":"2024-02-14T15:01:14.588182+0000","last_became_active":"2024-02-14T15:00:48.276576+0000","last_became_peered":"2024-02-14T15:00:48.276576+0000","last_unstale":"2024-02-14T15:01:14.588182+0000","last_undegraded":"2024-02-14T15:01:14.588182+0000","last_fullsized":"2024-02-14T15:01:14.588182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:25.793368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670022+0000","last_change":"2024-02-14T15:00:49.277602+0000","last_active":"2024-02-14T15:00:54.670022+0000","last_peered":"2024-02-14T15:00:54.670022+0000","last_clean":"2024-02-14T15:00:54.670022+0000","last_became_active":"2024-02-14T15:00:49.277249+0000","last_became_peered":"2024-02-14T15:00:49.277249+0000","last_unstale":"2024-02-14T15:00:54.670022+0000","last_undegraded":"2024-02-14T15:00:54.670022+0000","last_fullsized":"2024-02-14T15:00:54.670022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.347962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670918+0000","last_change":"2024-02-14T15:00:53.587838+0000","last_active":"2024-02-14T15:00:54.670918+0000","last_peered":"2024-02-14T15:00:54.670918+0000","last_clean":"2024-02-14T15:00:54.670918+0000","last_became_active":"2024-02-14T15:00:53.587012+0000","last_became_peered":"2024-02-14T15:00:53.587012+0000","last_unstale":"2024-02-14T15:00:54.670918+0000","last_undegraded":"2024-02-14T15:00:54.670918+0000","last_fullsized":"2024-02-14T15:00:54.670918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:32:12.276578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670317+0000","last_change":"2024-02-14T15:00:48.273529+0000","last_active":"2024-02-14T15:00:54.670317+0000","last_peered":"2024-02-14T15:00:54.670317+0000","last_clean":"2024-02-14T15:00:54.670317+0000","last_became_active":"2024-02-14T15:00:48.272555+0000","last_became_peered":"2024-02-14T15:00:48.272555+0000","last_unstale":"2024-02-14T15:00:54.670317+0000","last_undegraded":"2024-02-14T15:00:54.670317+0000","last_fullsized":"2024-02-14T15:00:54.670317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:19.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670995+0000","last_change":"2024-02-14T15:00:49.278834+0000","last_active":"2024-02-14T15:00:54.670995+0000","last_peered":"2024-02-14T15:00:54.670995+0000","last_clean":"2024-02-14T15:00:54.670995+0000","last_became_active":"2024-02-14T15:00:49.277095+0000","last_became_peered":"2024-02-14T15:00:49.277095+0000","last_unstale":"2024-02-14T15:00:54.670995+0000","last_undegraded":"2024-02-14T15:00:54.670995+0000","last_fullsized":"2024-02-14T15:00:54.670995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:27.291974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566954+0000","last_change":"2024-02-14T15:00:53.577162+0000","last_active":"2024-02-14T15:00:54.566954+0000","last_peered":"2024-02-14T15:00:54.566954+0000","last_clean":"2024-02-14T15:00:54.566954+0000","last_became_active":"2024-02-14T15:00:53.576809+0000","last_became_peered":"2024-02-14T15:00:53.576809+0000","last_unstale":"2024-02-14T15:00:54.566954+0000","last_undegraded":"2024-02-14T15:00:54.566954+0000","last_fullsized":"2024-02-14T15:00:54.566954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:37.698842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.584429+0000","last_change":"2024-02-14T15:00:48.276146+0000","last_active":"2024-02-14T15:01:13.584429+0000","last_peered":"2024-02-14T15:01:13.584429+0000","last_clean":"2024-02-14T15:01:13.584429+0000","last_became_active":"2024-02-14T15:00:48.275833+0000","last_became_peered":"2024-02-14T15:00:48.275833+0000","last_unstale":"2024-02-14T15:01:13.584429+0000","last_undegraded":"2024-02-14T15:01:13.584429+0000","last_fullsized":"2024-02-14T15:01:13.584429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:46.649486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669866+0000","last_change":"2024-02-14T15:00:49.267285+0000","last_active":"2024-02-14T15:00:54.669866+0000","last_peered":"2024-02-14T15:00:54.669866+0000","last_clean":"2024-02-14T15:00:54.669866+0000","last_became_active":"2024-02-14T15:00:49.265592+0000","last_became_peered":"2024-02-14T15:00:49.265592+0000","last_unstale":"2024-02-14T15:00:54.669866+0000","last_undegraded":"2024-02-14T15:00:54.669866+0000","last_fullsized":"2024-02-14T15:00:54.669866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:18.469835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566800+0000","last_change":"2024-02-14T15:00:53.582085+0000","last_active":"2024-02-14T15:00:54.566800+0000","last_peered":"2024-02-14T15:00:54.566800+0000","last_clean":"2024-02-14T15:00:54.566800+0000","last_became_active":"2024-02-14T15:00:53.581706+0000","last_became_peered":"2024-02-14T15:00:53.581706+0000","last_unstale":"2024-02-14T15:00:54.566800+0000","last_undegraded":"2024-02-14T15:00:54.566800+0000","last_fullsized":"2024-02-14T15:00:54.566800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:32.481142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.589838+0000","last_change":"2024-02-14T15:00:48.268975+0000","last_active":"2024-02-14T15:01:14.589838+0000","last_peered":"2024-02-14T15:01:14.589838+0000","last_clean":"2024-02-14T15:01:14.589838+0000","last_became_active":"2024-02-14T15:00:48.268547+0000","last_became_peered":"2024-02-14T15:00:48.268547+0000","last_unstale":"2024-02-14T15:01:14.589838+0000","last_undegraded":"2024-02-14T15:01:14.589838+0000","last_fullsized":"2024-02-14T15:01:14.589838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:13.937917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670470+0000","last_change":"2024-02-14T15:00:49.276909+0000","last_active":"2024-02-14T15:00:54.670470+0000","last_peered":"2024-02-14T15:00:54.670470+0000","last_clean":"2024-02-14T15:00:54.670470+0000","last_became_active":"2024-02-14T15:00:49.276533+0000","last_became_peered":"2024-02-14T15:00:49.276533+0000","last_unstale":"2024-02-14T15:00:54.670470+0000","last_undegraded":"2024-02-14T15:00:54.670470+0000","last_fullsized":"2024-02-14T15:00:54.670470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:35:31.321455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670402+0000","last_change":"2024-02-14T15:00:53.592223+0000","last_active":"2024-02-14T15:00:54.670402+0000","last_peered":"2024-02-14T15:00:54.670402+0000","last_clean":"2024-02-14T15:00:54.670402+0000","last_became_active":"2024-02-14T15:00:53.591883+0000","last_became_peered":"2024-02-14T15:00:53.591883+0000","last_unstale":"2024-02-14T15:00:54.670402+0000","last_undegraded":"2024-02-14T15:00:54.670402+0000","last_fullsized":"2024-02-14T15:00:54.670402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:10.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.590374+0000","last_change":"2024-02-14T15:00:48.274485+0000","last_active":"2024-02-14T15:01:14.590374+0000","last_peered":"2024-02-14T15:01:14.590374+0000","last_clean":"2024-02-14T15:01:14.590374+0000","last_became_active":"2024-02-14T15:00:48.274160+0000","last_became_peered":"2024-02-14T15:00:48.274160+0000","last_unstale":"2024-02-14T15:01:14.590374+0000","last_undegraded":"2024-02-14T15:01:14.590374+0000","last_fullsized":"2024-02-14T15:01:14.590374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:00:47.105069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566302+0000","last_change":"2024-02-14T15:00:49.272577+0000","last_active":"2024-02-14T15:00:54.566302+0000","last_peered":"2024-02-14T15:00:54.566302+0000","last_clean":"2024-02-14T15:00:54.566302+0000","last_became_active":"2024-02-14T15:00:49.271670+0000","last_became_peered":"2024-02-14T15:00:49.271670+0000","last_unstale":"2024-02-14T15:00:54.566302+0000","last_undegraded":"2024-02-14T15:00:54.566302+0000","last_fullsized":"2024-02-14T15:00:54.566302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:19.968389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569872+0000","last_change":"2024-02-14T15:00:53.581212+0000","last_active":"2024-02-14T15:00:54.569872+0000","last_peered":"2024-02-14T15:00:54.569872+0000","last_clean":"2024-02-14T15:00:54.569872+0000","last_became_active":"2024-02-14T15:00:53.580782+0000","last_became_peered":"2024-02-14T15:00:53.580782+0000","last_unstale":"2024-02-14T15:00:54.569872+0000","last_undegraded":"2024-02-14T15:00:54.569872+0000","last_fullsized":"2024-02-14T15:00:54.569872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:14.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669988+0000","last_change":"2024-02-14T15:00:48.273572+0000","last_active":"2024-02-14T15:00:54.669988+0000","last_peered":"2024-02-14T15:00:54.669988+0000","last_clean":"2024-02-14T15:00:54.669988+0000","last_became_active":"2024-02-14T15:00:48.273138+0000","last_became_peered":"2024-02-14T15:00:48.273138+0000","last_unstale":"2024-02-14T15:00:54.669988+0000","last_undegraded":"2024-02-14T15:00:54.669988+0000","last_fullsized":"2024-02-14T15:00:54.669988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:04.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670295+0000","last_change":"2024-02-14T15:00:49.288226+0000","last_active":"2024-02-14T15:00:54.670295+0000","last_peered":"2024-02-14T15:00:54.670295+0000","last_clean":"2024-02-14T15:00:54.670295+0000","last_became_active":"2024-02-14T15:00:49.287874+0000","last_became_peered":"2024-02-14T15:00:49.287874+0000","last_unstale":"2024-02-14T15:00:54.670295+0000","last_undegraded":"2024-02-14T15:00:54.670295+0000","last_fullsized":"2024-02-14T15:00:54.670295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:24:02.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566490+0000","last_change":"2024-02-14T15:00:53.590543+0000","last_active":"2024-02-14T15:00:54.566490+0000","last_peered":"2024-02-14T15:00:54.566490+0000","last_clean":"2024-02-14T15:00:54.566490+0000","last_became_active":"2024-02-14T15:00:53.589072+0000","last_became_peered":"2024-02-14T15:00:53.589072+0000","last_unstale":"2024-02-14T15:00:54.566490+0000","last_undegraded":"2024-02-14T15:00:54.566490+0000","last_fullsized":"2024-02-14T15:00:54.566490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:13.685655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:17.565151+0000","last_change":"2024-02-14T15:00:48.272027+0000","last_active":"2024-02-14T15:01:17.565151+0000","last_peered":"2024-02-14T15:01:17.565151+0000","last_clean":"2024-02-14T15:01:17.565151+0000","last_became_active":"2024-02-14T15:00:48.271587+0000","last_became_peered":"2024-02-14T15:00:48.271587+0000","last_unstale":"2024-02-14T15:01:17.565151+0000","last_undegraded":"2024-02-14T15:01:17.565151+0000","last_fullsized":"2024-02-14T15:01:17.565151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:33:03.566189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669727+0000","last_change":"2024-02-14T15:00:49.278936+0000","last_active":"2024-02-14T15:00:54.669727+0000","last_peered":"2024-02-14T15:00:54.669727+0000","last_clean":"2024-02-14T15:00:54.669727+0000","last_became_active":"2024-02-14T15:00:49.278318+0000","last_became_peered":"2024-02-14T15:00:49.278318+0000","last_unstale":"2024-02-14T15:00:54.669727+0000","last_undegraded":"2024-02-14T15:00:54.669727+0000","last_fullsized":"2024-02-14T15:00:54.669727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:01.327303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669815+0000","last_change":"2024-02-14T15:00:53.580340+0000","last_active":"2024-02-14T15:00:54.669815+0000","last_peered":"2024-02-14T15:00:54.669815+0000","last_clean":"2024-02-14T15:00:54.669815+0000","last_became_active":"2024-02-14T15:00:53.579742+0000","last_became_peered":"2024-02-14T15:00:53.579742+0000","last_unstale":"2024-02-14T15:00:54.669815+0000","last_undegraded":"2024-02-14T15:00:54.669815+0000","last_fullsized":"2024-02-14T15:00:54.669815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:38:33.182328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583025+0000","last_change":"2024-02-14T15:00:48.271891+0000","last_active":"2024-02-14T15:01:13.583025+0000","last_peered":"2024-02-14T15:01:13.583025+0000","last_clean":"2024-02-14T15:01:13.583025+0000","last_became_active":"2024-02-14T15:00:48.271419+0000","last_became_peered":"2024-02-14T15:00:48.271419+0000","last_unstale":"2024-02-14T15:01:13.583025+0000","last_undegraded":"2024-02-14T15:01:13.583025+0000","last_fullsized":"2024-02-14T15:01:13.583025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:03:04.725647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569792+0000","last_change":"2024-02-14T15:00:49.277025+0000","last_active":"2024-02-14T15:00:54.569792+0000","last_peered":"2024-02-14T15:00:54.569792+0000","last_clean":"2024-02-14T15:00:54.569792+0000","last_became_active":"2024-02-14T15:00:49.276694+0000","last_became_peered":"2024-02-14T15:00:49.276694+0000","last_unstale":"2024-02-14T15:00:54.569792+0000","last_undegraded":"2024-02-14T15:00:54.569792+0000","last_fullsized":"2024-02-14T15:00:54.569792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:43:07.249407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669532+0000","last_change":"2024-02-14T15:00:53.588208+0000","last_active":"2024-02-14T15:00:54.669532+0000","last_peered":"2024-02-14T15:00:54.669532+0000","last_clean":"2024-02-14T15:00:54.669532+0000","last_became_active":"2024-02-14T15:00:53.587153+0000","last_became_peered":"2024-02-14T15:00:53.587153+0000","last_unstale":"2024-02-14T15:00:54.669532+0000","last_undegraded":"2024-02-14T15:00:54.669532+0000","last_fullsized":"2024-02-14T15:00:54.669532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:14:02.707991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670939+0000","last_change":"2024-02-14T15:00:48.272866+0000","last_active":"2024-02-14T15:00:54.670939+0000","last_peered":"2024-02-14T15:00:54.670939+0000","last_clean":"2024-02-14T15:00:54.670939+0000","last_became_active":"2024-02-14T15:00:48.272520+0000","last_became_peered":"2024-02-14T15:00:48.272520+0000","last_unstale":"2024-02-14T15:00:54.670939+0000","last_undegraded":"2024-02-14T15:00:54.670939+0000","last_fullsized":"2024-02-14T15:00:54.670939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:39.434577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670182+0000","last_change":"2024-02-14T15:00:49.280994+0000","last_active":"2024-02-14T15:00:54.670182+0000","last_peered":"2024-02-14T15:00:54.670182+0000","last_clean":"2024-02-14T15:00:54.670182+0000","last_became_active":"2024-02-14T15:00:49.280205+0000","last_became_peered":"2024-02-14T15:00:49.280205+0000","last_unstale":"2024-02-14T15:00:54.670182+0000","last_undegraded":"2024-02-14T15:00:54.670182+0000","last_fullsized":"2024-02-14T15:00:54.670182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:53:48.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670214+0000","last_change":"2024-02-14T15:00:53.582599+0000","last_active":"2024-02-14T15:00:54.670214+0000","last_peered":"2024-02-14T15:00:54.670214+0000","last_clean":"2024-02-14T15:00:54.670214+0000","last_became_active":"2024-02-14T15:00:53.582231+0000","last_became_peered":"2024-02-14T15:00:53.582231+0000","last_unstale":"2024-02-14T15:00:54.670214+0000","last_undegraded":"2024-02-14T15:00:54.670214+0000","last_fullsized":"2024-02-14T15:00:54.670214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:19.126264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'24","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:18.570186+0000","last_change":"2024-02-14T15:00:48.274178+0000","last_active":"2024-02-14T15:01:18.570186+0000","last_peered":"2024-02-14T15:01:18.570186+0000","last_clean":"2024-02-14T15:01:18.570186+0000","last_became_active":"2024-02-14T15:00:48.273533+0000","last_became_peered":"2024-02-14T15:00:48.273533+0000","last_unstale":"2024-02-14T15:01:18.570186+0000","last_undegraded":"2024-02-14T15:01:18.570186+0000","last_fullsized":"2024-02-14T15:01:18.570186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:33:08.621601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45671,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":51,"num_write":25,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670865+0000","last_change":"2024-02-14T15:00:49.270144+0000","last_active":"2024-02-14T15:00:54.670865+0000","last_peered":"2024-02-14T15:00:54.670865+0000","last_clean":"2024-02-14T15:00:54.670865+0000","last_became_active":"2024-02-14T15:00:49.269806+0000","last_became_peered":"2024-02-14T15:00:49.269806+0000","last_unstale":"2024-02-14T15:00:54.670865+0000","last_undegraded":"2024-02-14T15:00:54.670865+0000","last_fullsized":"2024-02-14T15:00:54.670865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:42:18.559226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.671366+0000","last_change":"2024-02-14T15:00:53.582755+0000","last_active":"2024-02-14T15:00:54.671366+0000","last_peered":"2024-02-14T15:00:54.671366+0000","last_clean":"2024-02-14T15:00:54.671366+0000","last_became_active":"2024-02-14T15:00:53.582375+0000","last_became_peered":"2024-02-14T15:00:53.582375+0000","last_unstale":"2024-02-14T15:00:54.671366+0000","last_undegraded":"2024-02-14T15:00:54.671366+0000","last_fullsized":"2024-02-14T15:00:54.671366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:11:49.115136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583242+0000","last_change":"2024-02-14T15:00:48.268982+0000","last_active":"2024-02-14T15:01:13.583242+0000","last_peered":"2024-02-14T15:01:13.583242+0000","last_clean":"2024-02-14T15:01:13.583242+0000","last_became_active":"2024-02-14T15:00:48.267137+0000","last_became_peered":"2024-02-14T15:00:48.267137+0000","last_unstale":"2024-02-14T15:01:13.583242+0000","last_undegraded":"2024-02-14T15:01:13.583242+0000","last_fullsized":"2024-02-14T15:01:13.583242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:44.351561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670062+0000","last_change":"2024-02-14T15:00:49.267493+0000","last_active":"2024-02-14T15:00:54.670062+0000","last_peered":"2024-02-14T15:00:54.670062+0000","last_clean":"2024-02-14T15:00:54.670062+0000","last_became_active":"2024-02-14T15:00:49.267161+0000","last_became_peered":"2024-02-14T15:00:49.267161+0000","last_unstale":"2024-02-14T15:00:54.670062+0000","last_undegraded":"2024-02-14T15:00:54.670062+0000","last_fullsized":"2024-02-14T15:00:54.670062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:35:22.997999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670040+0000","last_change":"2024-02-14T15:00:53.591201+0000","last_active":"2024-02-14T15:00:54.670040+0000","last_peered":"2024-02-14T15:00:54.670040+0000","last_clean":"2024-02-14T15:00:54.670040+0000","last_became_active":"2024-02-14T15:00:53.590769+0000","last_became_peered":"2024-02-14T15:00:53.590769+0000","last_unstale":"2024-02-14T15:00:54.670040+0000","last_undegraded":"2024-02-14T15:00:54.670040+0000","last_fullsized":"2024-02-14T15:00:54.670040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.881271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583767+0000","last_change":"2024-02-14T15:00:48.282085+0000","last_active":"2024-02-14T15:01:13.583767+0000","last_peered":"2024-02-14T15:01:13.583767+0000","last_clean":"2024-02-14T15:01:13.583767+0000","last_became_active":"2024-02-14T15:00:48.281836+0000","last_became_peered":"2024-02-14T15:00:48.281836+0000","last_unstale":"2024-02-14T15:01:13.583767+0000","last_undegraded":"2024-02-14T15:01:13.583767+0000","last_fullsized":"2024-02-14T15:01:13.583767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:07:28.145074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566527+0000","last_change":"2024-02-14T15:00:49.280013+0000","last_active":"2024-02-14T15:00:54.566527+0000","last_peered":"2024-02-14T15:00:54.566527+0000","last_clean":"2024-02-14T15:00:54.566527+0000","last_became_active":"2024-02-14T15:00:49.279667+0000","last_became_peered":"2024-02-14T15:00:49.279667+0000","last_unstale":"2024-02-14T15:00:54.566527+0000","last_undegraded":"2024-02-14T15:00:54.566527+0000","last_fullsized":"2024-02-14T15:00:54.566527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:03:16.559295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669511+0000","last_change":"2024-02-14T15:00:53.580269+0000","last_active":"2024-02-14T15:00:54.669511+0000","last_peered":"2024-02-14T15:00:54.669511+0000","last_clean":"2024-02-14T15:00:54.669511+0000","last_became_active":"2024-02-14T15:00:53.579863+0000","last_became_peered":"2024-02-14T15:00:53.579863+0000","last_unstale":"2024-02-14T15:00:54.669511+0000","last_undegraded":"2024-02-14T15:00:54.669511+0000","last_fullsized":"2024-02-14T15:00:54.669511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:37:19.275654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.588316+0000","last_change":"2024-02-14T15:00:48.267310+0000","last_active":"2024-02-14T15:01:14.588316+0000","last_peered":"2024-02-14T15:01:14.588316+0000","last_clean":"2024-02-14T15:01:14.588316+0000","last_became_active":"2024-02-14T15:00:48.266864+0000","last_became_peered":"2024-02-14T15:00:48.266864+0000","last_unstale":"2024-02-14T15:01:14.588316+0000","last_undegraded":"2024-02-14T15:01:14.588316+0000","last_fullsized":"2024-02-14T15:01:14.588316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:07.703549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566387+0000","last_change":"2024-02-14T15:00:49.288568+0000","last_active":"2024-02-14T15:00:54.566387+0000","last_peered":"2024-02-14T15:00:54.566387+0000","last_clean":"2024-02-14T15:00:54.566387+0000","last_became_active":"2024-02-14T15:00:49.288222+0000","last_became_peered":"2024-02-14T15:00:49.288222+0000","last_unstale":"2024-02-14T15:00:54.566387+0000","last_undegraded":"2024-02-14T15:00:54.566387+0000","last_fullsized":"2024-02-14T15:00:54.566387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:38:38.909769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669939+0000","last_change":"2024-02-14T15:00:53.585673+0000","last_active":"2024-02-14T15:00:54.669939+0000","last_peered":"2024-02-14T15:00:54.669939+0000","last_clean":"2024-02-14T15:00:54.669939+0000","last_became_active":"2024-02-14T15:00:53.585238+0000","last_became_peered":"2024-02-14T15:00:53.585238+0000","last_unstale":"2024-02-14T15:00:54.669939+0000","last_undegraded":"2024-02-14T15:00:54.669939+0000","last_fullsized":"2024-02-14T15:00:54.669939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:48:11.845921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.590807+0000","last_change":"2024-02-14T15:00:48.275833+0000","last_active":"2024-02-14T15:01:14.590807+0000","last_peered":"2024-02-14T15:01:14.590807+0000","last_clean":"2024-02-14T15:01:14.590807+0000","last_became_active":"2024-02-14T15:00:48.275499+0000","last_became_peered":"2024-02-14T15:00:48.275499+0000","last_unstale":"2024-02-14T15:01:14.590807+0000","last_undegraded":"2024-02-14T15:01:14.590807+0000","last_fullsized":"2024-02-14T15:01:14.590807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:35:10.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670195+0000","last_change":"2024-02-14T15:00:49.276231+0000","last_active":"2024-02-14T15:00:54.670195+0000","last_peered":"2024-02-14T15:00:54.670195+0000","last_clean":"2024-02-14T15:00:54.670195+0000","last_became_active":"2024-02-14T15:00:49.275678+0000","last_became_peered":"2024-02-14T15:00:49.275678+0000","last_unstale":"2024-02-14T15:00:54.670195+0000","last_undegraded":"2024-02-14T15:00:54.670195+0000","last_fullsized":"2024-02-14T15:00:54.670195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:53:46.402577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669981+0000","last_change":"2024-02-14T15:00:53.586406+0000","last_active":"2024-02-14T15:00:54.669981+0000","last_peered":"2024-02-14T15:00:54.669981+0000","last_clean":"2024-02-14T15:00:54.669981+0000","last_became_active":"2024-02-14T15:00:53.586034+0000","last_became_peered":"2024-02-14T15:00:53.586034+0000","last_unstale":"2024-02-14T15:00:54.669981+0000","last_undegraded":"2024-02-14T15:00:54.669981+0000","last_fullsized":"2024-02-14T15:00:54.669981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:46.886154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670762+0000","last_change":"2024-02-14T15:00:48.274305+0000","last_active":"2024-02-14T15:00:54.670762+0000","last_peered":"2024-02-14T15:00:54.670762+0000","last_clean":"2024-02-14T15:00:54.670762+0000","last_became_active":"2024-02-14T15:00:48.273966+0000","last_became_peered":"2024-02-14T15:00:48.273966+0000","last_unstale":"2024-02-14T15:00:54.670762+0000","last_undegraded":"2024-02-14T15:00:54.670762+0000","last_fullsized":"2024-02-14T15:00:54.670762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:49.198099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670637+0000","last_change":"2024-02-14T15:00:49.278473+0000","last_active":"2024-02-14T15:00:54.670637+0000","last_peered":"2024-02-14T15:00:54.670637+0000","last_clean":"2024-02-14T15:00:54.670637+0000","last_became_active":"2024-02-14T15:00:49.278145+0000","last_became_peered":"2024-02-14T15:00:49.278145+0000","last_unstale":"2024-02-14T15:00:54.670637+0000","last_undegraded":"2024-02-14T15:00:54.670637+0000","last_fullsized":"2024-02-14T15:00:54.670637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:49:11.718035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669811+0000","last_change":"2024-02-14T15:00:53.581164+0000","last_active":"2024-02-14T15:00:54.669811+0000","last_peered":"2024-02-14T15:00:54.669811+0000","last_clean":"2024-02-14T15:00:54.669811+0000","last_became_active":"2024-02-14T15:00:53.580796+0000","last_became_peered":"2024-02-14T15:00:53.580796+0000","last_unstale":"2024-02-14T15:00:54.669811+0000","last_undegraded":"2024-02-14T15:00:54.669811+0000","last_fullsized":"2024-02-14T15:00:54.669811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:37.245035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670142+0000","last_change":"2024-02-14T15:00:48.271427+0000","last_active":"2024-02-14T15:00:54.670142+0000","last_peered":"2024-02-14T15:00:54.670142+0000","last_clean":"2024-02-14T15:00:54.670142+0000","last_became_active":"2024-02-14T15:00:48.271115+0000","last_became_peered":"2024-02-14T15:00:48.271115+0000","last_unstale":"2024-02-14T15:00:54.670142+0000","last_undegraded":"2024-02-14T15:00:54.670142+0000","last_fullsized":"2024-02-14T15:00:54.670142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:56:01.008058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670429+0000","last_change":"2024-02-14T15:00:49.269964+0000","last_active":"2024-02-14T15:00:54.670429+0000","last_peered":"2024-02-14T15:00:54.670429+0000","last_clean":"2024-02-14T15:00:54.670429+0000","last_became_active":"2024-02-14T15:00:49.269628+0000","last_became_peered":"2024-02-14T15:00:49.269628+0000","last_unstale":"2024-02-14T15:00:54.670429+0000","last_undegraded":"2024-02-14T15:00:54.670429+0000","last_fullsized":"2024-02-14T15:00:54.670429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:04:26.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670615+0000","last_change":"2024-02-14T15:00:53.579843+0000","last_active":"2024-02-14T15:00:54.670615+0000","last_peered":"2024-02-14T15:00:54.670615+0000","last_clean":"2024-02-14T15:00:54.670615+0000","last_became_active":"2024-02-14T15:00:53.579457+0000","last_became_peered":"2024-02-14T15:00:53.579457+0000","last_unstale":"2024-02-14T15:00:54.670615+0000","last_undegraded":"2024-02-14T15:00:54.670615+0000","last_fullsized":"2024-02-14T15:00:54.670615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:34:49.438317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566622+0000","last_change":"2024-02-14T15:00:48.275016+0000","last_active":"2024-02-14T15:00:54.566622+0000","last_peered":"2024-02-14T15:00:54.566622+0000","last_clean":"2024-02-14T15:00:54.566622+0000","last_became_active":"2024-02-14T15:00:48.274636+0000","last_became_peered":"2024-02-14T15:00:48.274636+0000","last_unstale":"2024-02-14T15:00:54.566622+0000","last_undegraded":"2024-02-14T15:00:54.566622+0000","last_fullsized":"2024-02-14T15:00:54.566622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:12:12.961276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670061+0000","last_change":"2024-02-14T15:00:49.292025+0000","last_active":"2024-02-14T15:00:54.670061+0000","last_peered":"2024-02-14T15:00:54.670061+0000","last_clean":"2024-02-14T15:00:54.670061+0000","last_became_active":"2024-02-14T15:00:49.291130+0000","last_became_peered":"2024-02-14T15:00:49.291130+0000","last_unstale":"2024-02-14T15:00:54.670061+0000","last_undegraded":"2024-02-14T15:00:54.670061+0000","last_fullsized":"2024-02-14T15:00:54.670061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:37.150827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669702+0000","last_change":"2024-02-14T15:00:53.584068+0000","last_active":"2024-02-14T15:00:54.669702+0000","last_peered":"2024-02-14T15:00:54.669702+0000","last_clean":"2024-02-14T15:00:54.669702+0000","last_became_active":"2024-02-14T15:00:53.583263+0000","last_became_peered":"2024-02-14T15:00:53.583263+0000","last_unstale":"2024-02-14T15:00:54.669702+0000","last_undegraded":"2024-02-14T15:00:54.669702+0000","last_fullsized":"2024-02-14T15:00:54.669702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:15.726442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670125+0000","last_change":"2024-02-14T15:00:49.271265+0000","last_active":"2024-02-14T15:00:54.670125+0000","last_peered":"2024-02-14T15:00:54.670125+0000","last_clean":"2024-02-14T15:00:54.670125+0000","last_became_active":"2024-02-14T15:00:49.270931+0000","last_became_peered":"2024-02-14T15:00:49.270931+0000","last_unstale":"2024-02-14T15:00:54.670125+0000","last_undegraded":"2024-02-14T15:00:54.670125+0000","last_fullsized":"2024-02-14T15:00:54.670125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:15:40.570730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.590133+0000","last_change":"2024-02-14T15:00:48.286669+0000","last_active":"2024-02-14T15:01:14.590133+0000","last_peered":"2024-02-14T15:01:14.590133+0000","last_clean":"2024-02-14T15:01:14.590133+0000","last_became_active":"2024-02-14T15:00:48.286252+0000","last_became_peered":"2024-02-14T15:00:48.286252+0000","last_unstale":"2024-02-14T15:01:14.590133+0000","last_undegraded":"2024-02-14T15:01:14.590133+0000","last_fullsized":"2024-02-14T15:01:14.590133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:40.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566567+0000","last_change":"2024-02-14T15:00:53.583353+0000","last_active":"2024-02-14T15:00:54.566567+0000","last_peered":"2024-02-14T15:00:54.566567+0000","last_clean":"2024-02-14T15:00:54.566567+0000","last_became_active":"2024-02-14T15:00:53.582449+0000","last_became_peered":"2024-02-14T15:00:53.582449+0000","last_unstale":"2024-02-14T15:00:54.566567+0000","last_undegraded":"2024-02-14T15:00:54.566567+0000","last_fullsized":"2024-02-14T15:00:54.566567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:09:01.514603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566701+0000","last_change":"2024-02-14T15:00:49.285579+0000","last_active":"2024-02-14T15:00:54.566701+0000","last_peered":"2024-02-14T15:00:54.566701+0000","last_clean":"2024-02-14T15:00:54.566701+0000","last_became_active":"2024-02-14T15:00:49.283741+0000","last_became_peered":"2024-02-14T15:00:49.283741+0000","last_unstale":"2024-02-14T15:00:54.566701+0000","last_undegraded":"2024-02-14T15:00:54.566701+0000","last_fullsized":"2024-02-14T15:00:54.566701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:05.813218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.582185+0000","last_change":"2024-02-14T15:00:48.264434+0000","last_active":"2024-02-14T15:01:13.582185+0000","last_peered":"2024-02-14T15:01:13.582185+0000","last_clean":"2024-02-14T15:01:13.582185+0000","last_became_active":"2024-02-14T15:00:48.264048+0000","last_became_peered":"2024-02-14T15:00:48.264048+0000","last_unstale":"2024-02-14T15:01:13.582185+0000","last_undegraded":"2024-02-14T15:01:13.582185+0000","last_fullsized":"2024-02-14T15:01:13.582185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:52:32.498128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566155+0000","last_change":"2024-02-14T14:59:36.539524+0000","last_active":"2024-02-14T15:00:54.566155+0000","last_peered":"2024-02-14T15:00:54.566155+0000","last_clean":"2024-02-14T15:00:54.566155+0000","last_became_active":"2024-02-14T14:59:36.538922+0000","last_became_peered":"2024-02-14T14:59:36.538922+0000","last_unstale":"2024-02-14T15:00:54.566155+0000","last_undegraded":"2024-02-14T15:00:54.566155+0000","last_fullsized":"2024-02-14T15:00:54.566155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_clean_scrub_stamp":"2024-02-14T14:56:09.567049+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:58.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670878+0000","last_change":"2024-02-14T15:00:53.585747+0000","last_active":"2024-02-14T15:00:54.670878+0000","last_peered":"2024-02-14T15:00:54.670878+0000","last_clean":"2024-02-14T15:00:54.670878+0000","last_became_active":"2024-02-14T15:00:53.583817+0000","last_became_peered":"2024-02-14T15:00:53.583817+0000","last_unstale":"2024-02-14T15:00:54.670878+0000","last_undegraded":"2024-02-14T15:00:54.670878+0000","last_fullsized":"2024-02-14T15:00:54.670878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:56:35.081078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669181+0000","last_change":"2024-02-14T15:00:49.272692+0000","last_active":"2024-02-14T15:00:54.669181+0000","last_peered":"2024-02-14T15:00:54.669181+0000","last_clean":"2024-02-14T15:00:54.669181+0000","last_became_active":"2024-02-14T15:00:49.272353+0000","last_became_peered":"2024-02-14T15:00:49.272353+0000","last_unstale":"2024-02-14T15:00:54.669181+0000","last_undegraded":"2024-02-14T15:00:54.669181+0000","last_fullsized":"2024-02-14T15:00:54.669181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:47.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:14.588917+0000","last_change":"2024-02-14T15:00:48.272470+0000","last_active":"2024-02-14T15:01:14.588917+0000","last_peered":"2024-02-14T15:01:14.588917+0000","last_clean":"2024-02-14T15:01:14.588917+0000","last_became_active":"2024-02-14T15:00:48.272126+0000","last_became_peered":"2024-02-14T15:00:48.272126+0000","last_unstale":"2024-02-14T15:01:14.588917+0000","last_undegraded":"2024-02-14T15:01:14.588917+0000","last_fullsized":"2024-02-14T15:01:14.588917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:21:02.601411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669473+0000","last_change":"2024-02-14T15:00:53.585950+0000","last_active":"2024-02-14T15:00:54.669473+0000","last_peered":"2024-02-14T15:00:54.669473+0000","last_clean":"2024-02-14T15:00:54.669473+0000","last_became_active":"2024-02-14T15:00:53.585547+0000","last_became_peered":"2024-02-14T15:00:53.585547+0000","last_unstale":"2024-02-14T15:00:54.669473+0000","last_undegraded":"2024-02-14T15:00:54.669473+0000","last_fullsized":"2024-02-14T15:00:54.669473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:22.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583167+0000","last_change":"2024-02-14T15:00:48.274650+0000","last_active":"2024-02-14T15:01:13.583167+0000","last_peered":"2024-02-14T15:01:13.583167+0000","last_clean":"2024-02-14T15:01:13.583167+0000","last_became_active":"2024-02-14T15:00:48.274085+0000","last_became_peered":"2024-02-14T15:00:48.274085+0000","last_unstale":"2024-02-14T15:01:13.583167+0000","last_undegraded":"2024-02-14T15:01:13.583167+0000","last_fullsized":"2024-02-14T15:01:13.583167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:53.307596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670651+0000","last_change":"2024-02-14T15:00:49.268419+0000","last_active":"2024-02-14T15:00:54.670651+0000","last_peered":"2024-02-14T15:00:54.670651+0000","last_clean":"2024-02-14T15:00:54.670651+0000","last_became_active":"2024-02-14T15:00:49.268120+0000","last_became_peered":"2024-02-14T15:00:49.268120+0000","last_unstale":"2024-02-14T15:00:54.670651+0000","last_undegraded":"2024-02-14T15:00:54.670651+0000","last_fullsized":"2024-02-14T15:00:54.670651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:10:36.319337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566309+0000","last_change":"2024-02-14T15:00:53.586151+0000","last_active":"2024-02-14T15:00:54.566309+0000","last_peered":"2024-02-14T15:00:54.566309+0000","last_clean":"2024-02-14T15:00:54.566309+0000","last_became_active":"2024-02-14T15:00:53.585727+0000","last_became_peered":"2024-02-14T15:00:53.585727+0000","last_unstale":"2024-02-14T15:00:54.566309+0000","last_undegraded":"2024-02-14T15:00:54.566309+0000","last_fullsized":"2024-02-14T15:00:54.566309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:40:03.741063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:16.277272+0000","last_change":"2024-02-14T15:00:48.269837+0000","last_active":"2024-02-14T15:01:16.277272+0000","last_peered":"2024-02-14T15:01:16.277272+0000","last_clean":"2024-02-14T15:01:16.277272+0000","last_became_active":"2024-02-14T15:00:48.268699+0000","last_became_peered":"2024-02-14T15:00:48.268699+0000","last_unstale":"2024-02-14T15:01:16.277272+0000","last_undegraded":"2024-02-14T15:01:16.277272+0000","last_fullsized":"2024-02-14T15:01:16.277272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:55:03.013888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566821+0000","last_change":"2024-02-14T15:00:49.285726+0000","last_active":"2024-02-14T15:00:54.566821+0000","last_peered":"2024-02-14T15:00:54.566821+0000","last_clean":"2024-02-14T15:00:54.566821+0000","last_became_active":"2024-02-14T15:00:49.283728+0000","last_became_peered":"2024-02-14T15:00:49.283728+0000","last_unstale":"2024-02-14T15:00:54.566821+0000","last_undegraded":"2024-02-14T15:00:54.566821+0000","last_fullsized":"2024-02-14T15:00:54.566821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.764048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566421+0000","last_change":"2024-02-14T15:00:53.583644+0000","last_active":"2024-02-14T15:00:54.566421+0000","last_peered":"2024-02-14T15:00:54.566421+0000","last_clean":"2024-02-14T15:00:54.566421+0000","last_became_active":"2024-02-14T15:00:53.582801+0000","last_became_peered":"2024-02-14T15:00:53.582801+0000","last_unstale":"2024-02-14T15:00:54.566421+0000","last_undegraded":"2024-02-14T15:00:54.566421+0000","last_fullsized":"2024-02-14T15:00:54.566421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:41:40.622091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.584078+0000","last_change":"2024-02-14T15:00:48.270231+0000","last_active":"2024-02-14T15:01:13.584078+0000","last_peered":"2024-02-14T15:01:13.584078+0000","last_clean":"2024-02-14T15:01:13.584078+0000","last_became_active":"2024-02-14T15:00:48.269880+0000","last_became_peered":"2024-02-14T15:00:48.269880+0000","last_unstale":"2024-02-14T15:01:13.584078+0000","last_undegraded":"2024-02-14T15:01:13.584078+0000","last_fullsized":"2024-02-14T15:01:13.584078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:08:47.376417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569865+0000","last_change":"2024-02-14T15:00:49.277227+0000","last_active":"2024-02-14T15:00:54.569865+0000","last_peered":"2024-02-14T15:00:54.569865+0000","last_clean":"2024-02-14T15:00:54.569865+0000","last_became_active":"2024-02-14T15:00:49.276904+0000","last_became_peered":"2024-02-14T15:00:49.276904+0000","last_unstale":"2024-02-14T15:00:54.569865+0000","last_undegraded":"2024-02-14T15:00:54.569865+0000","last_fullsized":"2024-02-14T15:00:54.569865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:01.091327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669924+0000","last_change":"2024-02-14T15:00:53.587855+0000","last_active":"2024-02-14T15:00:54.669924+0000","last_peered":"2024-02-14T15:00:54.669924+0000","last_clean":"2024-02-14T15:00:54.669924+0000","last_became_active":"2024-02-14T15:00:53.586313+0000","last_became_peered":"2024-02-14T15:00:53.586313+0000","last_unstale":"2024-02-14T15:00:54.669924+0000","last_undegraded":"2024-02-14T15:00:54.669924+0000","last_fullsized":"2024-02-14T15:00:54.669924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:36:41.279582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583364+0000","last_change":"2024-02-14T15:00:48.267385+0000","last_active":"2024-02-14T15:01:13.583364+0000","last_peered":"2024-02-14T15:01:13.583364+0000","last_clean":"2024-02-14T15:01:13.583364+0000","last_became_active":"2024-02-14T15:00:48.267036+0000","last_became_peered":"2024-02-14T15:00:48.267036+0000","last_unstale":"2024-02-14T15:01:13.583364+0000","last_undegraded":"2024-02-14T15:01:13.583364+0000","last_fullsized":"2024-02-14T15:01:13.583364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:29:24.840030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669367+0000","last_change":"2024-02-14T15:00:49.272233+0000","last_active":"2024-02-14T15:00:54.669367+0000","last_peered":"2024-02-14T15:00:54.669367+0000","last_clean":"2024-02-14T15:00:54.669367+0000","last_became_active":"2024-02-14T15:00:49.271813+0000","last_became_peered":"2024-02-14T15:00:49.271813+0000","last_unstale":"2024-02-14T15:00:54.669367+0000","last_undegraded":"2024-02-14T15:00:54.669367+0000","last_fullsized":"2024-02-14T15:00:54.669367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:30:32.521443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669790+0000","last_change":"2024-02-14T15:00:53.583385+0000","last_active":"2024-02-14T15:00:54.669790+0000","last_peered":"2024-02-14T15:00:54.669790+0000","last_clean":"2024-02-14T15:00:54.669790+0000","last_became_active":"2024-02-14T15:00:53.583023+0000","last_became_peered":"2024-02-14T15:00:53.583023+0000","last_unstale":"2024-02-14T15:00:54.669790+0000","last_undegraded":"2024-02-14T15:00:54.669790+0000","last_fullsized":"2024-02-14T15:00:54.669790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:40:53.500939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.584073+0000","last_change":"2024-02-14T15:00:48.261376+0000","last_active":"2024-02-14T15:01:13.584073+0000","last_peered":"2024-02-14T15:01:13.584073+0000","last_clean":"2024-02-14T15:01:13.584073+0000","last_became_active":"2024-02-14T15:00:48.260461+0000","last_became_peered":"2024-02-14T15:00:48.260461+0000","last_unstale":"2024-02-14T15:01:13.584073+0000","last_undegraded":"2024-02-14T15:01:13.584073+0000","last_fullsized":"2024-02-14T15:01:13.584073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:35.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669401+0000","last_change":"2024-02-14T15:00:49.273117+0000","last_active":"2024-02-14T15:00:54.669401+0000","last_peered":"2024-02-14T15:00:54.669401+0000","last_clean":"2024-02-14T15:00:54.669401+0000","last_became_active":"2024-02-14T15:00:49.272810+0000","last_became_peered":"2024-02-14T15:00:49.272810+0000","last_unstale":"2024-02-14T15:00:54.669401+0000","last_undegraded":"2024-02-14T15:00:54.669401+0000","last_fullsized":"2024-02-14T15:00:54.669401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:37:00.992212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670882+0000","last_change":"2024-02-14T15:00:53.583972+0000","last_active":"2024-02-14T15:00:54.670882+0000","last_peered":"2024-02-14T15:00:54.670882+0000","last_clean":"2024-02-14T15:00:54.670882+0000","last_became_active":"2024-02-14T15:00:53.583563+0000","last_became_peered":"2024-02-14T15:00:53.583563+0000","last_unstale":"2024-02-14T15:00:54.670882+0000","last_undegraded":"2024-02-14T15:00:54.670882+0000","last_fullsized":"2024-02-14T15:00:54.670882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:08:55.811082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566490+0000","last_change":"2024-02-14T15:00:48.270363+0000","last_active":"2024-02-14T15:00:54.566490+0000","last_peered":"2024-02-14T15:00:54.566490+0000","last_clean":"2024-02-14T15:00:54.566490+0000","last_became_active":"2024-02-14T15:00:48.270030+0000","last_became_peered":"2024-02-14T15:00:48.270030+0000","last_unstale":"2024-02-14T15:00:54.566490+0000","last_undegraded":"2024-02-14T15:00:54.566490+0000","last_fullsized":"2024-02-14T15:00:54.566490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:28.885451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566482+0000","last_change":"2024-02-14T15:00:49.285647+0000","last_active":"2024-02-14T15:00:54.566482+0000","last_peered":"2024-02-14T15:00:54.566482+0000","last_clean":"2024-02-14T15:00:54.566482+0000","last_became_active":"2024-02-14T15:00:49.285267+0000","last_became_peered":"2024-02-14T15:00:49.285267+0000","last_unstale":"2024-02-14T15:00:54.566482+0000","last_undegraded":"2024-02-14T15:00:54.566482+0000","last_fullsized":"2024-02-14T15:00:54.566482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:03.143177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670946+0000","last_change":"2024-02-14T15:00:53.591029+0000","last_active":"2024-02-14T15:00:54.670946+0000","last_peered":"2024-02-14T15:00:54.670946+0000","last_clean":"2024-02-14T15:00:54.670946+0000","last_became_active":"2024-02-14T15:00:53.590570+0000","last_became_peered":"2024-02-14T15:00:53.590570+0000","last_unstale":"2024-02-14T15:00:54.670946+0000","last_undegraded":"2024-02-14T15:00:54.670946+0000","last_fullsized":"2024-02-14T15:00:54.670946+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:41.809227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:01:13.583711+0000","last_change":"2024-02-14T15:00:48.277652+0000","last_active":"2024-02-14T15:01:13.583711+0000","last_peered":"2024-02-14T15:01:13.583711+0000","last_clean":"2024-02-14T15:01:13.583711+0000","last_became_active":"2024-02-14T15:00:48.277317+0000","last_became_peered":"2024-02-14T15:00:48.277317+0000","last_unstale":"2024-02-14T15:01:13.583711+0000","last_undegraded":"2024-02-14T15:01:13.583711+0000","last_fullsized":"2024-02-14T15:01:13.583711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:40:22.474551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566391+0000","last_change":"2024-02-14T15:00:49.276346+0000","last_active":"2024-02-14T15:00:54.566391+0000","last_peered":"2024-02-14T15:00:54.566391+0000","last_clean":"2024-02-14T15:00:54.566391+0000","last_became_active":"2024-02-14T15:00:49.276000+0000","last_became_peered":"2024-02-14T15:00:49.276000+0000","last_unstale":"2024-02-14T15:00:54.566391+0000","last_undegraded":"2024-02-14T15:00:54.566391+0000","last_fullsized":"2024-02-14T15:00:54.566391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:14:03.811981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669265+0000","last_change":"2024-02-14T15:00:53.587021+0000","last_active":"2024-02-14T15:00:54.669265+0000","last_peered":"2024-02-14T15:00:54.669265+0000","last_clean":"2024-02-14T15:00:54.669265+0000","last_became_active":"2024-02-14T15:00:53.586492+0000","last_became_peered":"2024-02-14T15:00:53.586492+0000","last_unstale":"2024-02-14T15:00:54.669265+0000","last_undegraded":"2024-02-14T15:00:54.669265+0000","last_fullsized":"2024-02-14T15:00:54.669265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:51:02.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566923+0000","last_change":"2024-02-14T15:00:48.270322+0000","last_active":"2024-02-14T15:00:54.566923+0000","last_peered":"2024-02-14T15:00:54.566923+0000","last_clean":"2024-02-14T15:00:54.566923+0000","last_became_active":"2024-02-14T15:00:48.269806+0000","last_became_peered":"2024-02-14T15:00:48.269806+0000","last_unstale":"2024-02-14T15:00:54.566923+0000","last_undegraded":"2024-02-14T15:00:54.566923+0000","last_fullsized":"2024-02-14T15:00:54.566923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:20:12.010694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566694+0000","last_change":"2024-02-14T15:00:49.276110+0000","last_active":"2024-02-14T15:00:54.566694+0000","last_peered":"2024-02-14T15:00:54.566694+0000","last_clean":"2024-02-14T15:00:54.566694+0000","last_became_active":"2024-02-14T15:00:49.275729+0000","last_became_peered":"2024-02-14T15:00:49.275729+0000","last_unstale":"2024-02-14T15:00:54.566694+0000","last_undegraded":"2024-02-14T15:00:54.566694+0000","last_fullsized":"2024-02-14T15:00:54.566694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:07:29.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569270+0000","last_change":"2024-02-14T15:00:53.582061+0000","last_active":"2024-02-14T15:00:54.569270+0000","last_peered":"2024-02-14T15:00:54.569270+0000","last_clean":"2024-02-14T15:00:54.569270+0000","last_became_active":"2024-02-14T15:00:53.581659+0000","last_became_peered":"2024-02-14T15:00:53.581659+0000","last_unstale":"2024-02-14T15:00:54.569270+0000","last_undegraded":"2024-02-14T15:00:54.569270+0000","last_fullsized":"2024-02-14T15:00:54.569270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:13.814496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.671090+0000","last_change":"2024-02-14T15:00:48.283300+0000","last_active":"2024-02-14T15:00:54.671090+0000","last_peered":"2024-02-14T15:00:54.671090+0000","last_clean":"2024-02-14T15:00:54.671090+0000","last_became_active":"2024-02-14T15:00:48.282939+0000","last_became_peered":"2024-02-14T15:00:48.282939+0000","last_unstale":"2024-02-14T15:00:54.671090+0000","last_undegraded":"2024-02-14T15:00:54.671090+0000","last_fullsized":"2024-02-14T15:00:54.671090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:56:52.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566464+0000","last_change":"2024-02-14T15:00:49.273435+0000","last_active":"2024-02-14T15:00:54.566464+0000","last_peered":"2024-02-14T15:00:54.566464+0000","last_clean":"2024-02-14T15:00:54.566464+0000","last_became_active":"2024-02-14T15:00:49.272980+0000","last_became_peered":"2024-02-14T15:00:49.272980+0000","last_unstale":"2024-02-14T15:00:54.566464+0000","last_undegraded":"2024-02-14T15:00:54.566464+0000","last_fullsized":"2024-02-14T15:00:54.566464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:19:05.294943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669872+0000","last_change":"2024-02-14T15:00:53.583509+0000","last_active":"2024-02-14T15:00:54.669872+0000","last_peered":"2024-02-14T15:00:54.669872+0000","last_clean":"2024-02-14T15:00:54.669872+0000","last_became_active":"2024-02-14T15:00:53.583148+0000","last_became_peered":"2024-02-14T15:00:53.583148+0000","last_unstale":"2024-02-14T15:00:54.669872+0000","last_undegraded":"2024-02-14T15:00:54.669872+0000","last_fullsized":"2024-02-14T15:00:54.669872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:06:47.059643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669695+0000","last_change":"2024-02-14T15:00:48.286707+0000","last_active":"2024-02-14T15:00:54.669695+0000","last_peered":"2024-02-14T15:00:54.669695+0000","last_clean":"2024-02-14T15:00:54.669695+0000","last_became_active":"2024-02-14T15:00:48.286355+0000","last_became_peered":"2024-02-14T15:00:48.286355+0000","last_unstale":"2024-02-14T15:00:54.669695+0000","last_undegraded":"2024-02-14T15:00:54.669695+0000","last_fullsized":"2024-02-14T15:00:54.669695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:48:31.024464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566799+0000","last_change":"2024-02-14T15:00:49.271598+0000","last_active":"2024-02-14T15:00:54.566799+0000","last_peered":"2024-02-14T15:00:54.566799+0000","last_clean":"2024-02-14T15:00:54.566799+0000","last_became_active":"2024-02-14T15:00:49.270572+0000","last_became_peered":"2024-02-14T15:00:49.270572+0000","last_unstale":"2024-02-14T15:00:54.566799+0000","last_undegraded":"2024-02-14T15:00:54.566799+0000","last_fullsized":"2024-02-14T15:00:54.566799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:12.666557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669530+0000","last_change":"2024-02-14T15:00:49.276697+0000","last_active":"2024-02-14T15:00:54.669530+0000","last_peered":"2024-02-14T15:00:54.669530+0000","last_clean":"2024-02-14T15:00:54.669530+0000","last_became_active":"2024-02-14T15:00:49.276279+0000","last_became_peered":"2024-02-14T15:00:49.276279+0000","last_unstale":"2024-02-14T15:00:54.669530+0000","last_undegraded":"2024-02-14T15:00:54.669530+0000","last_fullsized":"2024-02-14T15:00:54.669530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:11:31.824974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566723+0000","last_change":"2024-02-14T15:00:49.270104+0000","last_active":"2024-02-14T15:00:54.566723+0000","last_peered":"2024-02-14T15:00:54.566723+0000","last_clean":"2024-02-14T15:00:54.566723+0000","last_became_active":"2024-02-14T15:00:49.269742+0000","last_became_peered":"2024-02-14T15:00:49.269742+0000","last_unstale":"2024-02-14T15:00:54.566723+0000","last_undegraded":"2024-02-14T15:00:54.566723+0000","last_fullsized":"2024-02-14T15:00:54.566723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:10:07.494478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566445+0000","last_change":"2024-02-14T15:00:49.274219+0000","last_active":"2024-02-14T15:00:54.566445+0000","last_peered":"2024-02-14T15:00:54.566445+0000","last_clean":"2024-02-14T15:00:54.566445+0000","last_became_active":"2024-02-14T15:00:49.273833+0000","last_became_peered":"2024-02-14T15:00:49.273833+0000","last_unstale":"2024-02-14T15:00:54.566445+0000","last_undegraded":"2024-02-14T15:00:54.566445+0000","last_fullsized":"2024-02-14T15:00:54.566445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:06:02.720273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.569416+0000","last_change":"2024-02-14T15:00:49.273576+0000","last_active":"2024-02-14T15:00:54.569416+0000","last_peered":"2024-02-14T15:00:54.569416+0000","last_clean":"2024-02-14T15:00:54.569416+0000","last_became_active":"2024-02-14T15:00:49.272920+0000","last_became_peered":"2024-02-14T15:00:49.272920+0000","last_unstale":"2024-02-14T15:00:54.569416+0000","last_undegraded":"2024-02-14T15:00:54.569416+0000","last_fullsized":"2024-02-14T15:00:54.569416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:22:03.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670610+0000","last_change":"2024-02-14T15:00:49.274549+0000","last_active":"2024-02-14T15:00:54.670610+0000","last_peered":"2024-02-14T15:00:54.670610+0000","last_clean":"2024-02-14T15:00:54.670610+0000","last_became_active":"2024-02-14T15:00:49.274097+0000","last_became_peered":"2024-02-14T15:00:49.274097+0000","last_unstale":"2024-02-14T15:00:54.670610+0000","last_undegraded":"2024-02-14T15:00:54.670610+0000","last_fullsized":"2024-02-14T15:00:54.670610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:47:44.017607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669957+0000","last_change":"2024-02-14T15:00:49.275699+0000","last_active":"2024-02-14T15:00:54.669957+0000","last_peered":"2024-02-14T15:00:54.669957+0000","last_clean":"2024-02-14T15:00:54.669957+0000","last_became_active":"2024-02-14T15:00:49.275361+0000","last_became_peered":"2024-02-14T15:00:49.275361+0000","last_unstale":"2024-02-14T15:00:54.669957+0000","last_undegraded":"2024-02-14T15:00:54.669957+0000","last_fullsized":"2024-02-14T15:00:54.669957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:34:02.638758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567302+0000","last_change":"2024-02-14T15:00:49.275024+0000","last_active":"2024-02-14T15:00:54.567302+0000","last_peered":"2024-02-14T15:00:54.567302+0000","last_clean":"2024-02-14T15:00:54.567302+0000","last_became_active":"2024-02-14T15:00:49.274710+0000","last_became_peered":"2024-02-14T15:00:49.274710+0000","last_unstale":"2024-02-14T15:00:54.567302+0000","last_undegraded":"2024-02-14T15:00:54.567302+0000","last_fullsized":"2024-02-14T15:00:54.567302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:42.471112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.567055+0000","last_change":"2024-02-14T15:00:49.282638+0000","last_active":"2024-02-14T15:00:54.567055+0000","last_peered":"2024-02-14T15:00:54.567055+0000","last_clean":"2024-02-14T15:00:54.567055+0000","last_became_active":"2024-02-14T15:00:49.280223+0000","last_became_peered":"2024-02-14T15:00:49.280223+0000","last_unstale":"2024-02-14T15:00:54.567055+0000","last_undegraded":"2024-02-14T15:00:54.567055+0000","last_fullsized":"2024-02-14T15:00:54.567055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:40.250242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669972+0000","last_change":"2024-02-14T15:00:49.273025+0000","last_active":"2024-02-14T15:00:54.669972+0000","last_peered":"2024-02-14T15:00:54.669972+0000","last_clean":"2024-02-14T15:00:54.669972+0000","last_became_active":"2024-02-14T15:00:49.272700+0000","last_became_peered":"2024-02-14T15:00:49.272700+0000","last_unstale":"2024-02-14T15:00:54.669972+0000","last_undegraded":"2024-02-14T15:00:54.669972+0000","last_fullsized":"2024-02-14T15:00:54.669972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:08.677428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.566252+0000","last_change":"2024-02-14T15:00:49.282263+0000","last_active":"2024-02-14T15:00:54.566252+0000","last_peered":"2024-02-14T15:00:54.566252+0000","last_clean":"2024-02-14T15:00:54.566252+0000","last_became_active":"2024-02-14T15:00:49.279677+0000","last_became_peered":"2024-02-14T15:00:49.279677+0000","last_unstale":"2024-02-14T15:00:54.566252+0000","last_undegraded":"2024-02-14T15:00:54.566252+0000","last_fullsized":"2024-02-14T15:00:54.566252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:48.036316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669507+0000","last_change":"2024-02-14T15:00:49.282178+0000","last_active":"2024-02-14T15:00:54.669507+0000","last_peered":"2024-02-14T15:00:54.669507+0000","last_clean":"2024-02-14T15:00:54.669507+0000","last_became_active":"2024-02-14T15:00:49.281804+0000","last_became_peered":"2024-02-14T15:00:49.281804+0000","last_unstale":"2024-02-14T15:00:54.669507+0000","last_undegraded":"2024-02-14T15:00:54.669507+0000","last_fullsized":"2024-02-14T15:00:54.669507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:58.672808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.669881+0000","last_change":"2024-02-14T15:00:49.269882+0000","last_active":"2024-02-14T15:00:54.669881+0000","last_peered":"2024-02-14T15:00:54.669881+0000","last_clean":"2024-02-14T15:00:54.669881+0000","last_became_active":"2024-02-14T15:00:49.269489+0000","last_became_peered":"2024-02-14T15:00:49.269489+0000","last_unstale":"2024-02-14T15:00:54.669881+0000","last_undegraded":"2024-02-14T15:00:54.669881+0000","last_fullsized":"2024-02-14T15:00:54.669881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:57:40.465044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-14T15:00:54.670052+0000","last_change":"2024-02-14T15:00:49.277688+0000","last_active":"2024-02-14T15:00:54.670052+0000","last_peered":"2024-02-14T15:00:54.670052+0000","last_clean":"2024-02-14T15:00:54.670052+0000","last_became_active":"2024-02-14T15:00:49.277348+0000","last_became_peered":"2024-02-14T15:00:49.277348+0000","last_unstale":"2024-02-14T15:00:54.670052+0000","last_undegraded":"2024-02-14T15:00:54.670052+0000","last_fullsized":"2024-02-14T15:00:54.670052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:33.946319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48699,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":23,"num_read_kb":61,"num_write":136,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":63210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":476575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64300000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.91200000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.752}]},{"osd":7,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.85699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":9,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.871},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.88300000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.90000000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1654784,"data_stored":1109383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.79900000000000004}]},{"osd":3,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.52700000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.88200000000000001}]},{"osd":6,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65500000000000003}]},{"osd":9,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Wed Feb 14 15:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.81699999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":485585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61299999999999999,"5min":1.22,"15min":1.22},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88900000000000001,"5min":1.246,"15min":1.246},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.60799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":1.069,"15min":1.069},"last":0.44}]},{"osd":3,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.745,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.57499999999999996,"15min":0.58699999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.875,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.141,"15min":1.141},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.6100000000000001,"5min":0.93000000000000005,"15min":0.81699999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":19.152000000000001,"5min":19.152000000000001,"15min":19.152000000000001},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.31},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":0.97699999999999998,"15min":0.875},"min":{"1min":0.504,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":18.734999999999999,"5min":18.734999999999999,"15min":18.734999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.92300000000000004,"15min":0.95899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0409999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.71499999999999997}]},{"osd":8,"last update":"Wed Feb 14 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":0.97999999999999998,"15min":0.87},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":19.135999999999999,"5min":19.135999999999999,"15min":19.135999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.83199999999999996,"15min":0.85399999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0229999999999999,"5min":1.23,"15min":1.23},"last":0.59399999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:00:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55200000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.746}]},{"osd":2,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42099999999999999,"15min":0.42199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71799999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73399999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66700000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.751,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.627,"5min":1.161,"15min":1.25},"min":{"1min":0.47099999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79100000000000004,"5min":13.618,"15min":13.618},"last":0.68700000000000006}]},{"osd":7,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.87},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.76600000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83099999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68000000000000005,"15min":0.66900000000000004},"min":{"1min":0.56200000000000006,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80300000000000005}]},{"osd":10,"last update":"Wed Feb 14 15:00:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.90000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1068735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.38200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.39900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.502,"5min":0.499,"15min":0.499},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68899999999999995}]},{"osd":4,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63700000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48899999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83299999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84099999999999997}]},{"osd":6,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.56699999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.58299999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":1.0169999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.59999999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.018,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.0029999999999999}]},{"osd":9,"last update":"Wed Feb 14 15:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":1.052},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.84399999999999997,"15min":0.85399999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.998,"5min":1.175,"15min":1.175},"last":0.98399999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.94799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":476665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38100000000000001,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.434}]},{"osd":1,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78300000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80700000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.755,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.875},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76500000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.443},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.309,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94499999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90900000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0369999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.502,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.79000000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.73199999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.85599999999999998}]},{"osd":9,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93500000000000005,"5min":0.996,"15min":0.996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":1.226,"15min":1.226},"last":0.70199999999999996}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53200000000000003,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.376},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53400000000000003,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56799999999999995,"15min":0.52200000000000002},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.77000000000000002,"15min":0.57399999999999995},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":18.332000000000001,"15min":18.332000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Feb 14 15:00:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.72999999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.626,"15min":0.59399999999999997},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64600000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:01:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.497,"15min":0.46300000000000002},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.81899999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.35199999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:00:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.56399999999999995},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.751},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.626,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66500000000000004}]},{"osd":6,"last update":"Wed Feb 14 15:00:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.36699999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.65900000000000003,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:00:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.442}]},{"osd":8,"last update":"Wed Feb 14 15:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.73799999999999999,"15min":0.72699999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80400000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90000000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71499999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.70399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":478053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 15:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.502},"min":{"1min":0.31,"5min":0.253,"15min":0.253},"max":{"1min":0.74299999999999999,"5min":1.002,"15min":1.002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52300000000000002,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:00:17 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53800000000000003,"15min":0.502},"min":{"1min":0.379,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.51400000000000001},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.745}]},{"osd":3,"last update":"Wed Feb 14 14:59:40 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.65500000000000003,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.45000000000000001,"15min":0.42599999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39400000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54700000000000004,"15min":0.51200000000000001},"min":{"1min":0.38600000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:00:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56499999999999995,"15min":0.55600000000000005},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.69699999999999995}]},{"osd":6,"last update":"Wed Feb 14 15:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:00:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65800000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:00:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82599999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92700000000000005,"5min":1.052,"15min":1.052},"last":0.78100000000000003}]},{"osd":9,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.47299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.75900000000000001}]},{"osd":11,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.79600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48999999999999999,"15min":0.46999999999999997},"min":{"1min":0.31900000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.47499999999999998},"min":{"1min":0.36499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.627}]},{"osd":1,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.505},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.58799999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40899999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77100000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.42399999999999999},"min":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67400000000000004,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.435,"15min":0.42899999999999999},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.41599999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.36199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91400000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.499},{"interface":"front","average":{"1min":0.624,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.746},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81599999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70499999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64000000000000001,"15min":0.64800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58999999999999997}]},{"osd":9,"last update":"Wed Feb 14 15:01:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61199999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":476769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.314},"min":{"1min":0.25700000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.372,"15min":0.34799999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.83599999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72899999999999998,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.081,"5min":19.896999999999998,"15min":19.896999999999998},"last":0.504},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.74299999999999999,"15min":0.59299999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86599999999999999,"5min":19.911999999999999,"15min":19.911999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Wed Feb 14 15:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56000000000000005,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57699999999999996,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Feb 14 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83499999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73299999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65600000000000003}]},{"osd":5,"last update":"Wed Feb 14 15:00:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.2549999999999999,"15min":1.542},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89500000000000002,"5min":19.879999999999999,"15min":19.879999999999999},"last":0.628},{"interface":"front","average":{"1min":0.626,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80700000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.439},{"interface":"front","average":{"1min":0.496,"5min":0.5,"15min":0.48499999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40400000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91600000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.875,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55600000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70499999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.66300000000000003,"15min":0.67900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52700000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:00:37 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":476597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.29499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.63700000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.629},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.42899999999999999,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.58199999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71599999999999997}]},{"osd":2,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82099999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.58999999999999997},"min":{"1min":0.41299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83199999999999996,"5min":0.997,"15min":0.997},"last":0.76300000000000001}]},{"osd":3,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.39900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Wed Feb 14 15:00:31 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.42499999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.874},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91500000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83199999999999996}]},{"osd":6,"last update":"Wed Feb 14 15:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97899999999999998,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0249999999999999,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":0.91100000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:00:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68000000000000005,"15min":0.68799999999999994},"min":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.81000000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92100000000000004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55200000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.042,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.996}]},{"osd":9,"last update":"Wed Feb 14 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.57299999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.999,"5min":1.04,"15min":1.04},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.052}]},{"osd":10,"last update":"Wed Feb 14 15:00:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.032}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1068419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.373,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.48899999999999999,"15min":0.46800000000000003},"min":{"1min":0.38100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.624}]},{"osd":2,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36199999999999999,"15min":0.34899999999999998},"min":{"1min":0.28699999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27400000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.42899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.68799999999999994}]},{"osd":6,"last update":"Wed Feb 14 15:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.46500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:00:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71799999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60499999999999998}]},{"osd":8,"last update":"Wed Feb 14 15:00:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54800000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:00:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.75600000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":47640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T15:01:18.695 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-14T15:01:18.696 INFO:tasks.cephadm:Setting up client nodes... 2024-02-14T15:01:18.696 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T15:01:18.892 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b[67204]: 2024-02-14T15:01:18.557+0000 7f6127ec9700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1213254447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3161930851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:18.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:18 smithi050 ceph-mon[36512]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1213254447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3161930851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:18 smithi076 ceph-mon[37057]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:01:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-14T15:01:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:01:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-02-14T15:01:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1213254447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:01:19.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3161930851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:19.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:18 smithi017 ceph-mon[32549]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:01:19.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:19 smithi017 ceph-mon[32549]: pgmap v307: 129 pgs: 129 active+clean; 624 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-02-14T15:01:19.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:19 smithi017 ceph-mon[32549]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:01:19.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:19 smithi050 ceph-mon[36512]: pgmap v307: 129 pgs: 129 active+clean; 624 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-02-14T15:01:19.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:19 smithi050 ceph-mon[36512]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:01:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:19 smithi076 ceph-mon[37057]: pgmap v307: 129 pgs: 129 active+clean; 624 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-02-14T15:01:19.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:19 smithi076 ceph-mon[37057]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 -- 172.21.15.17:0/149012519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 msgr2=0x7f2a8c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 --2- 172.21.15.17:0/149012519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c109140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400b0d0 tx=0x7f2a8400b3e0 comp rx=0 tx=0).stop 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 -- 172.21.15.17:0/149012519 shutdown_connections 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 --2- 172.21.15.17:0/149012519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c109140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 --2- 172.21.15.17:0/149012519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a8c100cf0 0x7f2a8c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 --2- 172.21.15.17:0/149012519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8c101ef0 0x7f2a8c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 -- 172.21.15.17:0/149012519 >> 172.21.15.17:0/149012519 conn(0x7f2a8c0fc2b0 msgr2=0x7f2a8c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:20.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 -- 172.21.15.17:0/149012519 shutdown_connections 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.754+0000 7f2a915ec700 1 -- 172.21.15.17:0/149012519 wait complete. 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 Processor -- start 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 -- start start 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a8c100cf0 0x7f2a8c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8c101ef0 0x7f2a8c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c19f880 con 0x7f2a8c102890 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c19f9c0 con 0x7f2a8c101ef0 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a915ec700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c19fb80 con 0x7f2a8c100cf0 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a8b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a8b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53050/0 (socket says 172.21.15.17:53050) 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a8b7fe700 1 -- 172.21.15.17:0/3451477217 learned_addr learned my addr 172.21.15.17:0/3451477217 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.755+0000 7f2a8a7fc700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8c101ef0 0x7f2a8c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:20.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8affd700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a8c100cf0 0x7f2a8c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8b7fe700 1 -- 172.21.15.17:0/3451477217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a8c100cf0 msgr2=0x7f2a8c197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8b7fe700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a8c100cf0 0x7f2a8c197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8b7fe700 1 -- 172.21.15.17:0/3451477217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8c101ef0 msgr2=0x7f2a8c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8b7fe700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8c101ef0 0x7f2a8c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8b7fe700 1 -- 172.21.15.17:0/3451477217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a84009d00 con 0x7f2a8c102890 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8affd700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a8c100cf0 0x7f2a8c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a8b7fe700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c19f2b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a84007c90 tx=0x7f2a84009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a73fff700 1 -- 172.21.15.17:0/3451477217 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400e040 con 0x7f2a8c102890 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a73fff700 1 -- 172.21.15.17:0/3451477217 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8401dd70 con 0x7f2a8c102890 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c19fe00 con 0x7f2a8c102890 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a73fff700 1 -- 172.21.15.17:0/3451477217 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8401be60 con 0x7f2a8c102890 2024-02-14T15:01:20.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.756+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c1a01c0 con 0x7f2a8c102890 2024-02-14T15:01:20.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.757+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c050230 con 0x7f2a8c102890 2024-02-14T15:01:20.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.758+0000 7f2a73fff700 1 -- 172.21.15.17:0/3451477217 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2a8401b4e0 con 0x7f2a8c102890 2024-02-14T15:01:20.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.759+0000 7f2a73fff700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2a740a2290 0x7f2a740a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:20.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.759+0000 7f2a8affd700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2a740a2290 0x7f2a740a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:20.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.759+0000 7f2a73fff700 1 -- 172.21.15.17:0/3451477217 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2a840cfdb0 con 0x7f2a8c102890 2024-02-14T15:01:20.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.759+0000 7f2a8affd700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2a740a2290 0x7f2a740a4750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005950 tx=0x7f2a7c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:20.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:20.763+0000 7f2a73fff700 1 -- 172.21.15.17:0/3451477217 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a840d4070 con 0x7f2a8c102890 2024-02-14T15:01:21.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.051+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2a8c107fe0 con 0x7f2a8c102890 2024-02-14T15:01:21.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.053+0000 7f2a73fff700 1 -- 172.21.15.17:0/3451477217 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2a84098a60 con 0x7f2a8c102890 2024-02-14T15:01:21.054 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-02-14T15:01:21.054 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDB1cxlue8jAxAANmLxKiFk+Lolyw+gYV2dtA== 2024-02-14T15:01:21.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.055+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2a740a2290 msgr2=0x7f2a740a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:21.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.055+0000 7f2a915ec700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2a740a2290 0x7f2a740a4750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005950 tx=0x7f2a7c0058e0 comp rx=0 tx=0).stop 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.055+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 msgr2=0x7f2a8c19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.055+0000 7f2a915ec700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c19f2b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a84007c90 tx=0x7f2a84009380 comp rx=0 tx=0).stop 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 shutdown_connections 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2a740a2290 0x7f2a740a4750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a8c102890 0x7f2a8c19f2b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a8c100cf0 0x7f2a8c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 --2- 172.21.15.17:0/3451477217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a8c101ef0 0x7f2a8c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 >> 172.21.15.17:0/3451477217 conn(0x7f2a8c0fc2b0 msgr2=0x7f2a8c105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 shutdown_connections 2024-02-14T15:01:21.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:21.056+0000 7f2a915ec700 1 -- 172.21.15.17:0/3451477217 wait complete. 2024-02-14T15:01:21.849 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:21 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3451477217' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T15:01:21.850 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:21 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3451477217' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T15:01:21.851 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:01:21.851 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-14T15:01:21.851 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-14T15:01:21.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:21 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3451477217' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T15:01:21.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:21 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3451477217' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T15:01:21.893 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-14T15:01:21.893 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-14T15:01:21.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph mgr dump --format=json 2024-02-14T15:01:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:21 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3451477217' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T15:01:21.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:21 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3451477217' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T15:01:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:22.885 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:22 smithi076 ceph-mon[37057]: pgmap v308: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-14T15:01:22.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:22 smithi050 ceph-mon[36512]: pgmap v308: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-14T15:01:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:22 smithi017 ceph-mon[32549]: pgmap v308: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-14T15:01:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 -- 172.21.15.17:0/2232015317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f806a460 msgr2=0x7fd5f806a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/2232015317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f806a460 0x7fd5f806a8e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099c0 tx=0x7fd5ec009cd0 comp rx=0 tx=0).stop 2024-02-14T15:01:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 -- 172.21.15.17:0/2232015317 shutdown_connections 2024-02-14T15:01:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/2232015317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5f806ae20 0x7fd5f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/2232015317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5f8107e90 0x7fd5f8108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/2232015317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f806a460 0x7fd5f806a8e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 -- 172.21.15.17:0/2232015317 >> 172.21.15.17:0/2232015317 conn(0x7fd5f8077ab0 msgr2=0x7fd5f8077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 -- 172.21.15.17:0/2232015317 shutdown_connections 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 -- 172.21.15.17:0/2232015317 wait complete. 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 Processor -- start 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.954+0000 7fd5ff92b700 1 -- start start 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ff92b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5f806a460 0x7fd5f8198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ff92b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5f806ae20 0x7fd5f8198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ff92b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f8107e90 0x7fd5f819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ff92b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f819d020 con 0x7fd5f8107e90 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ff92b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f819d190 con 0x7fd5f806ae20 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ff92b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f819d350 con 0x7fd5f806a460 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f8107e90 0x7fd5f819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f8107e90 0x7fd5f819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53068/0 (socket says 172.21.15.17:53068) 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 -- 172.21.15.17:0/980964032 learned_addr learned my addr 172.21.15.17:0/980964032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fcec6700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5f806ae20 0x7fd5f8198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:23.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fd6c7700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5f806a460 0x7fd5f8198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 -- 172.21.15.17:0/980964032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5f806a460 msgr2=0x7fd5f8198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5f806a460 0x7fd5f8198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 -- 172.21.15.17:0/980964032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5f806ae20 msgr2=0x7fd5f8198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5f806ae20 0x7fd5f8198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 -- 172.21.15.17:0/980964032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0095f0 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5fdec8700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f8107e90 0x7fd5f819ca10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000cca0 tx=0x7fd5f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ea7fc700 1 -- 172.21.15.17:0/980964032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000aaf0 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ea7fc700 1 -- 172.21.15.17:0/980964032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5f000ac50 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f819d630 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.955+0000 7fd5ea7fc700 1 -- 172.21.15.17:0/980964032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0011600 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.956+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f81a4e90 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.956+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f80689e0 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.957+0000 7fd5ea7fc700 1 -- 172.21.15.17:0/980964032 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fd5f0012420 con 0x7fd5f8107e90 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.958+0000 7fd5ea7fc700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd5e40a21e0 0x7fd5e40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.958+0000 7fd5fd6c7700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd5e40a21e0 0x7fd5e40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:23.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.958+0000 7fd5ea7fc700 1 -- 172.21.15.17:0/980964032 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5f00ce0e0 con 0x7fd5f8107e90 2024-02-14T15:01:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.959+0000 7fd5fd6c7700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd5e40a21e0 0x7fd5e40a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400ac50 tx=0x7fd5f400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:23.963+0000 7fd5ea7fc700 1 -- 172.21.15.17:0/980964032 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5f0096e70 con 0x7fd5f8107e90 2024-02-14T15:01:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.224+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd5f8111df0 con 0x7fd5f8107e90 2024-02-14T15:01:24.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.229+0000 7fd5ea7fc700 1 -- 172.21.15.17:0/980964032 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261485 (secure 0 0 0) 0x7fd5f009a530 con 0x7fd5f8107e90 2024-02-14T15:01:24.233 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd5e40a21e0 msgr2=0x7fd5e40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd5e40a21e0 0x7fd5e40a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400ac50 tx=0x7fd5f400a380 comp rx=0 tx=0).stop 2024-02-14T15:01:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f8107e90 msgr2=0x7fd5f819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f8107e90 0x7fd5f819ca10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000cca0 tx=0x7fd5f000a060 comp rx=0 tx=0).stop 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 shutdown_connections 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fd5e40a21e0 0x7fd5e40a46a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5f8107e90 0x7fd5f819ca10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5f806a460 0x7fd5f8198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 --2- 172.21.15.17:0/980964032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5f806ae20 0x7fd5f8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 >> 172.21.15.17:0/980964032 conn(0x7fd5f8077ab0 msgr2=0x7fd5f80ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 shutdown_connections 2024-02-14T15:01:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:24.232+0000 7fd5ff92b700 1 -- 172.21.15.17:0/980964032 wait complete. 2024-02-14T15:01:24.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:24 smithi050 ceph-mon[36512]: pgmap v309: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-14T15:01:24.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:24 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/980964032' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T15:01:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:24 smithi076 ceph-mon[37057]: pgmap v309: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-14T15:01:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:24 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/980964032' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T15:01:25.015 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:24 smithi017 ceph-mon[32549]: pgmap v309: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-14T15:01:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:24 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/980964032' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T15:01:25.049 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3796717329},{"type":"v1","addr":"172.21.15.17:6801","nonce":3796717329}]},"active_addr":"172.21.15.17:6801/3796717329","active_change":"2024-02-14T14:53:42.570053+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24113,"name":"y","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24115,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.17:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":2755965102}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":1369411962}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":3775022683}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":2092370502}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":1887862276}]}]} 2024-02-14T15:01:25.054 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-14T15:01:25.054 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-14T15:01:25.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T15:01:25.308 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:26.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:26 smithi017 ceph-mon[32549]: pgmap v310: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.7 KiB/s wr, 14 op/s 2024-02-14T15:01:26.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:26 smithi050 ceph-mon[36512]: pgmap v310: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.7 KiB/s wr, 14 op/s 2024-02-14T15:01:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:26 smithi076 ceph-mon[37057]: pgmap v310: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.7 KiB/s wr, 14 op/s 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 -- 172.21.15.17:0/3115011098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414106cb0 msgr2=0x7fc414107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 --2- 172.21.15.17:0/3115011098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414106cb0 0x7fc414107090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc404009990 tx=0x7fc404009ca0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 -- 172.21.15.17:0/3115011098 shutdown_connections 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 --2- 172.21.15.17:0/3115011098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4141015c0 0x7fc41410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 --2- 172.21.15.17:0/3115011098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414106cb0 0x7fc414107090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 --2- 172.21.15.17:0/3115011098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc414100c00 0x7fc414101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 -- 172.21.15.17:0/3115011098 >> 172.21.15.17:0/3115011098 conn(0x7fc414077c30 msgr2=0x7fc414078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 -- 172.21.15.17:0/3115011098 shutdown_connections 2024-02-14T15:01:27.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.257+0000 7fc419bf7700 1 -- 172.21.15.17:0/3115011098 wait complete. 2024-02-14T15:01:27.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 Processor -- start 2024-02-14T15:01:27.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 -- start start 2024-02-14T15:01:27.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414100c00 0x7fc414199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:27.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4141015c0 0x7fc414199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:27.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc414106cb0 0x7fc4141a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414199dd0 con 0x7fc414100c00 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141a45e0 con 0x7fc4141015c0 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc419bf7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141a4720 con 0x7fc414106cb0 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414100c00 0x7fc414199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414100c00 0x7fc414199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53072/0 (socket says 172.21.15.17:53072) 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc4137fe700 1 -- 172.21.15.17:0/549131055 learned_addr learned my addr 172.21.15.17:0/549131055 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc412ffd700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4141015c0 0x7fc414199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.258+0000 7fc413fff700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc414106cb0 0x7fc4141a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc4137fe700 1 -- 172.21.15.17:0/549131055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc414106cb0 msgr2=0x7fc4141a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc4137fe700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc414106cb0 0x7fc4141a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc4137fe700 1 -- 172.21.15.17:0/549131055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4141015c0 msgr2=0x7fc414199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc4137fe700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4141015c0 0x7fc414199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc4137fe700 1 -- 172.21.15.17:0/549131055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040095f0 con 0x7fc414100c00 2024-02-14T15:01:27.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc4137fe700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414100c00 0x7fc414199020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4040098e0 tx=0x7fc40400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:27.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc410ff9700 1 -- 172.21.15.17:0/549131055 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40401d070 con 0x7fc414100c00 2024-02-14T15:01:27.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc410ff9700 1 -- 172.21.15.17:0/549131055 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc404003680 con 0x7fc414100c00 2024-02-14T15:01:27.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4141a4950 con 0x7fc414100c00 2024-02-14T15:01:27.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc410ff9700 1 -- 172.21.15.17:0/549131055 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4040173f0 con 0x7fc414100c00 2024-02-14T15:01:27.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.259+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4141a4e40 con 0x7fc414100c00 2024-02-14T15:01:27.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.260+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc414109120 con 0x7fc414100c00 2024-02-14T15:01:27.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.261+0000 7fc410ff9700 1 -- 172.21.15.17:0/549131055 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc404022720 con 0x7fc414100c00 2024-02-14T15:01:27.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.261+0000 7fc410ff9700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:27.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.262+0000 7fc412ffd700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:27.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.262+0000 7fc410ff9700 1 -- 172.21.15.17:0/549131055 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc4040cf2a0 con 0x7fc414100c00 2024-02-14T15:01:27.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.262+0000 7fc412ffd700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4140782a0 tx=0x7fc40800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:27.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.266+0000 7fc410ff9700 1 -- 172.21.15.17:0/549131055 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc4040980d0 con 0x7fc414100c00 2024-02-14T15:01:27.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.469+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc414110c40 con 0x7fc414100c00 2024-02-14T15:01:27.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.471+0000 7fc410ff9700 1 -- 172.21.15.17:0/549131055 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7fc40409b790 con 0x7fc414100c00 2024-02-14T15:01:27.472 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:27.472 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3fc0a2260 msgr2=0x7fc3fc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4140782a0 tx=0x7fc40800b410 comp rx=0 tx=0).stop 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414100c00 msgr2=0x7fc414199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414100c00 0x7fc414199020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4040098e0 tx=0x7fc40400f690 comp rx=0 tx=0).stop 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 shutdown_connections 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3fc0a2260 0x7fc3fc0a4720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc414106cb0 0x7fc4141a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc414100c00 0x7fc414199020 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 --2- 172.21.15.17:0/549131055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4141015c0 0x7fc414199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 >> 172.21.15.17:0/549131055 conn(0x7fc414077c30 msgr2=0x7fc4140ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 shutdown_connections 2024-02-14T15:01:27.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:27.473+0000 7fc419bf7700 1 -- 172.21.15.17:0/549131055 wait complete. 2024-02-14T15:01:27.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:27 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/549131055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:27.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:27 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/549131055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:27 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/549131055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:28.187 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-14T15:01:28.187 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T15:01:28.435 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:28.877 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:28 smithi017 ceph-mon[32549]: pgmap v311: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-02-14T15:01:28.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:28 smithi050 ceph-mon[36512]: pgmap v311: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-02-14T15:01:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:28 smithi076 ceph-mon[37057]: pgmap v311: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-02-14T15:01:29.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:29 smithi050 ceph-mon[36512]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-02-14T15:01:29.911 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:29 smithi017 ceph-mon[32549]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-02-14T15:01:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:29 smithi076 ceph-mon[37057]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-02-14T15:01:30.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 -- 172.21.15.17:0/858926814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c100c00 msgr2=0x7fab5c101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 --2- 172.21.15.17:0/858926814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c100c00 0x7fab5c101080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009990 tx=0x7fab4c009ca0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 -- 172.21.15.17:0/858926814 shutdown_connections 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 --2- 172.21.15.17:0/858926814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab5c1015c0 0x7fab5c10da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 --2- 172.21.15.17:0/858926814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab5c106cb0 0x7fab5c107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 --2- 172.21.15.17:0/858926814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c100c00 0x7fab5c101080 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 -- 172.21.15.17:0/858926814 >> 172.21.15.17:0/858926814 conn(0x7fab5c077c30 msgr2=0x7fab5c078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 -- 172.21.15.17:0/858926814 shutdown_connections 2024-02-14T15:01:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.145+0000 7fab63e17700 1 -- 172.21.15.17:0/858926814 wait complete. 2024-02-14T15:01:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 Processor -- start 2024-02-14T15:01:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 -- start start 2024-02-14T15:01:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab5c100c00 0x7fab5c198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab5c1015c0 0x7fab5c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c106cb0 0x7fab5c1a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c199dc0 con 0x7fab5c106cb0 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a4580 con 0x7fab5c1015c0 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab63e17700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a46c0 con 0x7fab5c100c00 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c106cb0 0x7fab5c1a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c106cb0 0x7fab5c1a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53088/0 (socket says 172.21.15.17:53088) 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 -- 172.21.15.17:0/3174775746 learned_addr learned my addr 172.21.15.17:0/3174775746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab61bb3700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab5c100c00 0x7fab5c198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab613b2700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab5c1015c0 0x7fab5c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 -- 172.21.15.17:0/3174775746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab5c100c00 msgr2=0x7fab5c198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab5c100c00 0x7fab5c198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 -- 172.21.15.17:0/3174775746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab5c1015c0 msgr2=0x7fab5c199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab5c1015c0 0x7fab5c199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.146+0000 7fab623b4700 1 -- 172.21.15.17:0/3174775746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c0095f0 con 0x7fab5c106cb0 2024-02-14T15:01:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.147+0000 7fab623b4700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c106cb0 0x7fab5c1a4040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fab54007ee0 tx=0x7fab5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.147+0000 7fab52ffd700 1 -- 172.21.15.17:0/3174775746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54017070 con 0x7fab5c106cb0 2024-02-14T15:01:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.147+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c1a49a0 con 0x7fab5c106cb0 2024-02-14T15:01:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.147+0000 7fab52ffd700 1 -- 172.21.15.17:0/3174775746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab5400f040 con 0x7fab5c106cb0 2024-02-14T15:01:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.147+0000 7fab52ffd700 1 -- 172.21.15.17:0/3174775746 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54005650 con 0x7fab5c106cb0 2024-02-14T15:01:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.147+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c1a4e20 con 0x7fab5c106cb0 2024-02-14T15:01:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.148+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c109000 con 0x7fab5c106cb0 2024-02-14T15:01:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.150+0000 7fab52ffd700 1 -- 172.21.15.17:0/3174775746 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fab54022020 con 0x7fab5c106cb0 2024-02-14T15:01:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.150+0000 7fab52ffd700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab480a21e0 0x7fab480a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.150+0000 7fab61bb3700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab480a21e0 0x7fab480a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.150+0000 7fab52ffd700 1 -- 172.21.15.17:0/3174775746 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fab540cd650 con 0x7fab5c106cb0 2024-02-14T15:01:30.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.151+0000 7fab61bb3700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab480a21e0 0x7fab480a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab58005950 tx=0x7fab5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.154+0000 7fab52ffd700 1 -- 172.21.15.17:0/3174775746 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab540962d0 con 0x7fab5c106cb0 2024-02-14T15:01:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.356+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab5c1022f0 con 0x7fab5c106cb0 2024-02-14T15:01:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.357+0000 7fab52ffd700 1 -- 172.21.15.17:0/3174775746 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23954 (secure 0 0 0) 0x7fab54099990 con 0x7fab5c106cb0 2024-02-14T15:01:30.359 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:01:30.359 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:00:53.931824+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:01:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab480a21e0 msgr2=0x7fab480a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:30.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab480a21e0 0x7fab480a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab58005950 tx=0x7fab5800b410 comp rx=0 tx=0).stop 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c106cb0 msgr2=0x7fab5c1a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c106cb0 0x7fab5c1a4040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fab54007ee0 tx=0x7fab5400d3b0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 shutdown_connections 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab480a21e0 0x7fab480a46a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab5c106cb0 0x7fab5c1a4040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab5c100c00 0x7fab5c198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 --2- 172.21.15.17:0/3174775746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab5c1015c0 0x7fab5c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 >> 172.21.15.17:0/3174775746 conn(0x7fab5c077c30 msgr2=0x7fab5c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 shutdown_connections 2024-02-14T15:01:30.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:30.359+0000 7fab63e17700 1 -- 172.21.15.17:0/3174775746 wait complete. 2024-02-14T15:01:30.763 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:30 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3174775746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:30.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:30 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3174775746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:30 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3174775746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:01:31.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-14T15:01:31.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-14T15:01:31.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-14T15:01:31.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-14T15:01:31.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-14T15:01:31.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-14T15:01:31.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-14T15:01:31.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-14T15:01:31.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-14T15:01:31.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-14T15:01:31.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-14T15:01:31.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-14T15:01:31.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:31 smithi050 ceph-mon[36512]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-14T15:01:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:31.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:31 smithi076 ceph-mon[37057]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-14T15:01:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:31 smithi017 ceph-mon[32549]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-14T15:01:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.186 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.196 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.196 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.200 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:32.201 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:33.969 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:33 smithi017 ceph-mon[32549]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-14T15:01:33.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:33 smithi076 ceph-mon[37057]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-14T15:01:34.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:33 smithi050 ceph-mon[36512]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-14T15:01:36.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:36 smithi017 ceph-mon[32549]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-14T15:01:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:36 smithi050 ceph-mon[36512]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-14T15:01:36.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:36 smithi076 ceph-mon[37057]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-14T15:01:38.753 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:38 smithi017 ceph-mon[32549]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:38.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:38 smithi050 ceph-mon[36512]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:38 smithi076 ceph-mon[37057]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:40.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:40 smithi050 ceph-mon[36512]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:40 smithi076 ceph-mon[37057]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:40 smithi017 ceph-mon[32549]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:42.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:42 smithi017 ceph-mon[32549]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:42 smithi050 ceph-mon[36512]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 -- 172.21.15.17:0/3933838599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040100c30 msgr2=0x7fe04010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 --2- 172.21.15.17:0/3933838599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040100c30 0x7fe04010ec40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b0d0 tx=0x7fe03800b3e0 comp rx=0 tx=0).stop 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 -- 172.21.15.17:0/3933838599 shutdown_connections 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 --2- 172.21.15.17:0/3933838599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040100c30 0x7fe04010ec40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 --2- 172.21.15.17:0/3933838599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040107e70 0x7fe040108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 --2- 172.21.15.17:0/3933838599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe040100270 0x7fe0401006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 -- 172.21.15.17:0/3933838599 >> 172.21.15.17:0/3933838599 conn(0x7fe0400771f0 msgr2=0x7fe0400fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 -- 172.21.15.17:0/3933838599 shutdown_connections 2024-02-14T15:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.977+0000 7fe04676a700 1 -- 172.21.15.17:0/3933838599 wait complete. 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 Processor -- start 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 -- start start 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe040100270 0x7fe040198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040100c30 0x7fe040198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040107e70 0x7fe04019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04019d2c0 con 0x7fe040107e70 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04019d430 con 0x7fe040100c30 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe04676a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04019d5f0 con 0x7fe040100270 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe044d07700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040107e70 0x7fe04019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe044d07700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040107e70 0x7fe04019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53382/0 (socket says 172.21.15.17:53382) 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe044d07700 1 -- 172.21.15.17:0/2181436520 learned_addr learned my addr 172.21.15.17:0/2181436520 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:42.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe03ffff700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe040100270 0x7fe040198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe03f7fe700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040100c30 0x7fe040198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe044d07700 1 -- 172.21.15.17:0/2181436520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe040100270 msgr2=0x7fe040198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.978+0000 7fe044d07700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe040100270 0x7fe040198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe044d07700 1 -- 172.21.15.17:0/2181436520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040100c30 msgr2=0x7fe040198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe044d07700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040100c30 0x7fe040198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe044d07700 1 -- 172.21.15.17:0/2181436520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009d00 con 0x7fe040107e70 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe03f7fe700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040100c30 0x7fe040198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe044d07700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040107e70 0x7fe04019ccb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0380079b0 tx=0x7fe038007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800e040 con 0x7fe040107e70 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04019d870 con 0x7fe040107e70 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe038004e60 con 0x7fe040107e70 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038012b60 con 0x7fe040107e70 2024-02-14T15:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.979+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0401a5130 con 0x7fe040107e70 2024-02-14T15:01:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.980+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0401120f0 con 0x7fe040107e70 2024-02-14T15:01:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.981+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe038012cc0 con 0x7fe040107e70 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.981+0000 7fe03d7fa700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03ffff700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe0380d01b0 con 0x7fe040107e70 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03d7fa700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] conn(0x7fe0280acde0 0x7fe0280af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 --> [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0280af8f0 con 0x7fe0280acde0 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0380d05c0 con 0x7fe040107e70 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03f7fe700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] conn(0x7fe0280acde0 0x7fe0280af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03ffff700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe030005fd0 tx=0x7fe03000af20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.982+0000 7fe03f7fe700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] conn(0x7fe0280acde0 0x7fe0280af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.983+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== osd.6 v2:172.21.15.17:6818/2957627579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe0280af8f0 con 0x7fe0280acde0 2024-02-14T15:01:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:42 smithi076 ceph-mon[37057]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-14T15:01:43.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:42.999+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 --> [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0400623b0 con 0x7fe0280acde0 2024-02-14T15:01:43.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe03d7fa700 1 -- 172.21.15.17:0/2181436520 <== osd.6 v2:172.21.15.17:6818/2957627579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe0400623b0 con 0x7fe0280acde0 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] conn(0x7fe0280acde0 msgr2=0x7fe0280af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] conn(0x7fe0280acde0 0x7fe0280af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 msgr2=0x7fe0280a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe030005fd0 tx=0x7fe03000af20 comp rx=0 tx=0).stop 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040107e70 msgr2=0x7fe04019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 --2- 172.21.15.17:0/2181436520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe040107e70 0x7fe04019ccb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0380079b0 tx=0x7fe038007a90 comp rx=0 tx=0).stop 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe044d07700 1 -- 172.21.15.17:0/2181436520 reap_dead start 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 shutdown_connections 2024-02-14T15:01:43.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 >> 172.21.15.17:0/2181436520 conn(0x7fe0400771f0 msgr2=0x7fe040104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:43.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 shutdown_connections 2024-02-14T15:01:43.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.000+0000 7fe04676a700 1 -- 172.21.15.17:0/2181436520 wait complete. 2024-02-14T15:01:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 -- 172.21.15.17:0/2293420791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 msgr2=0x7f76e810ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 --2- 172.21.15.17:0/2293420791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 0x7f76e810ebb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b0d0 tx=0x7f76e000b3e0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 -- 172.21.15.17:0/2293420791 shutdown_connections 2024-02-14T15:01:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 --2- 172.21.15.17:0/2293420791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 0x7f76e810ebb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 --2- 172.21.15.17:0/2293420791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e8107d70 0x7f76e8108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 --2- 172.21.15.17:0/2293420791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e80699d0 0x7f76e8069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 -- 172.21.15.17:0/2293420791 >> 172.21.15.17:0/2293420791 conn(0x7f76e8077b50 msgr2=0x7f76e8077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.437+0000 7f76efc21700 1 -- 172.21.15.17:0/2293420791 shutdown_connections 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 -- 172.21.15.17:0/2293420791 wait complete. 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 Processor -- start 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 -- start start 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e80699d0 0x7f76e8100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 0x7f76e8101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e8107d70 0x7f76e8101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e81a6680 con 0x7f76e806a370 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e81a67f0 con 0x7f76e80699d0 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76efc21700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e81a69b0 con 0x7f76e8107d70 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76ed1bc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 0x7f76e8101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76ed1bc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 0x7f76e8101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53396/0 (socket says 172.21.15.17:53396) 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76ed1bc700 1 -- 172.21.15.17:0/3007946882 learned_addr learned my addr 172.21.15.17:0/3007946882 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.438+0000 7f76ed9bd700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e80699d0 0x7f76e8100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76ee1be700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e8107d70 0x7f76e8101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76ed1bc700 1 -- 172.21.15.17:0/3007946882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e8107d70 msgr2=0x7f76e8101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76ed1bc700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e8107d70 0x7f76e8101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76ed1bc700 1 -- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e80699d0 msgr2=0x7f76e8100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76ed1bc700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76e80699d0 0x7f76e8100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76ed1bc700 1 -- 172.21.15.17:0/3007946882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0009d00 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76ed1bc700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 0x7f76e8101230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76d800d980 tx=0x7f76d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8004220 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76d8007640 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e81a6c90 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8005720 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.439+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e81a7190 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.440+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76e8111fe0 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.441+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f76d800b750 con 0x7f76e806a370 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76deffd700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76d40a21e0 0x7f76d40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:43.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76ed9bd700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76d40a21e0 0x7f76d40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f76d80ce4e0 con 0x7f76e806a370 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76deffd700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] conn(0x7f76d40acd60 0x7f76d40af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 --> [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76d40af870 con 0x7f76d40acd60 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d80ce850 con 0x7f76e806a370 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76ed9bd700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76d40a21e0 0x7f76d40a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76e400abd0 tx=0x7f76e400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.442+0000 7f76ee1be700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] conn(0x7f76d40acd60 0x7f76d40af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.443+0000 7f76ee1be700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] conn(0x7f76d40acd60 0x7f76d40af1a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.444+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== osd.1 v2:172.21.15.50:6800/2051103540 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f76d40af870 con 0x7f76d40acd60 2024-02-14T15:01:43.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.460+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 --> [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f76e8062430 con 0x7f76d40acd60 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76deffd700 1 -- 172.21.15.17:0/3007946882 <== osd.1 v2:172.21.15.50:6800/2051103540 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f76e8062430 con 0x7f76d40acd60 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] conn(0x7f76d40acd60 msgr2=0x7f76d40af1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] conn(0x7f76d40acd60 0x7f76d40af1a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76d40a21e0 msgr2=0x7f76d40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f76d40a21e0 0x7f76d40a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76e400abd0 tx=0x7f76e400a380 comp rx=0 tx=0).stop 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 msgr2=0x7f76e8101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 --2- 172.21.15.17:0/3007946882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76e806a370 0x7f76e8101230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76d800d980 tx=0x7f76d800dc90 comp rx=0 tx=0).stop 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76ee1be700 1 -- 172.21.15.17:0/3007946882 reap_dead start 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 shutdown_connections 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 >> 172.21.15.17:0/3007946882 conn(0x7f76e8077b50 msgr2=0x7f76e80ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 shutdown_connections 2024-02-14T15:01:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:43.461+0000 7f76efc21700 1 -- 172.21.15.17:0/3007946882 wait complete. 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-02-14T15:01:43.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:43 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-02-14T15:01:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-02-14T15:01:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-14T15:01:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-02-14T15:01:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-14T15:01:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-02-14T15:01:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-02-14T15:01:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]: dispatch 2024-02-14T15:01:43.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-02-14T15:01:43.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:43 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-02-14T15:01:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:43 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-02-14T15:01:44.678 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:44 smithi017 ceph-mon[32549]: osdmap e90: 12 total, 12 up, 12 in 2024-02-14T15:01:44.705 INFO:teuthology.orchestra.run.smithi017.stdout:197568495666 2024-02-14T15:01:44.705 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]': finished 2024-02-14T15:01:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-02-14T15:01:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-02-14T15:01:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:44 smithi050 ceph-mon[36512]: osdmap e90: 12 total, 12 up, 12 in 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9, 4, 6]}]': finished 2024-02-14T15:01:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-02-14T15:01:44.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-02-14T15:01:44.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:44 smithi076 ceph-mon[37057]: osdmap e90: 12 total, 12 up, 12 in 2024-02-14T15:01:45.752 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:45 smithi017 ceph-mon[32549]: osdmap e91: 12 total, 12 up, 12 in 2024-02-14T15:01:45.752 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:45 smithi017 ceph-mon[32549]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 5 op/s 2024-02-14T15:01:45.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:45 smithi050 ceph-mon[36512]: osdmap e91: 12 total, 12 up, 12 in 2024-02-14T15:01:45.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:45 smithi050 ceph-mon[36512]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 5 op/s 2024-02-14T15:01:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:45 smithi076 ceph-mon[37057]: osdmap e91: 12 total, 12 up, 12 in 2024-02-14T15:01:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:45 smithi076 ceph-mon[37057]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 5 op/s 2024-02-14T15:01:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:47 smithi076 ceph-mon[37057]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-14T15:01:47.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:47 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:01:48.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:47 smithi017 ceph-mon[32549]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-14T15:01:48.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:47 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:01:48.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:47 smithi050 ceph-mon[36512]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s; 72 B/s, 0 objects/s recovering 2024-02-14T15:01:48.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:47 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:01:48.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 -- 172.21.15.17:0/3185806175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0699d0 msgr2=0x7fcefc069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 --2- 172.21.15.17:0/3185806175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0699d0 0x7fcefc069e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcef00099c0 tx=0x7fcef0009cd0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 -- 172.21.15.17:0/3185806175 shutdown_connections 2024-02-14T15:01:48.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 --2- 172.21.15.17:0/3185806175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcefc06a370 0x7fcefc10ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 --2- 172.21.15.17:0/3185806175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcefc107d70 0x7fcefc108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 --2- 172.21.15.17:0/3185806175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0699d0 0x7fcefc069e30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 -- 172.21.15.17:0/3185806175 >> 172.21.15.17:0/3185806175 conn(0x7fcefc077b50 msgr2=0x7fcefc077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 -- 172.21.15.17:0/3185806175 shutdown_connections 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.650+0000 7fcf03e1b700 1 -- 172.21.15.17:0/3185806175 wait complete. 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 Processor -- start 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 -- start start 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcefc0699d0 0x7fcefc100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcefc06a370 0x7fcefc101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc107d70 0x7fcefc101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1a6600 con 0x7fcefc107d70 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1a6770 con 0x7fcefc06a370 2024-02-14T15:01:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf03e1b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1a6930 con 0x7fcefc0699d0 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc107d70 0x7fcefc101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc107d70 0x7fcefc101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53432/0 (socket says 172.21.15.17:53432) 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 -- 172.21.15.17:0/48456154 learned_addr learned my addr 172.21.15.17:0/48456154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf01bb7700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcefc0699d0 0x7fcefc100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf013b6700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcefc06a370 0x7fcefc101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 -- 172.21.15.17:0/48456154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcefc0699d0 msgr2=0x7fcefc100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcefc0699d0 0x7fcefc100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 -- 172.21.15.17:0/48456154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcefc06a370 msgr2=0x7fcefc101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcefc06a370 0x7fcefc101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.651+0000 7fcf023b8700 1 -- 172.21.15.17:0/48456154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00095f0 con 0x7fcefc107d70 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.652+0000 7fcf023b8700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc107d70 0x7fcefc101770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcef400e940 tx=0x7fcef400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.652+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40097b0 con 0x7fcefc107d70 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.652+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc1a6c10 con 0x7fcefc107d70 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.652+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcef4005020 con 0x7fcefc107d70 2024-02-14T15:01:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.652+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc1a70e0 con 0x7fcefc107d70 2024-02-14T15:01:48.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.652+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef400ccf0 con 0x7fcefc107d70 2024-02-14T15:01:48.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.653+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcefc111fa0 con 0x7fcefc107d70 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.654+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fcef4005490 con 0x7fcefc107d70 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.654+0000 7fcefaffd700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcee40a21e0 0x7fcee40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcf01bb7700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcee40a21e0 0x7fcee40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcef4014070 con 0x7fcefc107d70 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcefaffd700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] conn(0x7fcee40ad180 0x7fcee40af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 --> [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcee40afcf0 con 0x7fcee40ad180 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcef40ce9d0 con 0x7fcefc107d70 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcf013b6700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] conn(0x7fcee40ad180 0x7fcee40af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcf01bb7700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcee40a21e0 0x7fcee40a46a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fceec009550 tx=0x7fceec0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.655+0000 7fcf013b6700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] conn(0x7fcee40ad180 0x7fcee40af620 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:48.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.656+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== osd.0 v2:172.21.15.17:6802/1615583912 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcee40afcf0 con 0x7fcee40ad180 2024-02-14T15:01:48.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.672+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 --> [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcefc062430 con 0x7fcee40ad180 2024-02-14T15:01:48.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcefaffd700 1 -- 172.21.15.17:0/48456154 <== osd.0 v2:172.21.15.17:6802/1615583912 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcefc062430 con 0x7fcee40ad180 2024-02-14T15:01:48.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] conn(0x7fcee40ad180 msgr2=0x7fcee40af620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] conn(0x7fcee40ad180 0x7fcee40af620 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcee40a21e0 msgr2=0x7fcee40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcee40a21e0 0x7fcee40a46a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fceec009550 tx=0x7fceec0091b0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc107d70 msgr2=0x7fcefc101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 --2- 172.21.15.17:0/48456154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc107d70 0x7fcefc101770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcef400e940 tx=0x7fcef400ec50 comp rx=0 tx=0).stop 2024-02-14T15:01:48.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf023b8700 1 -- 172.21.15.17:0/48456154 reap_dead start 2024-02-14T15:01:48.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 shutdown_connections 2024-02-14T15:01:48.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 >> 172.21.15.17:0/48456154 conn(0x7fcefc077b50 msgr2=0x7fcefc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:48.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 shutdown_connections 2024-02-14T15:01:48.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.673+0000 7fcf03e1b700 1 -- 172.21.15.17:0/48456154 wait complete. 2024-02-14T15:01:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 -- 172.21.15.17:0/3393723870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 msgr2=0x7f1824107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 --2- 172.21.15.17:0/3393723870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 0x7f1824107170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1820009990 tx=0x7f1820009ca0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 -- 172.21.15.17:0/3393723870 shutdown_connections 2024-02-14T15:01:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 --2- 172.21.15.17:0/3393723870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1824100cd0 0x7f182410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 --2- 172.21.15.17:0/3393723870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 0x7f1824107170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 --2- 172.21.15.17:0/3393723870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1824100310 0x7f1824100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 -- 172.21.15.17:0/3393723870 >> 172.21.15.17:0/3393723870 conn(0x7f1824077220 msgr2=0x7f18240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 -- 172.21.15.17:0/3393723870 shutdown_connections 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.543+0000 7f182bd60700 1 -- 172.21.15.17:0/3393723870 wait complete. 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 Processor -- start 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 -- start start 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1824100310 0x7f1824198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1824100cd0 0x7f18241989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 0x7f182419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182419d2f0 con 0x7f1824106d90 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182419d460 con 0x7f1824100310 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182bd60700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182419d620 con 0x7f1824100cd0 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 0x7f182419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 0x7f182419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53422/0 (socket says 172.21.15.17:53422) 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 -- 172.21.15.17:0/2493945784 learned_addr learned my addr 172.21.15.17:0/2493945784 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f1829afc700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1824100310 0x7f1824198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f18292fb700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1824100cd0 0x7f18241989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 -- 172.21.15.17:0/2493945784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1824100cd0 msgr2=0x7f18241989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1824100cd0 0x7f18241989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 -- 172.21.15.17:0/2493945784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1824100310 msgr2=0x7f1824198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1824100310 0x7f1824198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 -- 172.21.15.17:0/2493945784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18200095f0 con 0x7f1824106d90 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f1829afc700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1824100310 0x7f1824198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.544+0000 7f182a2fd700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 0x7f182419cce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f181c0094f0 tx=0x7f181c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.545+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c004db0 con 0x7f1824106d90 2024-02-14T15:01:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.545+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f181c00f040 con 0x7f1824106d90 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.545+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182419d900 con 0x7f1824106d90 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.545+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c005aa0 con 0x7f1824106d90 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.545+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18241a5160 con 0x7f1824106d90 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.545+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18241121e0 con 0x7f1824106d90 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.547+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f181c00dbf0 con 0x7f1824106d90 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.547+0000 7f1816ffd700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f18100a2260 0x7f18100a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.547+0000 7f1829afc700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f18100a2260 0x7f18100a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.547+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f181c0ce670 con 0x7f1824106d90 2024-02-14T15:01:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.547+0000 7f1816ffd700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] conn(0x7f18100ad200 0x7f18100af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.547+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 --> [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18100afd70 con 0x7f18100ad200 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.547+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f181c0cea80 con 0x7f1824106d90 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.548+0000 7f18292fb700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] conn(0x7f18100ad200 0x7f18100af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.548+0000 7f1829afc700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f18100a2260 0x7f18100a4720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1820006010 tx=0x7f182000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.548+0000 7f18292fb700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] conn(0x7f18100ad200 0x7f18100af6a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.549+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== osd.3 v2:172.21.15.17:6810/1293297743 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f18100afd70 con 0x7f18100ad200 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 --> [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18240623b0 con 0x7f18100ad200 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f1816ffd700 1 -- 172.21.15.17:0/2493945784 <== osd.3 v2:172.21.15.17:6810/1293297743 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f18240623b0 con 0x7f18100ad200 2024-02-14T15:01:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] conn(0x7f18100ad200 msgr2=0x7f18100af6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] conn(0x7f18100ad200 0x7f18100af6a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f18100a2260 msgr2=0x7f18100a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f18100a2260 0x7f18100a4720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1820006010 tx=0x7f182000b540 comp rx=0 tx=0).stop 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 msgr2=0x7f182419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 --2- 172.21.15.17:0/2493945784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1824106d90 0x7f182419cce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f181c0094f0 tx=0x7f181c007f90 comp rx=0 tx=0).stop 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182a2fd700 1 -- 172.21.15.17:0/2493945784 reap_dead start 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 shutdown_connections 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.565+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 >> 172.21.15.17:0/2493945784 conn(0x7f1824077220 msgr2=0x7f1824104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.566+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 shutdown_connections 2024-02-14T15:01:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:46.566+0000 7f182bd60700 1 -- 172.21.15.17:0/2493945784 wait complete. 2024-02-14T15:01:49.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 -- 172.21.15.17:0/3487490138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c101d70 msgr2=0x7fe03c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 --2- 172.21.15.17:0/3487490138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c101d70 0x7fe03c1021f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe0240099c0 tx=0x7fe024009cd0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 -- 172.21.15.17:0/3487490138 shutdown_connections 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 --2- 172.21.15.17:0/3487490138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe03c102730 0x7fe03c10eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 --2- 172.21.15.17:0/3487490138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe03c107d50 0x7fe03c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 --2- 172.21.15.17:0/3487490138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c101d70 0x7fe03c1021f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 -- 172.21.15.17:0/3487490138 >> 172.21.15.17:0/3487490138 conn(0x7fe03c078e50 msgr2=0x7fe03c079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 -- 172.21.15.17:0/3487490138 shutdown_connections 2024-02-14T15:01:49.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 -- 172.21.15.17:0/3487490138 wait complete. 2024-02-14T15:01:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 Processor -- start 2024-02-14T15:01:49.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.288+0000 7fe040be8700 1 -- start start 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe040be8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe03c101d70 0x7fe03c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe040be8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c102730 0x7fe03c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe040be8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe03c107d50 0x7fe03c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe040be8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c19d0f0 con 0x7fe03c102730 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe040be8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c19d260 con 0x7fe03c107d50 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe040be8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c19d420 con 0x7fe03c101d70 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe033fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c102730 0x7fe03c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe03c101d70 0x7fe03c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe033fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c102730 0x7fe03c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53482/0 (socket says 172.21.15.17:53482) 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03ad9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe03c107d50 0x7fe03c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe033fff700 1 -- 172.21.15.17:0/3907212660 learned_addr learned my addr 172.21.15.17:0/3907212660 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:49.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe03c101d70 0x7fe03c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:57580/0 (socket says 172.21.15.17:57580) 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 -- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe03c107d50 msgr2=0x7fe03c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe03c107d50 0x7fe03c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 -- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c102730 msgr2=0x7fe03c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c102730 0x7fe03c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 -- 172.21.15.17:0/3907212660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240095f0 con 0x7fe03c101d70 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe033fff700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c102730 0x7fe03c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03ad9d700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe03c107d50 0x7fe03c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.289+0000 7fe03a59c700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe03c101d70 0x7fe03c198260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00e9f0 tx=0x7fe02c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.290+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00c9c0 con 0x7fe03c101d70 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.290+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe02c010430 con 0x7fe03c101d70 2024-02-14T15:01:49.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.290+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c19d700 con 0x7fe03c101d70 2024-02-14T15:01:49.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.290+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c018680 con 0x7fe03c101d70 2024-02-14T15:01:49.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.290+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c1a4f60 con 0x7fe03c101d70 2024-02-14T15:01:49.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.290+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe03c112020 con 0x7fe03c101d70 2024-02-14T15:01:49.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.292+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe02c010910 con 0x7fe03c101d70 2024-02-14T15:01:49.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.292+0000 7fe0337fe700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.292+0000 7fe033fff700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.292+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe02c014070 con 0x7fe03c101d70 2024-02-14T15:01:49.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.292+0000 7fe0337fe700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] conn(0x7fe0280ad200 0x7fe0280af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.293+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 --> [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0280afd70 con 0x7fe0280ad200 2024-02-14T15:01:49.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.293+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe02c0cec00 con 0x7fe03c101d70 2024-02-14T15:01:49.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.293+0000 7fe03ad9d700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] conn(0x7fe0280ad200 0x7fe0280af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.293+0000 7fe033fff700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe03c106c60 tx=0x7fe024005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.293+0000 7fe03ad9d700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] conn(0x7fe0280ad200 0x7fe0280af6a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.294+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== osd.4 v2:172.21.15.50:6808/4058610796 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe0280afd70 con 0x7fe0280ad200 2024-02-14T15:01:49.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.311+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 --> [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe03c0623b0 con 0x7fe0280ad200 2024-02-14T15:01:49.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.311+0000 7fe0337fe700 1 -- 172.21.15.17:0/3907212660 <== osd.4 v2:172.21.15.50:6808/4058610796 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe03c0623b0 con 0x7fe0280ad200 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.311+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] conn(0x7fe0280ad200 msgr2=0x7fe0280af6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.311+0000 7fe040be8700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] conn(0x7fe0280ad200 0x7fe0280af6a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.311+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 msgr2=0x7fe0280a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.311+0000 7fe040be8700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe0280a2260 0x7fe0280a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe03c106c60 tx=0x7fe024005fb0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.312+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe03c101d70 msgr2=0x7fe03c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.312+0000 7fe040be8700 1 --2- 172.21.15.17:0/3907212660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe03c101d70 0x7fe03c198260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00e9f0 tx=0x7fe02c00ed00 comp rx=0 tx=0).stop 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.312+0000 7fe03ad9d700 1 -- 172.21.15.17:0/3907212660 reap_dead start 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.312+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 shutdown_connections 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.312+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 >> 172.21.15.17:0/3907212660 conn(0x7fe03c078e50 msgr2=0x7fe03c100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.312+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 shutdown_connections 2024-02-14T15:01:49.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.312+0000 7fe040be8700 1 -- 172.21.15.17:0/3907212660 wait complete. 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 -- 172.21.15.17:0/1734487550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 msgr2=0x7f485010ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 --2- 172.21.15.17:0/1734487550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 0x7f485010ebd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f484800b0a0 tx=0x7f484800b3b0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 -- 172.21.15.17:0/1734487550 shutdown_connections 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 --2- 172.21.15.17:0/1734487550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 0x7f485010ebd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 --2- 172.21.15.17:0/1734487550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4850107e20 0x7f4850108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 --2- 172.21.15.17:0/1734487550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4850101d70 0x7f48501021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 -- 172.21.15.17:0/1734487550 >> 172.21.15.17:0/1734487550 conn(0x7f4850078e50 msgr2=0x7f4850079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 -- 172.21.15.17:0/1734487550 shutdown_connections 2024-02-14T15:01:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.875+0000 7f48586da700 1 -- 172.21.15.17:0/1734487550 wait complete. 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 Processor -- start 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 -- start start 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4850101d70 0x7f48501982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 0x7f4850198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4850107e20 0x7f485019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485019d1c0 con 0x7f4850102730 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485019d330 con 0x7f4850107e20 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f48586da700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485019d4f0 con 0x7f4850101d70 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 0x7f4850198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4856476700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4850101d70 0x7f48501982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 0x7f4850198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53460/0 (socket says 172.21.15.17:53460) 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4856c77700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4850107e20 0x7f485019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 -- 172.21.15.17:0/1609249652 learned_addr learned my addr 172.21.15.17:0/1609249652 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 -- 172.21.15.17:0/1609249652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4850101d70 msgr2=0x7f48501982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4850101d70 0x7f48501982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 -- 172.21.15.17:0/1609249652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4850107e20 msgr2=0x7f485019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4850107e20 0x7f485019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4855c75700 1 -- 172.21.15.17:0/1609249652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4848009d00 con 0x7f4850102730 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.876+0000 7f4856c77700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4850107e20 0x7f485019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.877+0000 7f4855c75700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 0x7f4850198820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f484000d980 tx=0x7f484000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.877+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840004220 con 0x7f4850102730 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.877+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485019d7d0 con 0x7f4850102730 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.877+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4840007640 con 0x7f4850102730 2024-02-14T15:01:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.877+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484000f690 con 0x7f4850102730 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.877+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48501a5030 con 0x7f4850102730 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.877+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4850112040 con 0x7f4850102730 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.879+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4840004410 con 0x7f4850102730 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.879+0000 7f48477fe700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f483c0a21e0 0x7f483c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f4856476700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f483c0a21e0 0x7f483c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f48400ce570 con 0x7f4850102730 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f48477fe700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] conn(0x7f483c0ad180 0x7f483c0af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 --> [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f483c0afcf0 con 0x7f483c0ad180 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48400ce980 con 0x7f4850102730 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f4856476700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f483c0a21e0 0x7f483c0a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f484c009be0 tx=0x7f484c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f4856c77700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] conn(0x7f483c0ad180 0x7f483c0af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.880+0000 7f4856c77700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] conn(0x7f483c0ad180 0x7f483c0af620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.881+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== osd.11 v2:172.21.15.76:6824/3113998070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f483c0afcf0 con 0x7f483c0ad180 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.897+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 --> [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4850062430 con 0x7f483c0ad180 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48477fe700 1 -- 172.21.15.17:0/1609249652 <== osd.11 v2:172.21.15.76:6824/3113998070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4850062430 con 0x7f483c0ad180 2024-02-14T15:01:49.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 >> [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] conn(0x7f483c0ad180 msgr2=0x7f483c0af620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] conn(0x7f483c0ad180 0x7f483c0af620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f483c0a21e0 msgr2=0x7f483c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f483c0a21e0 0x7f483c0a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f484c009be0 tx=0x7f484c009260 comp rx=0 tx=0).stop 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 msgr2=0x7f4850198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 --2- 172.21.15.17:0/1609249652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4850102730 0x7f4850198820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f484000d980 tx=0x7f484000dc90 comp rx=0 tx=0).stop 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f4856c77700 1 -- 172.21.15.17:0/1609249652 reap_dead start 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 shutdown_connections 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 >> 172.21.15.17:0/1609249652 conn(0x7f4850078e50 msgr2=0x7f4850100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 shutdown_connections 2024-02-14T15:01:49.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:48.898+0000 7f48586da700 1 -- 172.21.15.17:0/1609249652 wait complete. 2024-02-14T15:01:49.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 -- 172.21.15.17:0/1010709352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 msgr2=0x7f6da806a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 --2- 172.21.15.17:0/1010709352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 0x7f6da806a8e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009990 tx=0x7f6d94009ca0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 -- 172.21.15.17:0/1010709352 shutdown_connections 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 --2- 172.21.15.17:0/1010709352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6da806ae20 0x7f6da810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 --2- 172.21.15.17:0/1010709352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da8107f00 0x7f6da81082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 --2- 172.21.15.17:0/1010709352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 0x7f6da806a8e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 -- 172.21.15.17:0/1010709352 >> 172.21.15.17:0/1010709352 conn(0x7f6da8077a80 msgr2=0x7f6da8077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 -- 172.21.15.17:0/1010709352 shutdown_connections 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.924+0000 7f6daee07700 1 -- 172.21.15.17:0/1010709352 wait complete. 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 Processor -- start 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 -- start start 2024-02-14T15:01:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 0x7f6da8100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da806ae20 0x7f6da8101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6da8107f00 0x7f6da8101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81a66a0 con 0x7f6da806a460 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81a6810 con 0x7f6da806ae20 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6daee07700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81a69d0 con 0x7f6da8107f00 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 0x7f6da8100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 0x7f6da8100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53492/0 (socket says 172.21.15.17:53492) 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dad3a4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6da8107f00 0x7f6da8101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 -- 172.21.15.17:0/64126700 learned_addr learned my addr 172.21.15.17:0/64126700 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6da7fff700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da806ae20 0x7f6da8101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 -- 172.21.15.17:0/64126700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6da8107f00 msgr2=0x7f6da8101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6da8107f00 0x7f6da8101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 -- 172.21.15.17:0/64126700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da806ae20 msgr2=0x7f6da8101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6da806ae20 0x7f6da8101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 -- 172.21.15.17:0/64126700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940095f0 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.925+0000 7f6dacba3700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 0x7f6da8100fd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00d980 tx=0x7f6d9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.926+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c004020 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.926+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d9c00f690 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.926+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da81a6cb0 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.926+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c003b40 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.926+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da81a7180 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.926+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6da8112050 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.928+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6d9c00fb20 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.928+0000 7f6da5ffb700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6d980a2260 0x7f6d980a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.928+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6d9c0ce2f0 con 0x7f6da806a460 2024-02-14T15:01:49.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.928+0000 7f6da7fff700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6d980a2260 0x7f6d980a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.929+0000 7f6da5ffb700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] conn(0x7f6d980ad200 0x7f6d980af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.929+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 --> [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d980afd70 con 0x7f6d980ad200 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.929+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9c0ce790 con 0x7f6da806a460 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.929+0000 7f6da7fff700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6d980a2260 0x7f6d980a4720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400b2e0 tx=0x7f6d94005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.929+0000 7f6dad3a4700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] conn(0x7f6d980ad200 0x7f6d980af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.929+0000 7f6dad3a4700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] conn(0x7f6d980ad200 0x7f6d980af6a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.930+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== osd.2 v2:172.21.15.76:6800/2271935501 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6d980afd70 con 0x7f6d980ad200 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.946+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 --> [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6da80623b0 con 0x7f6d980ad200 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6da5ffb700 1 -- 172.21.15.17:0/64126700 <== osd.2 v2:172.21.15.76:6800/2271935501 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6da80623b0 con 0x7f6d980ad200 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 >> [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] conn(0x7f6d980ad200 msgr2=0x7f6d980af6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] conn(0x7f6d980ad200 0x7f6d980af6a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6d980a2260 msgr2=0x7f6d980a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6d980a2260 0x7f6d980a4720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400b2e0 tx=0x7f6d94005960 comp rx=0 tx=0).stop 2024-02-14T15:01:49.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 msgr2=0x7f6da8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:49.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 --2- 172.21.15.17:0/64126700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6da806a460 0x7f6da8100fd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00d980 tx=0x7f6d9c00dc90 comp rx=0 tx=0).stop 2024-02-14T15:01:49.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6dad3a4700 1 -- 172.21.15.17:0/64126700 reap_dead start 2024-02-14T15:01:49.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 shutdown_connections 2024-02-14T15:01:49.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 >> 172.21.15.17:0/64126700 conn(0x7f6da8077a80 msgr2=0x7f6da80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:49.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 shutdown_connections 2024-02-14T15:01:49.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:49.947+0000 7f6daee07700 1 -- 172.21.15.17:0/64126700 wait complete. 2024-02-14T15:01:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:50 smithi017 ceph-mon[32549]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s; 7.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:50 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:50 smithi050 ceph-mon[36512]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s; 7.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:50 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 -- 172.21.15.17:0/2456215078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 msgr2=0x7f793c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 --2- 172.21.15.17:0/2456215078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 0x7f793c101290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f792c009990 tx=0x7f792c009ca0 comp rx=0 tx=0).stop 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 -- 172.21.15.17:0/2456215078 shutdown_connections 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 --2- 172.21.15.17:0/2456215078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f793c105f00 0x7f793c10dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 --2- 172.21.15.17:0/2456215078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 0x7f793c101290 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 --2- 172.21.15.17:0/2456215078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f793c101860 0x7f793c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 -- 172.21.15.17:0/2456215078 >> 172.21.15.17:0/2456215078 conn(0x7f793c076c70 msgr2=0x7f793c077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 -- 172.21.15.17:0/2456215078 shutdown_connections 2024-02-14T15:01:50.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.480+0000 7f7941581700 1 -- 172.21.15.17:0/2456215078 wait complete. 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 Processor -- start 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 -- start start 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 0x7f793c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f793c101860 0x7f793c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f793c105f00 0x7f793c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c19f330 con 0x7f793c100eb0 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c19f4a0 con 0x7f793c101860 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f7941581700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c19f660 con 0x7f793c105f00 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.481+0000 7f793affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 0x7f793c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 0x7f793c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53504/0 (socket says 172.21.15.17:53504) 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 -- 172.21.15.17:0/173581378 learned_addr learned my addr 172.21.15.17:0/173581378 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:50.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793b7fe700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f793c105f00 0x7f793c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793a7fc700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f793c101860 0x7f793c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 -- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f793c105f00 msgr2=0x7f793c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f793c105f00 0x7f793c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 -- 172.21.15.17:0/173581378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f793c101860 msgr2=0x7f793c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f793c101860 0x7f793c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 -- 172.21.15.17:0/173581378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0095f0 con 0x7f793c100eb0 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793a7fc700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f793c101860 0x7f793c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f793affd700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 0x7f793c19a450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f792c0098e0 tx=0x7f792c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c01d070 con 0x7f793c100eb0 2024-02-14T15:01:50.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c19f8e0 con 0x7f793c100eb0 2024-02-14T15:01:50.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:50 smithi076 ceph-mon[37057]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s; 7.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:50.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.485 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:50 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:01:50.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.482+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c1a71a0 con 0x7f793c100eb0 2024-02-14T15:01:50.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.483+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f792c00fbc0 con 0x7f793c100eb0 2024-02-14T15:01:50.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.483+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c017650 con 0x7f793c100eb0 2024-02-14T15:01:50.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.483+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7928001030 con 0x7f793c100eb0 2024-02-14T15:01:50.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.485+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f792c022470 con 0x7f793c100eb0 2024-02-14T15:01:50.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.485+0000 7f7923fff700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79240a2260 0x7f79240a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:50.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.485+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f792c0cfe50 con 0x7f793c100eb0 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.485+0000 7f793a7fc700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79240a2260 0x7f79240a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.485+0000 7f7923fff700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] conn(0x7f79240ad200 0x7f79240af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.485+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 --> [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79240afd70 con 0x7f79240ad200 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.485+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f792c0d0260 con 0x7f793c100eb0 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.486+0000 7f793b7fe700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] conn(0x7f79240ad200 0x7f79240af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.486+0000 7f793a7fc700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79240a2260 0x7f79240a4720 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f793c100b30 tx=0x7f7930005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.486+0000 7f793b7fe700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] conn(0x7f79240ad200 0x7f79240af6a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:50.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.487+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== osd.8 v2:172.21.15.76:6816/416823437 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f79240afd70 con 0x7f79240ad200 2024-02-14T15:01:50.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.505+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 --> [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7928002dc0 con 0x7f79240ad200 2024-02-14T15:01:50.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7923fff700 1 -- 172.21.15.17:0/173581378 <== osd.8 v2:172.21.15.76:6816/416823437 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7928002dc0 con 0x7f79240ad200 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] conn(0x7f79240ad200 msgr2=0x7f79240af6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] conn(0x7f79240ad200 0x7f79240af6a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79240a2260 msgr2=0x7f79240a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79240a2260 0x7f79240a4720 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f793c100b30 tx=0x7f7930005f50 comp rx=0 tx=0).stop 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 msgr2=0x7f793c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 --2- 172.21.15.17:0/173581378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c100eb0 0x7f793c19a450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f792c0098e0 tx=0x7f792c00f690 comp rx=0 tx=0).stop 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f793b7fe700 1 -- 172.21.15.17:0/173581378 reap_dead start 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 shutdown_connections 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 >> 172.21.15.17:0/173581378 conn(0x7f793c076c70 msgr2=0x7f793c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:50.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 shutdown_connections 2024-02-14T15:01:50.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.506+0000 7f7941581700 1 -- 172.21.15.17:0/173581378 wait complete. 2024-02-14T15:01:51.814 INFO:teuthology.orchestra.run.smithi017.stdout:55834574923 2024-02-14T15:01:51.814 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-14T15:01:52.265 INFO:teuthology.orchestra.run.smithi017.stdout:107374182465 2024-02-14T15:01:52.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-14T15:01:52.673 INFO:teuthology.orchestra.run.smithi017.stdout:141733920828 2024-02-14T15:01:52.673 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-14T15:01:52.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:52 smithi050 ceph-mon[36512]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 6 op/s; 7.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:52.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:52 smithi076 ceph-mon[37057]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 6 op/s; 7.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:52 smithi017 ceph-mon[32549]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 6 op/s; 7.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:53.080 INFO:teuthology.orchestra.run.smithi017.stdout:343597383704 2024-02-14T15:01:53.080 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-14T15:01:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.553+0000 7f9eaee11700 1 -- 172.21.15.17:0/1111551771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 msgr2=0x7f9ea8108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.553+0000 7f9eaee11700 1 --2- 172.21.15.17:0/1111551771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 0x7f9ea8108150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0099c0 tx=0x7f9e9c009cd0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.553+0000 7f9eaee11700 1 -- 172.21.15.17:0/1111551771 shutdown_connections 2024-02-14T15:01:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.553+0000 7f9eaee11700 1 --2- 172.21.15.17:0/1111551771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ea806a370 0x7f9ea810ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.553+0000 7f9eaee11700 1 --2- 172.21.15.17:0/1111551771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 0x7f9ea8108150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.553+0000 7f9eaee11700 1 --2- 172.21.15.17:0/1111551771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ea80699d0 0x7f9ea8069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.553+0000 7f9eaee11700 1 -- 172.21.15.17:0/1111551771 >> 172.21.15.17:0/1111551771 conn(0x7f9ea8077b50 msgr2=0x7f9ea8077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 -- 172.21.15.17:0/1111551771 shutdown_connections 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 -- 172.21.15.17:0/1111551771 wait complete. 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 Processor -- start 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 -- start start 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ea80699d0 0x7f9ea8100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ea806a370 0x7f9ea8101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 0x7f9ea8101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81a6610 con 0x7f9ea8107d70 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81a6780 con 0x7f9ea806a370 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9eaee11700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81a6940 con 0x7f9ea80699d0 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9ead3ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 0x7f9ea8101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9ead3ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 0x7f9ea8101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38272/0 (socket says 172.21.15.17:38272) 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.554+0000 7f9ead3ae700 1 -- 172.21.15.17:0/1565161157 learned_addr learned my addr 172.21.15.17:0/1565161157 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9eacbad700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ea80699d0 0x7f9ea8100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:53.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ea7fff700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ea806a370 0x7f9ea8101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ead3ae700 1 -- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ea80699d0 msgr2=0x7f9ea8100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ead3ae700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ea80699d0 0x7f9ea8100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ead3ae700 1 -- 172.21.15.17:0/1565161157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ea806a370 msgr2=0x7f9ea8101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ead3ae700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ea806a370 0x7f9ea8101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ead3ae700 1 -- 172.21.15.17:0/1565161157 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0095f0 con 0x7f9ea8107d70 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9eacbad700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ea80699d0 0x7f9ea8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ead3ae700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 0x7f9ea8101770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000ada0 tx=0x7f9ea000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000f070 con 0x7f9ea8107d70 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea81a6c20 con 0x7f9ea8107d70 2024-02-14T15:01:53.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ea0007b30 con 0x7f9ea8107d70 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0005500 con 0x7f9ea8107d70 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.555+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea81a70d0 con 0x7f9ea8107d70 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.556+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ea8111fc0 con 0x7f9ea8107d70 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.557+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f9ea0004ad0 con 0x7f9ea8107d70 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9ea5ffb700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9e980a21e0 0x7f9e980a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9eacbad700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9e980a21e0 0x7f9e980a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9ea00cd830 con 0x7f9ea8107d70 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9ea5ffb700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] conn(0x7f9e980ad180 0x7f9e980af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 --> [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e980afcf0 con 0x7f9e980ad180 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea00cfe10 con 0x7f9ea8107d70 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9eacbad700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9e980a21e0 0x7f9e980a46a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005b40 tx=0x7f9e9c005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.558+0000 7f9ea7fff700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] conn(0x7f9e980ad180 0x7f9e980af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.559+0000 7f9ea7fff700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] conn(0x7f9e980ad180 0x7f9e980af620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.559+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== osd.5 v2:172.21.15.76:6808/75527481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9e980afcf0 con 0x7f9e980ad180 2024-02-14T15:01:53.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.576+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 --> [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ea80623e0 con 0x7f9e980ad180 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.576+0000 7f9ea5ffb700 1 -- 172.21.15.17:0/1565161157 <== osd.5 v2:172.21.15.76:6808/75527481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9ea80623e0 con 0x7f9e980ad180 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.576+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] conn(0x7f9e980ad180 msgr2=0x7f9e980af620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.576+0000 7f9eaee11700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] conn(0x7f9e980ad180 0x7f9e980af620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9e980a21e0 msgr2=0x7f9e980a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9e980a21e0 0x7f9e980a46a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005b40 tx=0x7f9e9c005ab0 comp rx=0 tx=0).stop 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 msgr2=0x7f9ea8101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 --2- 172.21.15.17:0/1565161157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea8107d70 0x7f9ea8101770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000ada0 tx=0x7f9ea000c330 comp rx=0 tx=0).stop 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9ead3ae700 1 -- 172.21.15.17:0/1565161157 reap_dead start 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 shutdown_connections 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 >> 172.21.15.17:0/1565161157 conn(0x7f9ea8077b50 msgr2=0x7f9ea80ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 shutdown_connections 2024-02-14T15:01:53.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:51.577+0000 7f9eaee11700 1 -- 172.21.15.17:0/1565161157 wait complete. 2024-02-14T15:01:54.532 INFO:teuthology.orchestra.run.smithi017.stdout:34359738448 2024-02-14T15:01:54.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-14T15:01:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:54 smithi017 ceph-mon[32549]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 338 B/s wr, 5 op/s; 6.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:54.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:54 smithi050 ceph-mon[36512]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 338 B/s wr, 5 op/s; 6.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:54.904 INFO:teuthology.orchestra.run.smithi017.stdout:253403070504 2024-02-14T15:01:54.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-14T15:01:54.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:54 smithi076 ceph-mon[37057]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 338 B/s wr, 5 op/s; 6.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:55.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 -- 172.21.15.17:0/2565794676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0100eb0 msgr2=0x7fabb0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 --2- 172.21.15.17:0/2565794676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0100eb0 0x7fabb0101290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faba0009990 tx=0x7faba0009ca0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 -- 172.21.15.17:0/2565794676 shutdown_connections 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 --2- 172.21.15.17:0/2565794676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabb0105f00 0x7fabb010dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 --2- 172.21.15.17:0/2565794676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0100eb0 0x7fabb0101290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 --2- 172.21.15.17:0/2565794676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb0101860 0x7fabb0105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 -- 172.21.15.17:0/2565794676 >> 172.21.15.17:0/2565794676 conn(0x7fabb0076c70 msgr2=0x7fabb0077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 -- 172.21.15.17:0/2565794676 shutdown_connections 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.696+0000 7fabb4f75700 1 -- 172.21.15.17:0/2565794676 wait complete. 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 Processor -- start 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 -- start start 2024-02-14T15:01:55.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb0100eb0 0x7fabb0198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabb0101860 0x7fabb0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0105f00 0x7fabb019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb019d030 con 0x7fabb0105f00 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb019d1a0 con 0x7fabb0101860 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabb4f75700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb019d360 con 0x7fabb0100eb0 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0105f00 0x7fabb019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabae59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb0100eb0 0x7fabb0198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0105f00 0x7fabb019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38262/0 (socket says 172.21.15.17:38262) 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7fabaed9d700 1 -- 172.21.15.17:0/3371551781 learned_addr learned my addr 172.21.15.17:0/3371551781 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.697+0000 7faba7fff700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabb0101860 0x7fabb0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabaed9d700 1 -- 172.21.15.17:0/3371551781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb0100eb0 msgr2=0x7fabb0198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabaed9d700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabb0100eb0 0x7fabb0198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabaed9d700 1 -- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabb0101860 msgr2=0x7fabb0198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabaed9d700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabb0101860 0x7fabb0198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabaed9d700 1 -- 172.21.15.17:0/3371551781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba00095f0 con 0x7fabb0105f00 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7faba7fff700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabb0101860 0x7fabb0198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabaed9d700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0105f00 0x7fabb019ca20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faba8007ee0 tx=0x7faba800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8017070 con 0x7fabb0105f00 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb019d640 con 0x7fabb0105f00 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faba800f040 con 0x7fabb0105f00 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005760 con 0x7fabb0105f00 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.698+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb01a4ea0 con 0x7fabb0105f00 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.699+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fabb0110f60 con 0x7fabb0105f00 2024-02-14T15:01:55.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.700+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7faba80058c0 con 0x7fabb0105f00 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7faba77fe700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab9c0a21e0 0x7fab9c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7fabae59c700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab9c0a21e0 0x7fab9c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faba80cdfe0 con 0x7fabb0105f00 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7faba77fe700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] conn(0x7fab9c0ad180 0x7fab9c0af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 --> [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab9c0afcf0 con 0x7fab9c0ad180 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7faba80ce3f0 con 0x7fabb0105f00 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7fabae59c700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab9c0a21e0 0x7fab9c0a46a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faba000b5c0 tx=0x7faba0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.701+0000 7faba7fff700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] conn(0x7fab9c0ad180 0x7fab9c0af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.702+0000 7faba7fff700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] conn(0x7fab9c0ad180 0x7fab9c0af620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.703+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== osd.7 v2:172.21.15.50:6816/3407611050 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fab9c0afcf0 con 0x7fab9c0ad180 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.719+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 --> [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fabb0062430 con 0x7fab9c0ad180 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7faba77fe700 1 -- 172.21.15.17:0/3371551781 <== osd.7 v2:172.21.15.50:6816/3407611050 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fabb0062430 con 0x7fab9c0ad180 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] conn(0x7fab9c0ad180 msgr2=0x7fab9c0af620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7fabb4f75700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] conn(0x7fab9c0ad180 0x7fab9c0af620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab9c0a21e0 msgr2=0x7fab9c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7fabb4f75700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fab9c0a21e0 0x7fab9c0a46a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faba000b5c0 tx=0x7faba0005960 comp rx=0 tx=0).stop 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0105f00 msgr2=0x7fabb019ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7fabb4f75700 1 --2- 172.21.15.17:0/3371551781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0105f00 0x7fabb019ca20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faba8007ee0 tx=0x7faba800d3b0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.720+0000 7fabaed9d700 1 -- 172.21.15.17:0/3371551781 reap_dead start 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.721+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 shutdown_connections 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.721+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 >> 172.21.15.17:0/3371551781 conn(0x7fabb0076c70 msgr2=0x7fabb0104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.721+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 shutdown_connections 2024-02-14T15:01:55.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:50.721+0000 7fabb4f75700 1 -- 172.21.15.17:0/3371551781 wait complete. 2024-02-14T15:01:55.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 -- 172.21.15.17:0/3549958057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 msgr2=0x7f49a410db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 --2- 172.21.15.17:0/3549958057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 0x7f49a410db00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f499c00b0a0 tx=0x7f499c00b3b0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 -- 172.21.15.17:0/3549958057 shutdown_connections 2024-02-14T15:01:55.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 --2- 172.21.15.17:0/3549958057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 0x7f49a410db00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 --2- 172.21.15.17:0/3549958057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49a4100e70 0x7f49a4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 --2- 172.21.15.17:0/3549958057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49a4101820 0x7f49a4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 -- 172.21.15.17:0/3549958057 >> 172.21.15.17:0/3549958057 conn(0x7f49a4077b90 msgr2=0x7f49a4077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 -- 172.21.15.17:0/3549958057 shutdown_connections 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 -- 172.21.15.17:0/3549958057 wait complete. 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.217+0000 7f49a9366700 1 Processor -- start 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a9366700 1 -- start start 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a9366700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49a4100e70 0x7f49a4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a9366700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49a4101820 0x7f49a4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a9366700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 0x7f49a419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a9366700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a419c100 con 0x7f49a4105f30 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a9366700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a419c270 con 0x7f49a4101820 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a9366700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a419c430 con 0x7f49a4100e70 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 0x7f49a419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 0x7f49a419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38290/0 (socket says 172.21.15.17:38290) 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 -- 172.21.15.17:0/488748592 learned_addr learned my addr 172.21.15.17:0/488748592 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a27fc700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49a4101820 0x7f49a4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a2ffd700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49a4100e70 0x7f49a4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 -- 172.21.15.17:0/488748592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49a4100e70 msgr2=0x7f49a4197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49a4100e70 0x7f49a4197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 -- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49a4101820 msgr2=0x7f49a4197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49a4101820 0x7f49a4197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 -- 172.21.15.17:0/488748592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c009d00 con 0x7f49a4105f30 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.218+0000 7f49a37fe700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 0x7f49a419baf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f499c00b070 tx=0x7f499c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.219+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00e040 con 0x7f49a4105f30 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.219+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f499c01d470 con 0x7f49a4105f30 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.219+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c005260 con 0x7f49a4105f30 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.219+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a419c6b0 con 0x7f49a4105f30 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.219+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a41a3f70 con 0x7f49a4105f30 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.219+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49a4110fc0 con 0x7f49a4105f30 2024-02-14T15:01:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.221+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f499c01d5e0 con 0x7f49a4105f30 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.221+0000 7f498bfff700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f498c0a2260 0x7f498c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.221+0000 7f49a2ffd700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f498c0a2260 0x7f498c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.221+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f499c0cfb70 con 0x7f49a4105f30 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.221+0000 7f498bfff700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] conn(0x7f498c0ad200 0x7f498c0af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.221+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 --> [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f498c0afd70 con 0x7f498c0ad200 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.221+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f499c0cff80 con 0x7f49a4105f30 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.222+0000 7f49a27fc700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] conn(0x7f498c0ad200 0x7f498c0af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.222+0000 7f49a2ffd700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f498c0a2260 0x7f498c0a4720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4998009bf0 tx=0x7f4998009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.222+0000 7f49a27fc700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] conn(0x7f498c0ad200 0x7f498c0af6a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.223+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== osd.10 v2:172.21.15.50:6824/156762867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f498c0afd70 con 0x7f498c0ad200 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.239+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 --> [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49a40623b0 con 0x7f498c0ad200 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f498bfff700 1 -- 172.21.15.17:0/488748592 <== osd.10 v2:172.21.15.50:6824/156762867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f49a40623b0 con 0x7f498c0ad200 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] conn(0x7f498c0ad200 msgr2=0x7f498c0af6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] conn(0x7f498c0ad200 0x7f498c0af6a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f498c0a2260 msgr2=0x7f498c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f498c0a2260 0x7f498c0a4720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4998009bf0 tx=0x7f4998009260 comp rx=0 tx=0).stop 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 msgr2=0x7f49a419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 --2- 172.21.15.17:0/488748592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a4105f30 0x7f49a419baf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f499c00b070 tx=0x7f499c00bdd0 comp rx=0 tx=0).stop 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a37fe700 1 -- 172.21.15.17:0/488748592 reap_dead start 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 shutdown_connections 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 >> 172.21.15.17:0/488748592 conn(0x7f49a4077b90 msgr2=0x7f49a40fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 shutdown_connections 2024-02-14T15:01:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:54.240+0000 7f49a9366700 1 -- 172.21.15.17:0/488748592 wait complete. 2024-02-14T15:01:55.890 INFO:teuthology.orchestra.run.smithi017.stdout:163208757305 2024-02-14T15:01:55.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-14T15:01:56.316 INFO:teuthology.orchestra.run.smithi017.stdout:219043332142 2024-02-14T15:01:56.316 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-14T15:01:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 -- 172.21.15.17:0/2891356081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 msgr2=0x7f245010daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 --2- 172.21.15.17:0/2891356081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 0x7f245010daa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f244800b0a0 tx=0x7f244800b3b0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 -- 172.21.15.17:0/2891356081 shutdown_connections 2024-02-14T15:01:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 --2- 172.21.15.17:0/2891356081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 0x7f245010daa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 --2- 172.21.15.17:0/2891356081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2450106c60 0x7f2450107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 --2- 172.21.15.17:0/2891356081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2450100c20 0x7f2450101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 -- 172.21.15.17:0/2891356081 >> 172.21.15.17:0/2891356081 conn(0x7f2450077b50 msgr2=0x7f2450077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 -- 172.21.15.17:0/2891356081 shutdown_connections 2024-02-14T15:01:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.500+0000 7f2457eee700 1 -- 172.21.15.17:0/2891356081 wait complete. 2024-02-14T15:01:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 Processor -- start 2024-02-14T15:01:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 -- start start 2024-02-14T15:01:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2450100c20 0x7f2450198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 0x7f2450198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2450106c60 0x7f245019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245019d1e0 con 0x7f24501015c0 2024-02-14T15:01:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245019d350 con 0x7f2450100c20 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2457eee700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245019d510 con 0x7f2450106c60 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2455489700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 0x7f2450198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2455489700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 0x7f2450198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38312/0 (socket says 172.21.15.17:38312) 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.501+0000 7f2455489700 1 -- 172.21.15.17:0/305269747 learned_addr learned my addr 172.21.15.17:0/305269747 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f245648b700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2450106c60 0x7f245019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455c8a700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2450100c20 0x7f2450198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455489700 1 -- 172.21.15.17:0/305269747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2450106c60 msgr2=0x7f245019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455489700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2450106c60 0x7f245019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455489700 1 -- 172.21.15.17:0/305269747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2450100c20 msgr2=0x7f2450198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455489700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2450100c20 0x7f2450198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455489700 1 -- 172.21.15.17:0/305269747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448009d00 con 0x7f24501015c0 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455c8a700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2450100c20 0x7f2450198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2455489700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 0x7f2450198840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f244000c920 tx=0x7f244000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440011b60 con 0x7f24501015c0 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f244000ce70 con 0x7f24501015c0 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245019d7f0 con 0x7f24501015c0 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.502+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440005610 con 0x7f24501015c0 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.503+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24501029d0 con 0x7f24501015c0 2024-02-14T15:01:56.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.504+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2450110f40 con 0x7f24501015c0 2024-02-14T15:01:56.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.505+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2440004750 con 0x7f24501015c0 2024-02-14T15:01:56.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.506+0000 7f2446ffd700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f243c0a21e0 0x7f243c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:56.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.506+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f24400d2470 con 0x7f24501015c0 2024-02-14T15:01:56.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.506+0000 7f2446ffd700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] conn(0x7f243c0ad180 0x7f243c0af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:01:56.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.506+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 --> [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f243c0afcf0 con 0x7f243c0ad180 2024-02-14T15:01:56.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.506+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24400d2880 con 0x7f24501015c0 2024-02-14T15:01:56.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.506+0000 7f2455c8a700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f243c0a21e0 0x7f243c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:56.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.506+0000 7f245648b700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] conn(0x7f243c0ad180 0x7f243c0af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:01:56.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.507+0000 7f245648b700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] conn(0x7f243c0ad180 0x7f243c0af620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:56.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.507+0000 7f2455c8a700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f243c0a21e0 0x7f243c0a46a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f244c005fd0 tx=0x7f244c005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:01:56.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.508+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== osd.9 v2:172.21.15.17:6826/2843045035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f243c0afcf0 con 0x7f243c0ad180 2024-02-14T15:01:56.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.534+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 --> [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2450062430 con 0x7f243c0ad180 2024-02-14T15:01:56.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.535+0000 7f2446ffd700 1 -- 172.21.15.17:0/305269747 <== osd.9 v2:172.21.15.17:6826/2843045035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2450062430 con 0x7f243c0ad180 2024-02-14T15:01:56.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.536+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] conn(0x7f243c0ad180 msgr2=0x7f243c0af620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:56.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.536+0000 7f2457eee700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] conn(0x7f243c0ad180 0x7f243c0af620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f243c0a21e0 msgr2=0x7f243c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f243c0a21e0 0x7f243c0a46a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f244c005fd0 tx=0x7f244c005db0 comp rx=0 tx=0).stop 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 msgr2=0x7f2450198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 --2- 172.21.15.17:0/305269747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24501015c0 0x7f2450198840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f244000c920 tx=0x7f244000cc30 comp rx=0 tx=0).stop 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f245648b700 1 -- 172.21.15.17:0/305269747 reap_dead start 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 shutdown_connections 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 >> 172.21.15.17:0/305269747 conn(0x7f2450077b50 msgr2=0x7f24500ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 shutdown_connections 2024-02-14T15:01:56.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:01:56.537+0000 7f2457eee700 1 -- 172.21.15.17:0/305269747 wait complete. 2024-02-14T15:01:56.848 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:56 smithi017 ceph-mon[32549]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 6.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:56.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:56 smithi050 ceph-mon[36512]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 6.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:56.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:56 smithi076 ceph-mon[37057]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 6.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-02-14T15:01:57.230 INFO:teuthology.orchestra.run.smithi017.stdout:77309411400 2024-02-14T15:01:57.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-14T15:01:57.677 INFO:teuthology.orchestra.run.smithi017.stdout:287762808869 2024-02-14T15:01:57.677 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-14T15:01:58.013 INFO:teuthology.orchestra.run.smithi017.stdout:309237645344 2024-02-14T15:01:58.013 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-14T15:01:58.027 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:58.138 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:58.374 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:58.379 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:58.382 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:58.383 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:01:58.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:58 smithi050 ceph-mon[36512]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.2 KiB/s, 0 keys/s, 0 objects/s recovering 2024-02-14T15:01:58.948 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:58 smithi017 ceph-mon[32549]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.2 KiB/s, 0 keys/s, 0 objects/s recovering 2024-02-14T15:01:58.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:58 smithi076 ceph-mon[37057]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 5.2 KiB/s, 0 keys/s, 0 objects/s recovering 2024-02-14T15:01:59.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:01:59 smithi050 ceph-mon[36512]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 5.2 KiB/s, 0 keys/s, 0 objects/s recovering 2024-02-14T15:01:59.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:01:59 smithi076 ceph-mon[37057]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 5.2 KiB/s, 0 keys/s, 0 objects/s recovering 2024-02-14T15:02:00.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:01:59 smithi017 ceph-mon[32549]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 5.2 KiB/s, 0 keys/s, 0 objects/s recovering 2024-02-14T15:02:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:02.125 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:02.714 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:02 smithi017 ceph-mon[32549]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-02-14T15:02:02.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:02 smithi050 ceph-mon[36512]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-02-14T15:02:02.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:02 smithi076 ceph-mon[37057]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-02-14T15:02:04.843 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:04 smithi017 ceph-mon[32549]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-14T15:02:04.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:04 smithi050 ceph-mon[36512]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-14T15:02:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:04 smithi076 ceph-mon[37057]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-14T15:02:05.329 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 -- 172.21.15.17:0/3267681672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 msgr2=0x7f5b24101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 --2- 172.21.15.17:0/3267681672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b24101290 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009990 tx=0x7f5b14009ca0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 -- 172.21.15.17:0/3267681672 shutdown_connections 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 --2- 172.21.15.17:0/3267681672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b24105f00 0x7f5b2410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 --2- 172.21.15.17:0/3267681672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b24101290 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 --2- 172.21.15.17:0/3267681672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b24101860 0x7f5b24105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 -- 172.21.15.17:0/3267681672 >> 172.21.15.17:0/3267681672 conn(0x7f5b24076c70 msgr2=0x7f5b24077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:06.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 -- 172.21.15.17:0/3267681672 shutdown_connections 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.527+0000 7f5b29587700 1 -- 172.21.15.17:0/3267681672 wait complete. 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 Processor -- start 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 -- start start 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b2410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b24101860 0x7f5b2410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b24105f00 0x7f5b2410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241a6630 con 0x7f5b24100eb0 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241a67a0 con 0x7f5b24105f00 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b29587700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241a6960 con 0x7f5b24101860 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b22ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b2410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b22ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b2410e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38534/0 (socket says 172.21.15.17:38534) 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.528+0000 7f5b22ffd700 1 -- 172.21.15.17:0/3296604348 learned_addr learned my addr 172.21.15.17:0/3296604348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b227fc700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b24101860 0x7f5b2410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b22ffd700 1 -- 172.21.15.17:0/3296604348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b24101860 msgr2=0x7f5b2410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:06.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b22ffd700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b24101860 0x7f5b2410e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b22ffd700 1 -- 172.21.15.17:0/3296604348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b24105f00 msgr2=0x7f5b2410ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b22ffd700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b24105f00 0x7f5b2410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b22ffd700 1 -- 172.21.15.17:0/3296604348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140095f0 con 0x7f5b24100eb0 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b22ffd700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b2410e3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000c00 tx=0x7f5b1400b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b0bfff700 1 -- 172.21.15.17:0/3296604348 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1401d070 con 0x7f5b24100eb0 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b241a6be0 con 0x7f5b24100eb0 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b0bfff700 1 -- 172.21.15.17:0/3296604348 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b14022470 con 0x7f5b24100eb0 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b0bfff700 1 -- 172.21.15.17:0/3296604348 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14017650 con 0x7f5b24100eb0 2024-02-14T15:02:06.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.529+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b241a6ff0 con 0x7f5b24100eb0 2024-02-14T15:02:06.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.530+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b241091c0 con 0x7f5b24100eb0 2024-02-14T15:02:06.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.531+0000 7f5b0bfff700 1 -- 172.21.15.17:0/3296604348 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f5b140225e0 con 0x7f5b24100eb0 2024-02-14T15:02:06.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.532+0000 7f5b0bfff700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5b0c0a21e0 0x7f5b0c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:06.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.532+0000 7f5b227fc700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5b0c0a21e0 0x7f5b0c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:06.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.532+0000 7f5b0bfff700 1 -- 172.21.15.17:0/3296604348 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5b140cf760 con 0x7f5b24100eb0 2024-02-14T15:02:06.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.533+0000 7f5b227fc700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5b0c0a21e0 0x7f5b0c0a46a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005950 tx=0x7f5b180058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:06.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.536+0000 7f5b0bfff700 1 -- 172.21.15.17:0/3296604348 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b14098300 con 0x7f5b24100eb0 2024-02-14T15:02:06.709 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:06 smithi017 ceph-mon[32549]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-14T15:02:06.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.741+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f5b24111e20 con 0x7f5b24100eb0 2024-02-14T15:02:06.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.741+0000 7f5b0bfff700 1 -- 172.21.15.17:0/3296604348 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b14031020 con 0x7f5b24100eb0 2024-02-14T15:02:06.742 INFO:teuthology.orchestra.run.smithi017.stdout:253403070507 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5b0c0a21e0 msgr2=0x7f5b0c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5b0c0a21e0 0x7f5b0c0a46a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005950 tx=0x7f5b180058e0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 msgr2=0x7f5b2410e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b2410e3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000c00 tx=0x7f5b1400b680 comp rx=0 tx=0).stop 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 shutdown_connections 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5b0c0a21e0 0x7f5b0c0a46a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b24105f00 0x7f5b2410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b24100eb0 0x7f5b2410e3e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 --2- 172.21.15.17:0/3296604348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b24101860 0x7f5b2410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 >> 172.21.15.17:0/3296604348 conn(0x7f5b24076c70 msgr2=0x7f5b241070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 shutdown_connections 2024-02-14T15:02:06.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:06.743+0000 7f5b29587700 1 -- 172.21.15.17:0/3296604348 wait complete. 2024-02-14T15:02:06.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:06 smithi050 ceph-mon[36512]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-14T15:02:06.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:06 smithi076 ceph-mon[37057]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-14T15:02:07.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:07 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3296604348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-14T15:02:07.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:07 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3296604348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-14T15:02:07.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:07 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3296604348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 -- 172.21.15.17:0/1556965142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314102730 msgr2=0x7fb31410eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 --2- 172.21.15.17:0/1556965142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314102730 0x7fb31410eb70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b0a0 tx=0x7fb30400b3b0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 -- 172.21.15.17:0/1556965142 shutdown_connections 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 --2- 172.21.15.17:0/1556965142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314102730 0x7fb31410eb70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 --2- 172.21.15.17:0/1556965142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb314107d50 0x7fb314108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 --2- 172.21.15.17:0/1556965142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb314101d70 0x7fb3141021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 -- 172.21.15.17:0/1556965142 >> 172.21.15.17:0/1556965142 conn(0x7fb314078e50 msgr2=0x7fb314079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 -- 172.21.15.17:0/1556965142 shutdown_connections 2024-02-14T15:02:08.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.172+0000 7fb31be8c700 1 -- 172.21.15.17:0/1556965142 wait complete. 2024-02-14T15:02:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 Processor -- start 2024-02-14T15:02:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 -- start start 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb314101d70 0x7fb31419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb314102730 0x7fb31419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314107d50 0x7fb31419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31419f3f0 con 0x7fb314107d50 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31419f560 con 0x7fb314102730 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31be8c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31419f720 con 0x7fb314101d70 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314107d50 0x7fb31419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314107d50 0x7fb31419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38554/0 (socket says 172.21.15.17:38554) 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 -- 172.21.15.17:0/3126064288 learned_addr learned my addr 172.21.15.17:0/3126064288 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb319c28700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb314101d70 0x7fb31419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb319427700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb314102730 0x7fb31419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 -- 172.21.15.17:0/3126064288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb314101d70 msgr2=0x7fb31419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb314101d70 0x7fb31419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 -- 172.21.15.17:0/3126064288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb314102730 msgr2=0x7fb31419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb314102730 0x7fb31419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.173+0000 7fb31a429700 1 -- 172.21.15.17:0/3126064288 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304009d00 con 0x7fb314107d50 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.174+0000 7fb31a429700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314107d50 0x7fb31419ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb3040095e0 tx=0x7fb30400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.174+0000 7fb312ffd700 1 -- 172.21.15.17:0/3126064288 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400e040 con 0x7fb314107d50 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.174+0000 7fb312ffd700 1 -- 172.21.15.17:0/3126064288 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb304007a50 con 0x7fb314107d50 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.174+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31419f9a0 con 0x7fb314107d50 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.174+0000 7fb312ffd700 1 -- 172.21.15.17:0/3126064288 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304005670 con 0x7fb314107d50 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.174+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3141a7260 con 0x7fb314107d50 2024-02-14T15:02:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.175+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb314103400 con 0x7fb314107d50 2024-02-14T15:02:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.176+0000 7fb312ffd700 1 -- 172.21.15.17:0/3126064288 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb30401bca0 con 0x7fb314107d50 2024-02-14T15:02:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.176+0000 7fb312ffd700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2fc0a21e0 0x7fb2fc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.176+0000 7fb319c28700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2fc0a21e0 0x7fb2fc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.177+0000 7fb312ffd700 1 -- 172.21.15.17:0/3126064288 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb3040cfb70 con 0x7fb314107d50 2024-02-14T15:02:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.177+0000 7fb319c28700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2fc0a21e0 0x7fb2fc0a46a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009a10 tx=0x7fb30c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.181+0000 7fb312ffd700 1 -- 172.21.15.17:0/3126064288 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb304098710 con 0x7fb314107d50 2024-02-14T15:02:08.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.390+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb3140689e0 con 0x7fb314107d50 2024-02-14T15:02:08.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.391+0000 7fb312ffd700 1 -- 172.21.15.17:0/3126064288 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb30409bdd0 con 0x7fb314107d50 2024-02-14T15:02:08.392 INFO:teuthology.orchestra.run.smithi017.stdout:163208757308 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2fc0a21e0 msgr2=0x7fb2fc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2fc0a21e0 0x7fb2fc0a46a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009a10 tx=0x7fb30c009190 comp rx=0 tx=0).stop 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314107d50 msgr2=0x7fb31419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314107d50 0x7fb31419ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb3040095e0 tx=0x7fb30400ce90 comp rx=0 tx=0).stop 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 shutdown_connections 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2fc0a21e0 0x7fb2fc0a46a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb314107d50 0x7fb31419ede0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb314101d70 0x7fb31419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 --2- 172.21.15.17:0/3126064288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb314102730 0x7fb31419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 >> 172.21.15.17:0/3126064288 conn(0x7fb314078e50 msgr2=0x7fb3141008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 shutdown_connections 2024-02-14T15:02:08.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:08.393+0000 7fb31be8c700 1 -- 172.21.15.17:0/3126064288 wait complete. 2024-02-14T15:02:08.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:08 smithi017 ceph-mon[32549]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-14T15:02:08.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:08 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3126064288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T15:02:08.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:08 smithi050 ceph-mon[36512]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-14T15:02:08.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:08 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3126064288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T15:02:08.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:08 smithi076 ceph-mon[37057]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-14T15:02:08.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:08 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3126064288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T15:02:09.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757305 got 163208757308 for osd.5 2024-02-14T15:02:09.104 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 -- 172.21.15.17:0/3479973499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da006a370 msgr2=0x7f2da010ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/3479973499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da006a370 0x7f2da010ebb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800b0d0 tx=0x7f2d9800b3e0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 -- 172.21.15.17:0/3479973499 shutdown_connections 2024-02-14T15:02:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/3479973499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da006a370 0x7f2da010ebb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/3479973499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da0107d70 0x7f2da0108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/3479973499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2da00699d0 0x7f2da0069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 -- 172.21.15.17:0/3479973499 >> 172.21.15.17:0/3479973499 conn(0x7f2da0077b50 msgr2=0x7f2da0077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:09.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 -- 172.21.15.17:0/3479973499 shutdown_connections 2024-02-14T15:02:09.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 -- 172.21.15.17:0/3479973499 wait complete. 2024-02-14T15:02:09.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.742+0000 7f2da5c7a700 1 Processor -- start 2024-02-14T15:02:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2da5c7a700 1 -- start start 2024-02-14T15:02:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2da5c7a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da00699d0 0x7f2da0100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2da5c7a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2da006a370 0x7f2da0101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2da5c7a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da0107d70 0x7f2da0101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2da5c7a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01a6720 con 0x7f2da0107d70 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2da5c7a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01a6890 con 0x7f2da00699d0 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2da5c7a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01a6a50 con 0x7f2da006a370 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da0107d70 0x7f2da0101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da0107d70 0x7f2da0101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38578/0 (socket says 172.21.15.17:38578) 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 -- 172.21.15.17:0/828969319 learned_addr learned my addr 172.21.15.17:0/828969319 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9effd700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2da006a370 0x7f2da0101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da00699d0 0x7f2da0100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 -- 172.21.15.17:0/828969319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2da006a370 msgr2=0x7f2da0101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2da006a370 0x7f2da0101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 -- 172.21.15.17:0/828969319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da00699d0 msgr2=0x7f2da0100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da00699d0 0x7f2da0100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9ffff700 1 -- 172.21.15.17:0/828969319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98009d00 con 0x7f2da0107d70 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.743+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da00699d0 0x7f2da0100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.744+0000 7f2d9ffff700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da0107d70 0x7f2da0101770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2d980079b0 tx=0x7f2d98007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.744+0000 7f2d9cff9700 1 -- 172.21.15.17:0/828969319 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800e040 con 0x7f2da0107d70 2024-02-14T15:02:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.744+0000 7f2d9cff9700 1 -- 172.21.15.17:0/828969319 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d98004e60 con 0x7f2da0107d70 2024-02-14T15:02:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.744+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da01a6cd0 con 0x7f2da0107d70 2024-02-14T15:02:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.744+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da01a7140 con 0x7f2da0107d70 2024-02-14T15:02:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.745+0000 7f2d9cff9700 1 -- 172.21.15.17:0/828969319 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98012b60 con 0x7f2da0107d70 2024-02-14T15:02:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.745+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da010a290 con 0x7f2da0107d70 2024-02-14T15:02:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.746+0000 7f2d9cff9700 1 -- 172.21.15.17:0/828969319 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2d98012cc0 con 0x7f2da0107d70 2024-02-14T15:02:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.747+0000 7f2d9cff9700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d8c0a21e0 0x7f2d8c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.747+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d8c0a21e0 0x7f2d8c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.747+0000 7f2d9cff9700 1 -- 172.21.15.17:0/828969319 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f2d980cfe90 con 0x7f2da0107d70 2024-02-14T15:02:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.747+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d8c0a21e0 0x7f2d8c0a46a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005fd0 tx=0x7f2d90005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:09.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.751+0000 7f2d9cff9700 1 -- 172.21.15.17:0/828969319 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d98098a90 con 0x7f2da0107d70 2024-02-14T15:02:09.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:09 smithi017 ceph-mon[32549]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-02-14T15:02:09.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:09 smithi050 ceph-mon[36512]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-02-14T15:02:09.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.958+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2da0111d50 con 0x7f2da0107d70 2024-02-14T15:02:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.958+0000 7f2d9cff9700 1 -- 172.21.15.17:0/828969319 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d9809c150 con 0x7f2da0107d70 2024-02-14T15:02:09.959 INFO:teuthology.orchestra.run.smithi017.stdout:197568495671 2024-02-14T15:02:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.960+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d8c0a21e0 msgr2=0x7f2d8c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.960+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d8c0a21e0 0x7f2d8c0a46a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005fd0 tx=0x7f2d90005d40 comp rx=0 tx=0).stop 2024-02-14T15:02:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.960+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da0107d70 msgr2=0x7f2da0101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.960+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da0107d70 0x7f2da0101770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2d980079b0 tx=0x7f2d98007a90 comp rx=0 tx=0).stop 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 shutdown_connections 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f2d8c0a21e0 0x7f2d8c0a46a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da0107d70 0x7f2da0101770 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2da00699d0 0x7f2da0100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 --2- 172.21.15.17:0/828969319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2da006a370 0x7f2da0101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 >> 172.21.15.17:0/828969319 conn(0x7f2da0077b50 msgr2=0x7f2da00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 shutdown_connections 2024-02-14T15:02:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.961+0000 7f2da5c7a700 1 -- 172.21.15.17:0/828969319 wait complete. 2024-02-14T15:02:09.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:09 smithi076 ceph-mon[37057]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3111834234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 msgr2=0x7fbe28110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3111834234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe28110e00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000b0a0 tx=0x7fbe2000b3b0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3111834234 shutdown_connections 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3111834234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe28110e00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3111834234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe28104150 0x7fbe28104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3111834234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe28104b00 0x7fbe28108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3111834234 >> 172.21.15.17:0/3111834234 conn(0x7fbe280ffa00 msgr2=0x7fbe28101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:09.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3111834234 shutdown_connections 2024-02-14T15:02:09.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.988+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3111834234 wait complete. 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.989+0000 7fbe2cb43700 1 Processor -- start 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.989+0000 7fbe2cb43700 1 -- start start 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.989+0000 7fbe2cb43700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe28104150 0x7fbe2819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.989+0000 7fbe2cb43700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe28104b00 0x7fbe2819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.989+0000 7fbe2cb43700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe2819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2cb43700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2819f180 con 0x7fbe28109230 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2cb43700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2819f2f0 con 0x7fbe28104150 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2cb43700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2819f4b0 con 0x7fbe28104b00 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe26d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe2819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe26d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe2819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38596/0 (socket says 172.21.15.17:38596) 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe26d9d700 1 -- 172.21.15.17:0/3623762441 learned_addr learned my addr 172.21.15.17:0/3623762441 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2659c700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe28104150 0x7fbe2819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe25d9b700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe28104b00 0x7fbe2819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2659c700 1 -- 172.21.15.17:0/3623762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe28104b00 msgr2=0x7fbe2819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2659c700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe28104b00 0x7fbe2819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2659c700 1 -- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 msgr2=0x7fbe2819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2659c700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe2819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe2659c700 1 -- 172.21.15.17:0/3623762441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20009d00 con 0x7fbe28104150 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.990+0000 7fbe26d9d700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe2819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.991+0000 7fbe2659c700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe28104150 0x7fbe2819a2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800c970 tx=0x7fbe1800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.991+0000 7fbe177fe700 1 -- 172.21.15.17:0/3623762441 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18004440 con 0x7fbe28104150 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.991+0000 7fbe177fe700 1 -- 172.21.15.17:0/3623762441 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe18003b60 con 0x7fbe28104150 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.991+0000 7fbe177fe700 1 -- 172.21.15.17:0/3623762441 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18007ca0 con 0x7fbe28104150 2024-02-14T15:02:09.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.991+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2819f790 con 0x7fbe28104150 2024-02-14T15:02:09.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.991+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe281a6ff0 con 0x7fbe28104150 2024-02-14T15:02:09.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.992+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe280689e0 con 0x7fbe28104150 2024-02-14T15:02:09.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.993+0000 7fbe177fe700 1 -- 172.21.15.17:0/3623762441 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fbe18003cd0 con 0x7fbe28104150 2024-02-14T15:02:09.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.994+0000 7fbe177fe700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbe100a2260 0x7fbe100a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:09.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.994+0000 7fbe25d9b700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbe100a2260 0x7fbe100a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:09.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.994+0000 7fbe177fe700 1 -- 172.21.15.17:0/3623762441 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbe180ce020 con 0x7fbe28104150 2024-02-14T15:02:09.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.994+0000 7fbe25d9b700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbe100a2260 0x7fbe100a4720 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe28101d00 tx=0x7fbe1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:09.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:09.999+0000 7fbe177fe700 1 -- 172.21.15.17:0/3623762441 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe18096b90 con 0x7fbe28104150 2024-02-14T15:02:10.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.203+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbe28113f90 con 0x7fbe28104150 2024-02-14T15:02:10.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.204+0000 7fbe177fe700 1 -- 172.21.15.17:0/3623762441 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe1809a250 con 0x7fbe28104150 2024-02-14T15:02:10.205 INFO:teuthology.orchestra.run.smithi017.stdout:107374182469 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbe100a2260 msgr2=0x7fbe100a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbe100a2260 0x7fbe100a4720 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbe28101d00 tx=0x7fbe1c00b410 comp rx=0 tx=0).stop 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe28104150 msgr2=0x7fbe2819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe28104150 0x7fbe2819a2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800c970 tx=0x7fbe1800cc80 comp rx=0 tx=0).stop 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 shutdown_connections 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbe100a2260 0x7fbe100a4720 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe28109230 0x7fbe2819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe28104150 0x7fbe2819a2a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 --2- 172.21.15.17:0/3623762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe28104b00 0x7fbe2819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 >> 172.21.15.17:0/3623762441 conn(0x7fbe280ffa00 msgr2=0x7fbe28100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 shutdown_connections 2024-02-14T15:02:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:10.206+0000 7fbe2cb43700 1 -- 172.21.15.17:0/3623762441 wait complete. 2024-02-14T15:02:10.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:10 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/828969319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T15:02:10.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:10 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3623762441' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T15:02:10.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/828969319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T15:02:10.942 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3623762441' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T15:02:10.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:10 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/828969319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T15:02:10.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:10 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3623762441' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T15:02:11.478 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070507 for osd.8 2024-02-14T15:02:11.478 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:11.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:11 smithi017 ceph-mon[32549]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:11.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:11 smithi050 ceph-mon[36512]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:11.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:11 smithi076 ceph-mon[37057]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:13.711 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:13.740 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495666 got 197568495671 for osd.6 2024-02-14T15:02:13.740 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:13 smithi076 ceph-mon[37057]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:13.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1225823577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T15:02:14.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:13 smithi017 ceph-mon[32549]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:14.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1225823577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T15:02:14.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:13 smithi050 ceph-mon[36512]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:14.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1225823577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T15:02:14.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 -- 172.21.15.17:0/1631206798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4105f00 msgr2=0x7fc6b410dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:14.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1631206798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4105f00 0x7fc6b410dad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b0a0 tx=0x7fc6ac00b3b0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 -- 172.21.15.17:0/1631206798 shutdown_connections 2024-02-14T15:02:14.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1631206798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4105f00 0x7fc6b410dad0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1631206798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b4100eb0 0x7fc6b4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1631206798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4101860 0x7fc6b4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 -- 172.21.15.17:0/1631206798 >> 172.21.15.17:0/1631206798 conn(0x7fc6b4076c70 msgr2=0x7fc6b4077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:14.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 -- 172.21.15.17:0/1631206798 shutdown_connections 2024-02-14T15:02:14.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.629+0000 7fc6b962b700 1 -- 172.21.15.17:0/1631206798 wait complete. 2024-02-14T15:02:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 Processor -- start 2024-02-14T15:02:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 -- start start 2024-02-14T15:02:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4100eb0 0x7fc6b4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4101860 0x7fc6b4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b4105f00 0x7fc6b419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b419c1e0 con 0x7fc6b4100eb0 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b419c350 con 0x7fc6b4101860 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.630+0000 7fc6b962b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b419c510 con 0x7fc6b4105f00 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4100eb0 0x7fc6b4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4100eb0 0x7fc6b4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54158/0 (socket says 172.21.15.17:54158) 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 -- 172.21.15.17:0/1794090847 learned_addr learned my addr 172.21.15.17:0/1794090847 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b37fe700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b4105f00 0x7fc6b419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b27fc700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4101860 0x7fc6b4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 -- 172.21.15.17:0/1794090847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b4105f00 msgr2=0x7fc6b419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b4105f00 0x7fc6b419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 -- 172.21.15.17:0/1794090847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4101860 msgr2=0x7fc6b4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4101860 0x7fc6b4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 -- 172.21.15.17:0/1794090847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac009d00 con 0x7fc6b4100eb0 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b27fc700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4101860 0x7fc6b4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b37fe700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b4105f00 0x7fc6b419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b2ffd700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4100eb0 0x7fc6b4197350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009970 tx=0x7fc6a4009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc69bfff700 1 -- 172.21.15.17:0/1794090847 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4012070 con 0x7fc6b4100eb0 2024-02-14T15:02:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc69bfff700 1 -- 172.21.15.17:0/1794090847 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6a400d260 con 0x7fc6b4100eb0 2024-02-14T15:02:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b419c7f0 con 0x7fc6b4100eb0 2024-02-14T15:02:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc69bfff700 1 -- 172.21.15.17:0/1794090847 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4017b00 con 0x7fc6b4100eb0 2024-02-14T15:02:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.631+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b406a430 con 0x7fc6b4100eb0 2024-02-14T15:02:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.632+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b40689e0 con 0x7fc6b4100eb0 2024-02-14T15:02:14.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.633+0000 7fc69bfff700 1 -- 172.21.15.17:0/1794090847 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc6a40076b0 con 0x7fc6b4100eb0 2024-02-14T15:02:14.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.634+0000 7fc69bfff700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc69c0a2260 0x7fc69c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:14.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.634+0000 7fc6b27fc700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc69c0a2260 0x7fc69c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:14.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.634+0000 7fc69bfff700 1 -- 172.21.15.17:0/1794090847 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc6a40cf510 con 0x7fc6b4100eb0 2024-02-14T15:02:14.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.635+0000 7fc6b27fc700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc69c0a2260 0x7fc69c0a4720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4107360 tx=0x7fc6a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:14.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.639+0000 7fc69bfff700 1 -- 172.21.15.17:0/1794090847 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6a4098080 con 0x7fc6b4100eb0 2024-02-14T15:02:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.847+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fc6b4198660 con 0x7fc6b4100eb0 2024-02-14T15:02:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.847+0000 7fc69bfff700 1 -- 172.21.15.17:0/1794090847 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc6a409b740 con 0x7fc6b4100eb0 2024-02-14T15:02:14.848 INFO:teuthology.orchestra.run.smithi017.stdout:343597383710 2024-02-14T15:02:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.849+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc69c0a2260 msgr2=0x7fc69c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.849+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc69c0a2260 0x7fc69c0a4720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4107360 tx=0x7fc6a8008040 comp rx=0 tx=0).stop 2024-02-14T15:02:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.849+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4100eb0 msgr2=0x7fc6b4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.849+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4100eb0 0x7fc6b4197350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009970 tx=0x7fc6a4009d30 comp rx=0 tx=0).stop 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 shutdown_connections 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc69c0a2260 0x7fc69c0a4720 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b4105f00 0x7fc6b419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6b4100eb0 0x7fc6b4197350 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 --2- 172.21.15.17:0/1794090847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4101860 0x7fc6b4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 >> 172.21.15.17:0/1794090847 conn(0x7fc6b4076c70 msgr2=0x7fc6b40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 shutdown_connections 2024-02-14T15:02:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:14.850+0000 7fc6b962b700 1 -- 172.21.15.17:0/1794090847 wait complete. 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.074+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3027283881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 msgr2=0x7fa8e810eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.074+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3027283881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e810eb70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000b0d0 tx=0x7fa8e000b3e0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.074+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3027283881 shutdown_connections 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.074+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3027283881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e810eb70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.074+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3027283881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e8107d50 0x7fa8e8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.074+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3027283881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8e8101d70 0x7fa8e81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.074+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3027283881 >> 172.21.15.17:0/3027283881 conn(0x7fa8e8078e50 msgr2=0x7fa8e8079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3027283881 shutdown_connections 2024-02-14T15:02:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3027283881 wait complete. 2024-02-14T15:02:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 Processor -- start 2024-02-14T15:02:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 -- start start 2024-02-14T15:02:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e8101d70 0x7fa8e8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8e8107d50 0x7fa8e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81a6740 con 0x7fa8e8102730 2024-02-14T15:02:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81a68b0 con 0x7fa8e8101d70 2024-02-14T15:02:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.075+0000 7fa8ee9fc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81a6a70 con 0x7fa8e8107d50 2024-02-14T15:02:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e8074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54174/0 (socket says 172.21.15.17:54174) 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 -- 172.21.15.17:0/3747796408 learned_addr learned my addr 172.21.15.17:0/3747796408 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8ecf99700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8e8107d50 0x7fa8e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e7fff700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e8101d70 0x7fa8e8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 -- 172.21.15.17:0/3747796408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8e8107d50 msgr2=0x7fa8e8074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8e8107d50 0x7fa8e8074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 -- 172.21.15.17:0/3747796408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e8101d70 msgr2=0x7fa8e8074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e8101d70 0x7fa8e8074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 -- 172.21.15.17:0/3747796408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0009d00 con 0x7fa8e8102730 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e77fe700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e8074560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d950 tx=0x7fa8dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e57fa700 1 -- 172.21.15.17:0/3747796408 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00f890 con 0x7fa8e8102730 2024-02-14T15:02:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e81a6d50 con 0x7fa8e8102730 2024-02-14T15:02:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e57fa700 1 -- 172.21.15.17:0/3747796408 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8dc010460 con 0x7fa8e8102730 2024-02-14T15:02:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8e57fa700 1 -- 172.21.15.17:0/3747796408 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00e610 con 0x7fa8e8102730 2024-02-14T15:02:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.076+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e81a71f0 con 0x7fa8e8102730 2024-02-14T15:02:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.077+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e810a230 con 0x7fa8e8102730 2024-02-14T15:02:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.079+0000 7fa8e57fa700 1 -- 172.21.15.17:0/3747796408 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa8dc010960 con 0x7fa8e8102730 2024-02-14T15:02:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.079+0000 7fa8e57fa700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8d00a21e0 0x7fa8d00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.079+0000 7fa8e7fff700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8d00a21e0 0x7fa8d00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.079+0000 7fa8e57fa700 1 -- 172.21.15.17:0/3747796408 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa8dc0cdeb0 con 0x7fa8e8102730 2024-02-14T15:02:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.080+0000 7fa8e7fff700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8d00a21e0 0x7fa8d00a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009a10 tx=0x7fa8d8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.083+0000 7fa8e57fa700 1 -- 172.21.15.17:0/3747796408 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8dc096a20 con 0x7fa8e8102730 2024-02-14T15:02:15.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.285+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa8e8111dd0 con 0x7fa8e8102730 2024-02-14T15:02:15.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.285+0000 7fa8e57fa700 1 -- 172.21.15.17:0/3747796408 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa8dc09a0e0 con 0x7fa8e8102730 2024-02-14T15:02:15.286 INFO:teuthology.orchestra.run.smithi017.stdout:219043332146 2024-02-14T15:02:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8d00a21e0 msgr2=0x7fa8d00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8d00a21e0 0x7fa8d00a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009a10 tx=0x7fa8d8009190 comp rx=0 tx=0).stop 2024-02-14T15:02:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 msgr2=0x7fa8e8074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e8074560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d950 tx=0x7fa8dc00dc60 comp rx=0 tx=0).stop 2024-02-14T15:02:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 shutdown_connections 2024-02-14T15:02:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa8d00a21e0 0x7fa8d00a46a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8e8107d50 0x7fa8e8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e8101d70 0x7fa8e8074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 --2- 172.21.15.17:0/3747796408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e8102730 0x7fa8e8074560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 >> 172.21.15.17:0/3747796408 conn(0x7fa8e8078e50 msgr2=0x7fa8e8100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 shutdown_connections 2024-02-14T15:02:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:15.287+0000 7fa8ee9fc700 1 -- 172.21.15.17:0/3747796408 wait complete. 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 -- 172.21.15.17:0/1172251909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640699d0 msgr2=0x7f5164069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 --2- 172.21.15.17:0/1172251909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640699d0 0x7f5164069db0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5160009990 tx=0x7f5160009ca0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 -- 172.21.15.17:0/1172251909 shutdown_connections 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 --2- 172.21.15.17:0/1172251909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f516410e7d0 0x7f5164110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 --2- 172.21.15.17:0/1172251909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640699d0 0x7f5164069db0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 --2- 172.21.15.17:0/1172251909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f516406a2f0 0x7f516410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 -- 172.21.15.17:0/1172251909 >> 172.21.15.17:0/1172251909 conn(0x7f5164077b50 msgr2=0x7f5164077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 -- 172.21.15.17:0/1172251909 shutdown_connections 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 -- 172.21.15.17:0/1172251909 wait complete. 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.970+0000 7f516c37f700 1 Processor -- start 2024-02-14T15:02:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516c37f700 1 -- start start 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516c37f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51640699d0 0x7f516419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516c37f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f516406a2f0 0x7f516419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516c37f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f516410e7d0 0x7f516419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516c37f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419f270 con 0x7f516410e7d0 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516c37f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419f3e0 con 0x7f51640699d0 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516c37f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419f5a0 con 0x7f516406a2f0 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f516410e7d0 0x7f516419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a11b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51640699d0 0x7f516419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f516410e7d0 0x7f516419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54136/0 (socket says 172.21.15.17:54136) 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 -- 172.21.15.17:0/1225823577 learned_addr learned my addr 172.21.15.17:0/1225823577 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516991a700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f516406a2f0 0x7f516419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 -- 172.21.15.17:0/1225823577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f516406a2f0 msgr2=0x7f516419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f516406a2f0 0x7f516419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 -- 172.21.15.17:0/1225823577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51640699d0 msgr2=0x7f516419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51640699d0 0x7f516419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 -- 172.21.15.17:0/1225823577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51600095f0 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516991a700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f516406a2f0 0x7f516419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.971+0000 7f516a91c700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f516410e7d0 0x7f516419ec60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f515c007d40 tx=0x7f515c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.972+0000 7f515b7fe700 1 -- 172.21.15.17:0/1225823577 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00ad40 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.972+0000 7f515b7fe700 1 -- 172.21.15.17:0/1225823577 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f515c007e90 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.972+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516419f880 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.972+0000 7f515b7fe700 1 -- 172.21.15.17:0/1225823577 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c011780 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.972+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51641a70e0 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.973+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51640689e0 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.975+0000 7f515b7fe700 1 -- 172.21.15.17:0/1225823577 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f515c012420 con 0x7f516410e7d0 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.975+0000 7f515b7fe700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f51500a2260 0x7f51500a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:15.604 INFO:teuthology.orchestra.run.smithi017.stdout:55834574929 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.975+0000 7f515b7fe700 1 -- 172.21.15.17:0/1225823577 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f515c0ce170 con 0x7f516410e7d0 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.975+0000 7f516a11b700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f51500a2260 0x7f51500a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.976+0000 7f516a11b700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f51500a2260 0x7f51500a4720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51600098e0 tx=0x7f5160005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:12.980+0000 7f515b7fe700 1 -- 172.21.15.17:0/1225823577 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f515c096df0 con 0x7f516410e7d0 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.185+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f516419b6f0 con 0x7f516410e7d0 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.185+0000 7f515b7fe700 1 -- 172.21.15.17:0/1225823577 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f515c09a4b0 con 0x7f516410e7d0 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f51500a2260 msgr2=0x7f51500a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f51500a2260 0x7f51500a4720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51600098e0 tx=0x7f5160005960 comp rx=0 tx=0).stop 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f516410e7d0 msgr2=0x7f516419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f516410e7d0 0x7f516419ec60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f515c007d40 tx=0x7f515c00a340 comp rx=0 tx=0).stop 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 shutdown_connections 2024-02-14T15:02:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f51500a2260 0x7f51500a4720 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f516410e7d0 0x7f516419ec60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51640699d0 0x7f516419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 --2- 172.21.15.17:0/1225823577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f516406a2f0 0x7f516419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 >> 172.21.15.17:0/1225823577 conn(0x7f5164077b50 msgr2=0x7f516410fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 shutdown_connections 2024-02-14T15:02:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:13.187+0000 7f516c37f700 1 -- 172.21.15.17:0/1225823577 wait complete. 2024-02-14T15:02:15.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:15 smithi017 ceph-mon[32549]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:15.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1794090847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-14T15:02:15.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3747796408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T15:02:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:15 smithi076 ceph-mon[37057]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1794090847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-14T15:02:15.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3747796408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T15:02:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:15 smithi050 ceph-mon[36512]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1794090847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-14T15:02:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3747796408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T15:02:16.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574929 for osd.1 2024-02-14T15:02:16.250 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:16.651 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332146 for osd.7 2024-02-14T15:02:16.652 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:17.062 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182469 for osd.3 2024-02-14T15:02:17.062 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:17.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:17 smithi017 ceph-mon[32549]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:17.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:17 smithi076 ceph-mon[37057]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:17 smithi050 ceph-mon[36512]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 -- 172.21.15.17:0/3145187358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c100eb0 msgr2=0x7f7a2c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 --2- 172.21.15.17:0/3145187358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c100eb0 0x7f7a2c101290 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b0a0 tx=0x7f7a1c00b3b0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 -- 172.21.15.17:0/3145187358 shutdown_connections 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 --2- 172.21.15.17:0/3145187358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c105f00 0x7f7a2c10dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 --2- 172.21.15.17:0/3145187358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c100eb0 0x7f7a2c101290 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 --2- 172.21.15.17:0/3145187358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a2c101860 0x7f7a2c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 -- 172.21.15.17:0/3145187358 >> 172.21.15.17:0/3145187358 conn(0x7f7a2c076c70 msgr2=0x7f7a2c077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 -- 172.21.15.17:0/3145187358 shutdown_connections 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 -- 172.21.15.17:0/3145187358 wait complete. 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 Processor -- start 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.215+0000 7f7a340ea700 1 -- start start 2024-02-14T15:02:18.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a340ea700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a2c100eb0 0x7f7a2c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a340ea700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c101860 0x7f7a2c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a340ea700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c105f00 0x7f7a2c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a340ea700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c19c180 con 0x7f7a2c101860 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a340ea700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c19c2f0 con 0x7f7a2c105f00 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a340ea700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c19c4b0 con 0x7f7a2c100eb0 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c101860 0x7f7a2c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c101860 0x7f7a2c1977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54208/0 (socket says 172.21.15.17:54208) 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 -- 172.21.15.17:0/1061124659 learned_addr learned my addr 172.21.15.17:0/1061124659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a32687700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c105f00 0x7f7a2c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31e86700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a2c100eb0 0x7f7a2c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 -- 172.21.15.17:0/1061124659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a2c100eb0 msgr2=0x7f7a2c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a2c100eb0 0x7f7a2c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 -- 172.21.15.17:0/1061124659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c105f00 msgr2=0x7f7a2c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c105f00 0x7f7a2c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 -- 172.21.15.17:0/1061124659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c009d00 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a31685700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c101860 0x7f7a2c1977e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000d980 tx=0x7f7a2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a2affd700 1 -- 172.21.15.17:0/1061124659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20009930 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a2affd700 1 -- 172.21.15.17:0/1061124659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a20010460 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.216+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2c19c790 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.217+0000 7f7a2affd700 1 -- 172.21.15.17:0/1061124659 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000f620 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.217+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2c1a3ff0 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.218+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2c109130 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.219+0000 7f7a2affd700 1 -- 172.21.15.17:0/1061124659 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7a200105d0 con 0x7f7a2c101860 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.219+0000 7f7a2affd700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7a140a21e0 0x7f7a140a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.219+0000 7f7a31e86700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7a140a21e0 0x7f7a140a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:18.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.219+0000 7f7a2affd700 1 -- 172.21.15.17:0/1061124659 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7a200cdc30 con 0x7f7a2c101860 2024-02-14T15:02:18.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.220+0000 7f7a31e86700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7a140a21e0 0x7f7a140a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006130 tx=0x7f7a1c0060a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:18.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.223+0000 7f7a2affd700 1 -- 172.21.15.17:0/1061124659 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a200967a0 con 0x7f7a2c101860 2024-02-14T15:02:18.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.439+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f7a2c198660 con 0x7f7a2c101860 2024-02-14T15:02:18.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.439+0000 7f7a2affd700 1 -- 172.21.15.17:0/1061124659 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a20099e60 con 0x7f7a2c101860 2024-02-14T15:02:18.440 INFO:teuthology.orchestra.run.smithi017.stdout:287762808873 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7a140a21e0 msgr2=0x7f7a140a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7a140a21e0 0x7f7a140a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006130 tx=0x7f7a1c0060a0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c101860 msgr2=0x7f7a2c1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c101860 0x7f7a2c1977e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000d980 tx=0x7f7a2000dc90 comp rx=0 tx=0).stop 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 shutdown_connections 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7a140a21e0 0x7f7a140a46a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c105f00 0x7f7a2c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a2c100eb0 0x7f7a2c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 --2- 172.21.15.17:0/1061124659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a2c101860 0x7f7a2c1977e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 >> 172.21.15.17:0/1061124659 conn(0x7f7a2c076c70 msgr2=0x7f7a2c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 shutdown_connections 2024-02-14T15:02:18.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.442+0000 7f7a340ea700 1 -- 172.21.15.17:0/1061124659 wait complete. 2024-02-14T15:02:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:18 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3567964271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T15:02:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:18 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1061124659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-14T15:02:19.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383704 got 343597383710 for osd.11 2024-02-14T15:02:19.104 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:18 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3567964271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T15:02:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:18 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1061124659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-14T15:02:19.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:18 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3567964271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T15:02:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:18 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1061124659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-14T15:02:19.958 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:19 smithi017 ceph-mon[32549]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:20.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:19 smithi050 ceph-mon[36512]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:20.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:19 smithi076 ceph-mon[37057]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:21.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 -- 172.21.15.17:0/3533513330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108590 msgr2=0x7f8214108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:21.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 --2- 172.21.15.17:0/3533513330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108590 0x7f8214108970 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b0a0 tx=0x7f820c00b3b0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 -- 172.21.15.17:0/3533513330 shutdown_connections 2024-02-14T15:02:21.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 --2- 172.21.15.17:0/3533513330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f821410d670 0x7f8214115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 --2- 172.21.15.17:0/3533513330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108590 0x7f8214108970 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 --2- 172.21.15.17:0/3533513330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8214108f40 0x7f821410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 -- 172.21.15.17:0/3533513330 >> 172.21.15.17:0/3533513330 conn(0x7f8214075460 msgr2=0x7f8214077880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 -- 172.21.15.17:0/3533513330 shutdown_connections 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.117+0000 7f821b282700 1 -- 172.21.15.17:0/3533513330 wait complete. 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 Processor -- start 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 -- start start 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8214108590 0x7f82141a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108f40 0x7f82141a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f821410d670 0x7f821419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419ef00 con 0x7f8214108f40 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419f070 con 0x7f821410d670 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821b282700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419f230 con 0x7f8214108590 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821881d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108f40 0x7f82141a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821881d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108f40 0x7f82141a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54192/0 (socket says 172.21.15.17:54192) 2024-02-14T15:02:21.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.118+0000 7f821881d700 1 -- 172.21.15.17:0/3567964271 learned_addr learned my addr 172.21.15.17:0/3567964271 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:21.043 INFO:teuthology.orchestra.run.smithi017.stdout:34359738454 2024-02-14T15:02:21.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821901e700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8214108590 0x7f82141a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:21.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821981f700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f821410d670 0x7f821419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:21.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821881d700 1 -- 172.21.15.17:0/3567964271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8214108590 msgr2=0x7f82141a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:21.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821881d700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8214108590 0x7f82141a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821881d700 1 -- 172.21.15.17:0/3567964271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f821410d670 msgr2=0x7f821419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821881d700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f821410d670 0x7f821419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821881d700 1 -- 172.21.15.17:0/3567964271 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c009d00 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821981f700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f821410d670 0x7f821419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821881d700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108f40 0x7f82141a4bc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f820400c9e0 tx=0x7f820400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f820a7fc700 1 -- 172.21.15.17:0/3567964271 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204007aa0 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821419f510 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f820a7fc700 1 -- 172.21.15.17:0/3567964271 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8204007c00 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f820a7fc700 1 -- 172.21.15.17:0/3567964271 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204018660 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.119+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141ab6b0 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.120+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214110930 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.121+0000 7f820a7fc700 1 -- 172.21.15.17:0/3567964271 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f820400f450 con 0x7f8214108f40 2024-02-14T15:02:21.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.122+0000 7f820a7fc700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f82000a21e0 0x7f82000a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.122+0000 7f821901e700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f82000a21e0 0x7f82000a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.122+0000 7f820a7fc700 1 -- 172.21.15.17:0/3567964271 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f82040ce2c0 con 0x7f8214108f40 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.123+0000 7f821901e700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f82000a21e0 0x7f82000a46a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b830 tx=0x7f820c006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.126+0000 7f820a7fc700 1 -- 172.21.15.17:0/3567964271 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8204096f60 con 0x7f8214108f40 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.332+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8214109a70 con 0x7f8214108f40 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.332+0000 7f820a7fc700 1 -- 172.21.15.17:0/3567964271 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f820409a620 con 0x7f8214108f40 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.334+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f82000a21e0 msgr2=0x7f82000a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.334+0000 7f821b282700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f82000a21e0 0x7f82000a46a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b830 tx=0x7f820c006210 comp rx=0 tx=0).stop 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.334+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108f40 msgr2=0x7f82141a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.334+0000 7f821b282700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108f40 0x7f82141a4bc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f820400c9e0 tx=0x7f820400ccf0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 shutdown_connections 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f82000a21e0 0x7f82000a46a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f821410d670 0x7f821419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8214108590 0x7f82141a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 --2- 172.21.15.17:0/3567964271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8214108f40 0x7f82141a4bc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:21.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 >> 172.21.15.17:0/3567964271 conn(0x7f8214075460 msgr2=0x7f82140777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:21.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 shutdown_connections 2024-02-14T15:02:21.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:18.335+0000 7f821b282700 1 -- 172.21.15.17:0/3567964271 wait complete. 2024-02-14T15:02:21.618 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738454 for osd.0 2024-02-14T15:02:21.619 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:22.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808869 got 287762808873 for osd.9 2024-02-14T15:02:22.006 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:22.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 -- 172.21.15.17:0/2651794409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48106be0 msgr2=0x7fac48106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:22.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/2651794409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48106be0 0x7fac48106fc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac440099c0 tx=0x7fac44009cd0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 -- 172.21.15.17:0/2651794409 shutdown_connections 2024-02-14T15:02:22.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/2651794409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac481014f0 0x7fac4810d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/2651794409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48106be0 0x7fac48106fc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/2651794409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac48100b30 0x7fac48100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 -- 172.21.15.17:0/2651794409 >> 172.21.15.17:0/2651794409 conn(0x7fac480fc5e0 msgr2=0x7fac480fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:22.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 -- 172.21.15.17:0/2651794409 shutdown_connections 2024-02-14T15:02:22.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.224+0000 7fac4e7a2700 1 -- 172.21.15.17:0/2651794409 wait complete. 2024-02-14T15:02:22.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 Processor -- start 2024-02-14T15:02:22.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 -- start start 2024-02-14T15:02:22.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48100b30 0x7fac48197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac481014f0 0x7fac48198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac48106be0 0x7fac4819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4819ced0 con 0x7fac48100b30 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4819d040 con 0x7fac48106be0 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.225+0000 7fac4e7a2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4819d200 con 0x7fac481014f0 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48100b30 0x7fac48197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48100b30 0x7fac48197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37752/0 (socket says 172.21.15.17:37752) 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4cf9f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac481014f0 0x7fac48198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 -- 172.21.15.17:0/1064666944 learned_addr learned my addr 172.21.15.17:0/1064666944 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4dfa1700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac48106be0 0x7fac4819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 -- 172.21.15.17:0/1064666944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac481014f0 msgr2=0x7fac48198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac481014f0 0x7fac48198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 -- 172.21.15.17:0/1064666944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac48106be0 msgr2=0x7fac4819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac48106be0 0x7fac4819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 -- 172.21.15.17:0/1064666944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac440095f0 con 0x7fac48100b30 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4d7a0700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48100b30 0x7fac48197ff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b580 tx=0x7fac44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac3e7fc700 1 -- 172.21.15.17:0/1064666944 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4401d070 con 0x7fac48100b30 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4819d480 con 0x7fac48100b30 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac3e7fc700 1 -- 172.21.15.17:0/1064666944 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac44003f50 con 0x7fac48100b30 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac3e7fc700 1 -- 172.21.15.17:0/1064666944 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44022d10 con 0x7fac48100b30 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.226+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac481026d0 con 0x7fac48100b30 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.228+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac481021e0 con 0x7fac48100b30 2024-02-14T15:02:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.229+0000 7fac3e7fc700 1 -- 172.21.15.17:0/1064666944 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fac44022470 con 0x7fac48100b30 2024-02-14T15:02:22.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.229+0000 7fac3e7fc700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac340a65f0 0x7fac340a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:22.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.229+0000 7fac4cf9f700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac340a65f0 0x7fac340a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:22.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.230+0000 7fac3e7fc700 1 -- 172.21.15.17:0/1064666944 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fac440ceed0 con 0x7fac48100b30 2024-02-14T15:02:22.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.230+0000 7fac4cf9f700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac340a65f0 0x7fac340a8ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac481061d0 tx=0x7fac3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:22.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.234+0000 7fac3e7fc700 1 -- 172.21.15.17:0/1064666944 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac44097ad0 con 0x7fac48100b30 2024-02-14T15:02:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.438+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fac48199350 con 0x7fac48100b30 2024-02-14T15:02:22.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.438+0000 7fac3e7fc700 1 -- 172.21.15.17:0/1064666944 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac4409b190 con 0x7fac48100b30 2024-02-14T15:02:22.439 INFO:teuthology.orchestra.run.smithi017.stdout:141733920835 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac340a65f0 msgr2=0x7fac340a8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac340a65f0 0x7fac340a8ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac481061d0 tx=0x7fac3800b410 comp rx=0 tx=0).stop 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48100b30 msgr2=0x7fac48197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48100b30 0x7fac48197ff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b580 tx=0x7fac44004730 comp rx=0 tx=0).stop 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 shutdown_connections 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fac340a65f0 0x7fac340a8ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac48106be0 0x7fac4819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac48100b30 0x7fac48197ff0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 --2- 172.21.15.17:0/1064666944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac481014f0 0x7fac48198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 >> 172.21.15.17:0/1064666944 conn(0x7fac480fc5e0 msgr2=0x7fac480fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 shutdown_connections 2024-02-14T15:02:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.440+0000 7fac4e7a2700 1 -- 172.21.15.17:0/1064666944 wait complete. 2024-02-14T15:02:22.845 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:22 smithi050 ceph-mon[36512]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:22.845 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:22 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1064666944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T15:02:22.937 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:22 smithi076 ceph-mon[37057]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:22.937 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:22 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1064666944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T15:02:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:22 smithi017 ceph-mon[32549]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:22 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1064666944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T15:02:23.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920835 for osd.4 2024-02-14T15:02:23.261 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:23.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 -- 172.21.15.17:0/3815841337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 msgr2=0x7fe1b8106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:23.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 --2- 172.21.15.17:0/3815841337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8106960 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009990 tx=0x7fe1a4009ca0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 -- 172.21.15.17:0/3815841337 shutdown_connections 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 --2- 172.21.15.17:0/3815841337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b8107040 0x7fe1b810ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 --2- 172.21.15.17:0/3815841337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b8101f60 0x7fe1b8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 --2- 172.21.15.17:0/3815841337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8106960 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 -- 172.21.15.17:0/3815841337 >> 172.21.15.17:0/3815841337 conn(0x7fe1b80fd7d0 msgr2=0x7fe1b80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 -- 172.21.15.17:0/3815841337 shutdown_connections 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 -- 172.21.15.17:0/3815841337 wait complete. 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 Processor -- start 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.556+0000 7fe1bef83700 1 -- start start 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bef83700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b8101f60 0x7fe1b8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bef83700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bef83700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b8107040 0x7fe1b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bef83700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b81a6650 con 0x7fe1b8102910 2024-02-14T15:02:23.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bef83700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b81a67c0 con 0x7fe1b8101f60 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bef83700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b81a6980 con 0x7fe1b8107040 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37766/0 (socket says 172.21.15.17:37766) 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 -- 172.21.15.17:0/1972134112 learned_addr learned my addr 172.21.15.17:0/1972134112 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bcd1f700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b8101f60 0x7fe1b8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stdout:309237645349 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bd520700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b8107040 0x7fe1b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:23.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 -- 172.21.15.17:0/1972134112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b8107040 msgr2=0x7fe1b8074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b8107040 0x7fe1b8074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 -- 172.21.15.17:0/1972134112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b8101f60 msgr2=0x7fe1b8074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b8101f60 0x7fe1b8074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 -- 172.21.15.17:0/1972134112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a40095f0 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bd520700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b8107040 0x7fe1b8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b7fff700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8074560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4000c00 tx=0x7fe1a400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b5ffb700 1 -- 172.21.15.17:0/1972134112 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a401d070 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b5ffb700 1 -- 172.21.15.17:0/1972134112 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1a4003680 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1b5ffb700 1 -- 172.21.15.17:0/1972134112 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4017650 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.557+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b81a6c00 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.558+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b81a7010 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.558+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b810a360 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.560+0000 7fe1b5ffb700 1 -- 172.21.15.17:0/1972134112 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe1a4022470 con 0x7fe1b8102910 2024-02-14T15:02:23.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.561+0000 7fe1b5ffb700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1a80a21e0 0x7fe1a80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.561+0000 7fe1b5ffb700 1 -- 172.21.15.17:0/1972134112 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe1a40cf4f0 con 0x7fe1b8102910 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.561+0000 7fe1bcd1f700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1a80a21e0 0x7fe1a80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.562+0000 7fe1bcd1f700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1a80a21e0 0x7fe1a80a46a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00ac50 tx=0x7fe1ac00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.565+0000 7fe1b5ffb700 1 -- 172.21.15.17:0/1972134112 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1a40980f0 con 0x7fe1b8102910 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.772+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fe1b80689e0 con 0x7fe1b8102910 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.772+0000 7fe1b5ffb700 1 -- 172.21.15.17:0/1972134112 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe1a4022e60 con 0x7fe1b8102910 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1a80a21e0 msgr2=0x7fe1a80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1a80a21e0 0x7fe1a80a46a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00ac50 tx=0x7fe1ac00a380 comp rx=0 tx=0).stop 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 msgr2=0x7fe1b8074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8074560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4000c00 tx=0x7fe1a400f670 comp rx=0 tx=0).stop 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 shutdown_connections 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe1a80a21e0 0x7fe1a80a46a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1b8107040 0x7fe1b8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b8101f60 0x7fe1b8074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 --2- 172.21.15.17:0/1972134112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1b8102910 0x7fe1b8074560 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:23.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 >> 172.21.15.17:0/1972134112 conn(0x7fe1b80fd7d0 msgr2=0x7fe1b80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:23.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 shutdown_connections 2024-02-14T15:02:23.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:22.775+0000 7fe1bef83700 1 -- 172.21.15.17:0/1972134112 wait complete. 2024-02-14T15:02:23.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:23 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1972134112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-14T15:02:23.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:23 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1972134112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-14T15:02:23.955 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:23 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1972134112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-14T15:02:24.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645344 got 309237645349 for osd.10 2024-02-14T15:02:24.379 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:24.616 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:24 smithi017 ceph-mon[32549]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 -- 172.21.15.17:0/2851535876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15880699d0 msgr2=0x7f1588069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 --2- 172.21.15.17:0/2851535876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15880699d0 0x7f1588069e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f157c0099c0 tx=0x7f157c009cd0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 -- 172.21.15.17:0/2851535876 shutdown_connections 2024-02-14T15:02:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 --2- 172.21.15.17:0/2851535876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f158806a370 0x7f158810ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 --2- 172.21.15.17:0/2851535876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1588107d70 0x7f1588108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 --2- 172.21.15.17:0/2851535876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15880699d0 0x7f1588069e30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 -- 172.21.15.17:0/2851535876 >> 172.21.15.17:0/2851535876 conn(0x7f1588077b50 msgr2=0x7f1588077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 -- 172.21.15.17:0/2851535876 shutdown_connections 2024-02-14T15:02:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.648+0000 7f158e47d700 1 -- 172.21.15.17:0/2851535876 wait complete. 2024-02-14T15:02:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.649+0000 7f158e47d700 1 Processor -- start 2024-02-14T15:02:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.649+0000 7f158e47d700 1 -- start start 2024-02-14T15:02:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f158e47d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15880699d0 0x7f1588198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f158e47d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f158806a370 0x7f15881987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f158e47d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1588107d70 0x7f158819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:24.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f158e47d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158819d160 con 0x7f1588107d70 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f158e47d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158819d2d0 con 0x7f158806a370 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f158e47d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158819d490 con 0x7f15880699d0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15880699d0 0x7f1588198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15880699d0 0x7f1588198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:39636/0 (socket says 172.21.15.17:39636) 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 -- 172.21.15.17:0/3428423 learned_addr learned my addr 172.21.15.17:0/3428423 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f158ca1a700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1588107d70 0x7f158819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f15877fe700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f158806a370 0x7f15881987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 -- 172.21.15.17:0/3428423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f158806a370 msgr2=0x7f15881987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f158806a370 0x7f15881987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 -- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1588107d70 msgr2=0x7f158819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1588107d70 0x7f158819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.650+0000 7f1587fff700 1 -- 172.21.15.17:0/3428423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0095f0 con 0x7f15880699d0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.651+0000 7f1587fff700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15880699d0 0x7f1588198280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f157800e9a0 tx=0x7f157800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.651+0000 7f15857fa700 1 -- 172.21.15.17:0/3428423 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578009790 con 0x7f15880699d0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.651+0000 7f15857fa700 1 -- 172.21.15.17:0/3428423 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f157800cbd0 con 0x7f15880699d0 2024-02-14T15:02:24.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.651+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158819d770 con 0x7f15880699d0 2024-02-14T15:02:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.651+0000 7f15857fa700 1 -- 172.21.15.17:0/3428423 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578018680 con 0x7f15880699d0 2024-02-14T15:02:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.651+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1588100df0 con 0x7f15880699d0 2024-02-14T15:02:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.652+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158810a290 con 0x7f15880699d0 2024-02-14T15:02:24.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.654+0000 7f15857fa700 1 -- 172.21.15.17:0/3428423 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f157800cd40 con 0x7f15880699d0 2024-02-14T15:02:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.654+0000 7f15857fa700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f15700a21e0 0x7f15700a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.654+0000 7f15857fa700 1 -- 172.21.15.17:0/3428423 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1578014070 con 0x7f15880699d0 2024-02-14T15:02:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.654+0000 7f15877fe700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f15700a21e0 0x7f15700a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:24.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.655+0000 7f15877fe700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f15700a21e0 0x7f15700a46a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f157c000c00 tx=0x7f157c005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:24.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.658+0000 7f15857fa700 1 -- 172.21.15.17:0/3428423 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1578097390 con 0x7f15880699d0 2024-02-14T15:02:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.884+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1588103420 con 0x7f15880699d0 2024-02-14T15:02:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.884+0000 7f15857fa700 1 -- 172.21.15.17:0/3428423 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1578026090 con 0x7f15880699d0 2024-02-14T15:02:24.885 INFO:teuthology.orchestra.run.smithi017.stdout:77309411406 2024-02-14T15:02:24.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f15700a21e0 msgr2=0x7f15700a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f15700a21e0 0x7f15700a46a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f157c000c00 tx=0x7f157c005940 comp rx=0 tx=0).stop 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15880699d0 msgr2=0x7f1588198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15880699d0 0x7f1588198280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f157800e9a0 tx=0x7f157800ecb0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 shutdown_connections 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f15700a21e0 0x7f15700a46a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1588107d70 0x7f158819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15880699d0 0x7f1588198280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 --2- 172.21.15.17:0/3428423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f158806a370 0x7f15881987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 >> 172.21.15.17:0/3428423 conn(0x7f1588077b50 msgr2=0x7f15880ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 shutdown_connections 2024-02-14T15:02:24.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:24.887+0000 7f158e47d700 1 -- 172.21.15.17:0/3428423 wait complete. 2024-02-14T15:02:24.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:24 smithi050 ceph-mon[36512]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:24 smithi076 ceph-mon[37057]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:25.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411406 for osd.2 2024-02-14T15:02:25.683 DEBUG:teuthology.parallel:result is None 2024-02-14T15:02:25.683 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-14T15:02:25.683 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T15:02:25.940 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:25 smithi076 ceph-mon[37057]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:25 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:25 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:25 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3428423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T15:02:26.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:25 smithi017 ceph-mon[32549]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:26.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:25 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:26.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:25 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:26.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:25 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3428423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T15:02:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:25 smithi050 ceph-mon[36512]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:25 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:25 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:25 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3428423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T15:02:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:02:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:02:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:26 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:02:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:02:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:26 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:02:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:02:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:26 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' 2024-02-14T15:02:27.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 -- 172.21.15.17:0/4158990285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8101860 msgr2=0x7ff3d8105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/4158990285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8101860 0x7ff3d8105860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009990 tx=0x7ff3cc009ca0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 -- 172.21.15.17:0/4158990285 shutdown_connections 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/4158990285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d8105f40 0x7ff3d810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/4158990285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d8100eb0 0x7ff3d8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/4158990285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8101860 0x7ff3d8105860 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 -- 172.21.15.17:0/4158990285 >> 172.21.15.17:0/4158990285 conn(0x7ff3d8076c40 msgr2=0x7ff3d8077050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 -- 172.21.15.17:0/4158990285 shutdown_connections 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.625+0000 7ff3dfd61700 1 -- 172.21.15.17:0/4158990285 wait complete. 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 Processor -- start 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 -- start start 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d8100eb0 0x7ff3d8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d8101860 0x7ff3d8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8105f40 0x7ff3d81a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d8199f20 con 0x7ff3d8105f40 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81a46e0 con 0x7ff3d8101860 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dfd61700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81a4820 con 0x7ff3d8100eb0 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3de2fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8105f40 0x7ff3d81a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3de2fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8105f40 0x7ff3d81a41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37806/0 (socket says 172.21.15.17:37806) 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3de2fe700 1 -- 172.21.15.17:0/647170900 learned_addr learned my addr 172.21.15.17:0/647170900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3ddafd700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d8100eb0 0x7ff3d8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.626+0000 7ff3dd2fc700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d8101860 0x7ff3d8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3de2fe700 1 -- 172.21.15.17:0/647170900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d8100eb0 msgr2=0x7ff3d8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3de2fe700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d8100eb0 0x7ff3d8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3de2fe700 1 -- 172.21.15.17:0/647170900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d8101860 msgr2=0x7ff3d8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3de2fe700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d8101860 0x7ff3d8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3de2fe700 1 -- 172.21.15.17:0/647170900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc0095f0 con 0x7ff3d8105f40 2024-02-14T15:02:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3dd2fc700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d8101860 0x7ff3d8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3de2fe700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8105f40 0x7ff3d81a41a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000d980 tx=0x7ff3d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3d6ffd700 1 -- 172.21.15.17:0/647170900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0004a30 con 0x7ff3d8105f40 2024-02-14T15:02:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d81a4b00 con 0x7ff3d8105f40 2024-02-14T15:02:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3d6ffd700 1 -- 172.21.15.17:0/647170900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3d000b5c0 con 0x7ff3d8105f40 2024-02-14T15:02:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3d6ffd700 1 -- 172.21.15.17:0/647170900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0009bf0 con 0x7ff3d8105f40 2024-02-14T15:02:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.627+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d81a5000 con 0x7ff3d8105f40 2024-02-14T15:02:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.628+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d81091f0 con 0x7ff3d8105f40 2024-02-14T15:02:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.629+0000 7ff3d6ffd700 1 -- 172.21.15.17:0/647170900 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff3d0009d50 con 0x7ff3d8105f40 2024-02-14T15:02:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.630+0000 7ff3d6ffd700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff3c00a21e0 0x7ff3c00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.630+0000 7ff3ddafd700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff3c00a21e0 0x7ff3c00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.630+0000 7ff3d6ffd700 1 -- 172.21.15.17:0/647170900 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff3d00ce2f0 con 0x7ff3d8105f40 2024-02-14T15:02:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.630+0000 7ff3ddafd700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff3c00a21e0 0x7ff3c00a46a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800a400 tx=0x7ff3c800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.633+0000 7ff3d6ffd700 1 -- 172.21.15.17:0/647170900 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3d0096f00 con 0x7ff3d8105f40 2024-02-14T15:02:27.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.832+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3d8110ce0 con 0x7ff3c00a21e0 2024-02-14T15:02:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.845+0000 7ff3d6ffd700 1 -- 172.21.15.17:0/647170900 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427763 (secure 0 0 0) 0x7ff3d8110ce0 con 0x7ff3c00a21e0 2024-02-14T15:02:27.853 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff3c00a21e0 msgr2=0x7ff3c00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff3c00a21e0 0x7ff3c00a46a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800a400 tx=0x7ff3c800d040 comp rx=0 tx=0).stop 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8105f40 msgr2=0x7ff3d81a41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8105f40 0x7ff3d81a41a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000d980 tx=0x7ff3d000dc90 comp rx=0 tx=0).stop 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 shutdown_connections 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff3c00a21e0 0x7ff3c00a46a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3d8105f40 0x7ff3d81a41a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d8100eb0 0x7ff3d8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 --2- 172.21.15.17:0/647170900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d8101860 0x7ff3d8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 >> 172.21.15.17:0/647170900 conn(0x7ff3d8076c40 msgr2=0x7ff3d81070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 shutdown_connections 2024-02-14T15:02:27.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:27.849+0000 7ff3dfd61700 1 -- 172.21.15.17:0/647170900 wait complete. 2024-02-14T15:02:27.858 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-02-14T15:02:28.159 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:27 smithi017 ceph-mon[32549]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:28.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:27 smithi076 ceph-mon[37057]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:28.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:27 smithi050 ceph-mon[36512]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:28.626 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-02-14T15:02:26.678536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657959,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":572,"num_read_kb":406,"num_write":344,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1244674,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388304,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553520,"statfs":{"total":1151940427776,"available":1151542804480,"internally_reserved":0,"allocated":14385152,"data_stored":7949519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003064"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604662+0000","last_change":"2024-02-14T15:00:49.281057+0000","last_active":"2024-02-14T15:01:44.604662+0000","last_peered":"2024-02-14T15:01:44.604662+0000","last_clean":"2024-02-14T15:01:44.604662+0000","last_became_active":"2024-02-14T15:00:49.280642+0000","last_became_peered":"2024-02-14T15:00:49.280642+0000","last_unstale":"2024-02-14T15:01:44.604662+0000","last_undegraded":"2024-02-14T15:01:44.604662+0000","last_fullsized":"2024-02-14T15:01:44.604662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:03.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597007+0000","last_change":"2024-02-14T15:00:49.282794+0000","last_active":"2024-02-14T15:01:44.597007+0000","last_peered":"2024-02-14T15:01:44.597007+0000","last_clean":"2024-02-14T15:01:44.597007+0000","last_became_active":"2024-02-14T15:00:49.281441+0000","last_became_peered":"2024-02-14T15:00:49.281441+0000","last_unstale":"2024-02-14T15:01:44.597007+0000","last_undegraded":"2024-02-14T15:01:44.597007+0000","last_fullsized":"2024-02-14T15:01:44.597007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:57:36.577281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596895+0000","last_change":"2024-02-14T15:00:49.286253+0000","last_active":"2024-02-14T15:01:44.596895+0000","last_peered":"2024-02-14T15:01:44.596895+0000","last_clean":"2024-02-14T15:01:44.596895+0000","last_became_active":"2024-02-14T15:00:49.285950+0000","last_became_peered":"2024-02-14T15:00:49.285950+0000","last_unstale":"2024-02-14T15:01:44.596895+0000","last_undegraded":"2024-02-14T15:01:44.596895+0000","last_fullsized":"2024-02-14T15:01:44.596895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:53.395339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598104+0000","last_change":"2024-02-14T15:00:49.272764+0000","last_active":"2024-02-14T15:01:44.598104+0000","last_peered":"2024-02-14T15:01:44.598104+0000","last_clean":"2024-02-14T15:01:44.598104+0000","last_became_active":"2024-02-14T15:00:49.272430+0000","last_became_peered":"2024-02-14T15:00:49.272430+0000","last_unstale":"2024-02-14T15:01:44.598104+0000","last_undegraded":"2024-02-14T15:01:44.598104+0000","last_fullsized":"2024-02-14T15:01:44.598104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:36.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603685+0000","last_change":"2024-02-14T15:00:49.275028+0000","last_active":"2024-02-14T15:01:44.603685+0000","last_peered":"2024-02-14T15:01:44.603685+0000","last_clean":"2024-02-14T15:01:44.603685+0000","last_became_active":"2024-02-14T15:00:49.274686+0000","last_became_peered":"2024-02-14T15:00:49.274686+0000","last_unstale":"2024-02-14T15:01:44.603685+0000","last_undegraded":"2024-02-14T15:01:44.603685+0000","last_fullsized":"2024-02-14T15:01:44.603685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:21:23.465321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601040+0000","last_change":"2024-02-14T15:00:49.287144+0000","last_active":"2024-02-14T15:01:44.601040+0000","last_peered":"2024-02-14T15:01:44.601040+0000","last_clean":"2024-02-14T15:01:44.601040+0000","last_became_active":"2024-02-14T15:00:49.286767+0000","last_became_peered":"2024-02-14T15:00:49.286767+0000","last_unstale":"2024-02-14T15:01:44.601040+0000","last_undegraded":"2024-02-14T15:01:44.601040+0000","last_fullsized":"2024-02-14T15:01:44.601040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:45:09.391595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604308+0000","last_change":"2024-02-14T15:00:49.274442+0000","last_active":"2024-02-14T15:01:44.604308+0000","last_peered":"2024-02-14T15:01:44.604308+0000","last_clean":"2024-02-14T15:01:44.604308+0000","last_became_active":"2024-02-14T15:00:49.274122+0000","last_became_peered":"2024-02-14T15:00:49.274122+0000","last_unstale":"2024-02-14T15:01:44.604308+0000","last_undegraded":"2024-02-14T15:01:44.604308+0000","last_fullsized":"2024-02-14T15:01:44.604308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:55.974889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601679+0000","last_change":"2024-02-14T15:00:49.285432+0000","last_active":"2024-02-14T15:01:44.601679+0000","last_peered":"2024-02-14T15:01:44.601679+0000","last_clean":"2024-02-14T15:01:44.601679+0000","last_became_active":"2024-02-14T15:00:49.283703+0000","last_became_peered":"2024-02-14T15:00:49.283703+0000","last_unstale":"2024-02-14T15:01:44.601679+0000","last_undegraded":"2024-02-14T15:01:44.601679+0000","last_fullsized":"2024-02-14T15:01:44.601679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:14.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600470+0000","last_change":"2024-02-14T15:00:49.283908+0000","last_active":"2024-02-14T15:01:44.600470+0000","last_peered":"2024-02-14T15:01:44.600470+0000","last_clean":"2024-02-14T15:01:44.600470+0000","last_became_active":"2024-02-14T15:00:49.283131+0000","last_became_peered":"2024-02-14T15:00:49.283131+0000","last_unstale":"2024-02-14T15:01:44.600470+0000","last_undegraded":"2024-02-14T15:01:44.600470+0000","last_fullsized":"2024-02-14T15:01:44.600470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:42.247624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600593+0000","last_change":"2024-02-14T15:00:49.287229+0000","last_active":"2024-02-14T15:01:44.600593+0000","last_peered":"2024-02-14T15:01:44.600593+0000","last_clean":"2024-02-14T15:01:44.600593+0000","last_became_active":"2024-02-14T15:00:49.286791+0000","last_became_peered":"2024-02-14T15:00:49.286791+0000","last_unstale":"2024-02-14T15:01:44.600593+0000","last_undegraded":"2024-02-14T15:01:44.600593+0000","last_fullsized":"2024-02-14T15:01:44.600593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:33:01.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605011+0000","last_change":"2024-02-14T15:00:49.281669+0000","last_active":"2024-02-14T15:01:44.605011+0000","last_peered":"2024-02-14T15:01:44.605011+0000","last_clean":"2024-02-14T15:01:44.605011+0000","last_became_active":"2024-02-14T15:00:49.281298+0000","last_became_peered":"2024-02-14T15:00:49.281298+0000","last_unstale":"2024-02-14T15:01:44.605011+0000","last_undegraded":"2024-02-14T15:01:44.605011+0000","last_fullsized":"2024-02-14T15:01:44.605011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:54:40.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598239+0000","last_change":"2024-02-14T15:00:49.289679+0000","last_active":"2024-02-14T15:01:44.598239+0000","last_peered":"2024-02-14T15:01:44.598239+0000","last_clean":"2024-02-14T15:01:44.598239+0000","last_became_active":"2024-02-14T15:00:49.288767+0000","last_became_peered":"2024-02-14T15:00:49.288767+0000","last_unstale":"2024-02-14T15:01:44.598239+0000","last_undegraded":"2024-02-14T15:01:44.598239+0000","last_fullsized":"2024-02-14T15:01:44.598239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:16:20.181517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605558+0000","last_change":"2024-02-14T15:00:49.257471+0000","last_active":"2024-02-14T15:01:44.605558+0000","last_peered":"2024-02-14T15:01:44.605558+0000","last_clean":"2024-02-14T15:01:44.605558+0000","last_became_active":"2024-02-14T15:00:49.256766+0000","last_became_peered":"2024-02-14T15:00:49.256766+0000","last_unstale":"2024-02-14T15:01:44.605558+0000","last_undegraded":"2024-02-14T15:01:44.605558+0000","last_fullsized":"2024-02-14T15:01:44.605558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:14:00.525621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596566+0000","last_change":"2024-02-14T15:00:49.278462+0000","last_active":"2024-02-14T15:01:44.596566+0000","last_peered":"2024-02-14T15:01:44.596566+0000","last_clean":"2024-02-14T15:01:44.596566+0000","last_became_active":"2024-02-14T15:00:49.278114+0000","last_became_peered":"2024-02-14T15:00:49.278114+0000","last_unstale":"2024-02-14T15:01:44.596566+0000","last_undegraded":"2024-02-14T15:01:44.596566+0000","last_fullsized":"2024-02-14T15:01:44.596566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:02:16.601359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596566+0000","last_change":"2024-02-14T15:00:49.270653+0000","last_active":"2024-02-14T15:01:44.596566+0000","last_peered":"2024-02-14T15:01:44.596566+0000","last_clean":"2024-02-14T15:01:44.596566+0000","last_became_active":"2024-02-14T15:00:49.270334+0000","last_became_peered":"2024-02-14T15:00:49.270334+0000","last_unstale":"2024-02-14T15:01:44.596566+0000","last_undegraded":"2024-02-14T15:01:44.596566+0000","last_fullsized":"2024-02-14T15:01:44.596566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:13:53.840909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599429+0000","last_change":"2024-02-14T15:00:49.280597+0000","last_active":"2024-02-14T15:01:44.599429+0000","last_peered":"2024-02-14T15:01:44.599429+0000","last_clean":"2024-02-14T15:01:44.599429+0000","last_became_active":"2024-02-14T15:00:49.279612+0000","last_became_peered":"2024-02-14T15:00:49.279612+0000","last_unstale":"2024-02-14T15:01:44.599429+0000","last_undegraded":"2024-02-14T15:01:44.599429+0000","last_fullsized":"2024-02-14T15:01:44.599429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:11.676608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604531+0000","last_change":"2024-02-14T15:00:49.281422+0000","last_active":"2024-02-14T15:01:44.604531+0000","last_peered":"2024-02-14T15:01:44.604531+0000","last_clean":"2024-02-14T15:01:44.604531+0000","last_became_active":"2024-02-14T15:00:49.281048+0000","last_became_peered":"2024-02-14T15:00:49.281048+0000","last_unstale":"2024-02-14T15:01:44.604531+0000","last_undegraded":"2024-02-14T15:01:44.604531+0000","last_fullsized":"2024-02-14T15:01:44.604531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:04.994390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599774+0000","last_change":"2024-02-14T15:00:49.288245+0000","last_active":"2024-02-14T15:01:44.599774+0000","last_peered":"2024-02-14T15:01:44.599774+0000","last_clean":"2024-02-14T15:01:44.599774+0000","last_became_active":"2024-02-14T15:00:49.287908+0000","last_became_peered":"2024-02-14T15:00:49.287908+0000","last_unstale":"2024-02-14T15:01:44.599774+0000","last_undegraded":"2024-02-14T15:01:44.599774+0000","last_fullsized":"2024-02-14T15:01:44.599774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:52.397826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598096+0000","last_change":"2024-02-14T15:00:49.275428+0000","last_active":"2024-02-14T15:01:44.598096+0000","last_peered":"2024-02-14T15:01:44.598096+0000","last_clean":"2024-02-14T15:01:44.598096+0000","last_became_active":"2024-02-14T15:00:49.273464+0000","last_became_peered":"2024-02-14T15:00:49.273464+0000","last_unstale":"2024-02-14T15:01:44.598096+0000","last_undegraded":"2024-02-14T15:01:44.598096+0000","last_fullsized":"2024-02-14T15:01:44.598096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:34.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597554+0000","last_change":"2024-02-14T15:00:53.587219+0000","last_active":"2024-02-14T15:01:44.597554+0000","last_peered":"2024-02-14T15:01:44.597554+0000","last_clean":"2024-02-14T15:01:44.597554+0000","last_became_active":"2024-02-14T15:00:53.586917+0000","last_became_peered":"2024-02-14T15:00:53.586917+0000","last_unstale":"2024-02-14T15:01:44.597554+0000","last_undegraded":"2024-02-14T15:01:44.597554+0000","last_fullsized":"2024-02-14T15:01:44.597554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:39:38.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599865+0000","last_change":"2024-02-14T15:00:48.269243+0000","last_active":"2024-02-14T15:01:44.599865+0000","last_peered":"2024-02-14T15:01:44.599865+0000","last_clean":"2024-02-14T15:01:44.599865+0000","last_became_active":"2024-02-14T15:00:48.268883+0000","last_became_peered":"2024-02-14T15:00:48.268883+0000","last_unstale":"2024-02-14T15:01:44.599865+0000","last_undegraded":"2024-02-14T15:01:44.599865+0000","last_fullsized":"2024-02-14T15:01:44.599865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:08.530092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602527+0000","last_change":"2024-02-14T15:00:49.282458+0000","last_active":"2024-02-14T15:01:44.602527+0000","last_peered":"2024-02-14T15:01:44.602527+0000","last_clean":"2024-02-14T15:01:44.602527+0000","last_became_active":"2024-02-14T15:00:49.281982+0000","last_became_peered":"2024-02-14T15:00:49.281982+0000","last_unstale":"2024-02-14T15:01:44.602527+0000","last_undegraded":"2024-02-14T15:01:44.602527+0000","last_fullsized":"2024-02-14T15:01:44.602527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:51:33.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597726+0000","last_change":"2024-02-14T15:00:53.586578+0000","last_active":"2024-02-14T15:01:44.597726+0000","last_peered":"2024-02-14T15:01:44.597726+0000","last_clean":"2024-02-14T15:01:44.597726+0000","last_became_active":"2024-02-14T15:00:53.586109+0000","last_became_peered":"2024-02-14T15:00:53.586109+0000","last_unstale":"2024-02-14T15:01:44.597726+0000","last_undegraded":"2024-02-14T15:01:44.597726+0000","last_fullsized":"2024-02-14T15:01:44.597726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:07.332435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602661+0000","last_change":"2024-02-14T15:00:48.281953+0000","last_active":"2024-02-14T15:01:44.602661+0000","last_peered":"2024-02-14T15:01:44.602661+0000","last_clean":"2024-02-14T15:01:44.602661+0000","last_became_active":"2024-02-14T15:00:48.281590+0000","last_became_peered":"2024-02-14T15:00:48.281590+0000","last_unstale":"2024-02-14T15:01:44.602661+0000","last_undegraded":"2024-02-14T15:01:44.602661+0000","last_fullsized":"2024-02-14T15:01:44.602661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:22.991037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597012+0000","last_change":"2024-02-14T15:00:49.284729+0000","last_active":"2024-02-14T15:01:44.597012+0000","last_peered":"2024-02-14T15:01:44.597012+0000","last_clean":"2024-02-14T15:01:44.597012+0000","last_became_active":"2024-02-14T15:00:49.284013+0000","last_became_peered":"2024-02-14T15:00:49.284013+0000","last_unstale":"2024-02-14T15:01:44.597012+0000","last_undegraded":"2024-02-14T15:01:44.597012+0000","last_fullsized":"2024-02-14T15:01:44.597012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:03:31.313032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598204+0000","last_change":"2024-02-14T15:00:53.589825+0000","last_active":"2024-02-14T15:01:44.598204+0000","last_peered":"2024-02-14T15:01:44.598204+0000","last_clean":"2024-02-14T15:01:44.598204+0000","last_became_active":"2024-02-14T15:00:53.589424+0000","last_became_peered":"2024-02-14T15:00:53.589424+0000","last_unstale":"2024-02-14T15:01:44.598204+0000","last_undegraded":"2024-02-14T15:01:44.598204+0000","last_fullsized":"2024-02-14T15:01:44.598204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:24.522152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602765+0000","last_change":"2024-02-14T15:00:48.276884+0000","last_active":"2024-02-14T15:01:44.602765+0000","last_peered":"2024-02-14T15:01:44.602765+0000","last_clean":"2024-02-14T15:01:44.602765+0000","last_became_active":"2024-02-14T15:00:48.276576+0000","last_became_peered":"2024-02-14T15:00:48.276576+0000","last_unstale":"2024-02-14T15:01:44.602765+0000","last_undegraded":"2024-02-14T15:01:44.602765+0000","last_fullsized":"2024-02-14T15:01:44.602765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:25.793368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600320+0000","last_change":"2024-02-14T15:00:49.277602+0000","last_active":"2024-02-14T15:01:44.600320+0000","last_peered":"2024-02-14T15:01:44.600320+0000","last_clean":"2024-02-14T15:01:44.600320+0000","last_became_active":"2024-02-14T15:00:49.277249+0000","last_became_peered":"2024-02-14T15:00:49.277249+0000","last_unstale":"2024-02-14T15:01:44.600320+0000","last_undegraded":"2024-02-14T15:01:44.600320+0000","last_fullsized":"2024-02-14T15:01:44.600320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.347962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597955+0000","last_change":"2024-02-14T15:00:53.587838+0000","last_active":"2024-02-14T15:01:44.597955+0000","last_peered":"2024-02-14T15:01:44.597955+0000","last_clean":"2024-02-14T15:01:44.597955+0000","last_became_active":"2024-02-14T15:00:53.587012+0000","last_became_peered":"2024-02-14T15:00:53.587012+0000","last_unstale":"2024-02-14T15:01:44.597955+0000","last_undegraded":"2024-02-14T15:01:44.597955+0000","last_fullsized":"2024-02-14T15:01:44.597955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:32:12.276578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602772+0000","last_change":"2024-02-14T15:00:48.273529+0000","last_active":"2024-02-14T15:01:44.602772+0000","last_peered":"2024-02-14T15:01:44.602772+0000","last_clean":"2024-02-14T15:01:44.602772+0000","last_became_active":"2024-02-14T15:00:48.272555+0000","last_became_peered":"2024-02-14T15:00:48.272555+0000","last_unstale":"2024-02-14T15:01:44.602772+0000","last_undegraded":"2024-02-14T15:01:44.602772+0000","last_fullsized":"2024-02-14T15:01:44.602772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:19.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602297+0000","last_change":"2024-02-14T15:00:49.278834+0000","last_active":"2024-02-14T15:01:44.602297+0000","last_peered":"2024-02-14T15:01:44.602297+0000","last_clean":"2024-02-14T15:01:44.602297+0000","last_became_active":"2024-02-14T15:00:49.277095+0000","last_became_peered":"2024-02-14T15:00:49.277095+0000","last_unstale":"2024-02-14T15:01:44.602297+0000","last_undegraded":"2024-02-14T15:01:44.602297+0000","last_fullsized":"2024-02-14T15:01:44.602297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:27.291974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597933+0000","last_change":"2024-02-14T15:00:53.577162+0000","last_active":"2024-02-14T15:01:44.597933+0000","last_peered":"2024-02-14T15:01:44.597933+0000","last_clean":"2024-02-14T15:01:44.597933+0000","last_became_active":"2024-02-14T15:00:53.576809+0000","last_became_peered":"2024-02-14T15:00:53.576809+0000","last_unstale":"2024-02-14T15:01:44.597933+0000","last_undegraded":"2024-02-14T15:01:44.597933+0000","last_fullsized":"2024-02-14T15:01:44.597933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:37.698842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:23.660014+0000","last_change":"2024-02-14T15:00:48.276146+0000","last_active":"2024-02-14T15:02:23.660014+0000","last_peered":"2024-02-14T15:02:23.660014+0000","last_clean":"2024-02-14T15:02:23.660014+0000","last_became_active":"2024-02-14T15:00:48.275833+0000","last_became_peered":"2024-02-14T15:00:48.275833+0000","last_unstale":"2024-02-14T15:02:23.660014+0000","last_undegraded":"2024-02-14T15:02:23.660014+0000","last_fullsized":"2024-02-14T15:02:23.660014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:46.649486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602443+0000","last_change":"2024-02-14T15:00:49.267285+0000","last_active":"2024-02-14T15:01:44.602443+0000","last_peered":"2024-02-14T15:01:44.602443+0000","last_clean":"2024-02-14T15:01:44.602443+0000","last_became_active":"2024-02-14T15:00:49.265592+0000","last_became_peered":"2024-02-14T15:00:49.265592+0000","last_unstale":"2024-02-14T15:01:44.602443+0000","last_undegraded":"2024-02-14T15:01:44.602443+0000","last_fullsized":"2024-02-14T15:01:44.602443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:18.469835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597929+0000","last_change":"2024-02-14T15:00:53.582085+0000","last_active":"2024-02-14T15:01:44.597929+0000","last_peered":"2024-02-14T15:01:44.597929+0000","last_clean":"2024-02-14T15:01:44.597929+0000","last_became_active":"2024-02-14T15:00:53.581706+0000","last_became_peered":"2024-02-14T15:00:53.581706+0000","last_unstale":"2024-02-14T15:01:44.597929+0000","last_undegraded":"2024-02-14T15:01:44.597929+0000","last_fullsized":"2024-02-14T15:01:44.597929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:32.481142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601026+0000","last_change":"2024-02-14T15:00:48.268975+0000","last_active":"2024-02-14T15:01:44.601026+0000","last_peered":"2024-02-14T15:01:44.601026+0000","last_clean":"2024-02-14T15:01:44.601026+0000","last_became_active":"2024-02-14T15:00:48.268547+0000","last_became_peered":"2024-02-14T15:00:48.268547+0000","last_unstale":"2024-02-14T15:01:44.601026+0000","last_undegraded":"2024-02-14T15:01:44.601026+0000","last_fullsized":"2024-02-14T15:01:44.601026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:13.937917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601675+0000","last_change":"2024-02-14T15:00:49.276909+0000","last_active":"2024-02-14T15:01:44.601675+0000","last_peered":"2024-02-14T15:01:44.601675+0000","last_clean":"2024-02-14T15:01:44.601675+0000","last_became_active":"2024-02-14T15:00:49.276533+0000","last_became_peered":"2024-02-14T15:00:49.276533+0000","last_unstale":"2024-02-14T15:01:44.601675+0000","last_undegraded":"2024-02-14T15:01:44.601675+0000","last_fullsized":"2024-02-14T15:01:44.601675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:35:31.321455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604508+0000","last_change":"2024-02-14T15:00:53.592223+0000","last_active":"2024-02-14T15:01:44.604508+0000","last_peered":"2024-02-14T15:01:44.604508+0000","last_clean":"2024-02-14T15:01:44.604508+0000","last_became_active":"2024-02-14T15:00:53.591883+0000","last_became_peered":"2024-02-14T15:00:53.591883+0000","last_unstale":"2024-02-14T15:01:44.604508+0000","last_undegraded":"2024-02-14T15:01:44.604508+0000","last_fullsized":"2024-02-14T15:01:44.604508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:10.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605326+0000","last_change":"2024-02-14T15:00:48.274485+0000","last_active":"2024-02-14T15:01:44.605326+0000","last_peered":"2024-02-14T15:01:44.605326+0000","last_clean":"2024-02-14T15:01:44.605326+0000","last_became_active":"2024-02-14T15:00:48.274160+0000","last_became_peered":"2024-02-14T15:00:48.274160+0000","last_unstale":"2024-02-14T15:01:44.605326+0000","last_undegraded":"2024-02-14T15:01:44.605326+0000","last_fullsized":"2024-02-14T15:01:44.605326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:00:47.105069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601840+0000","last_change":"2024-02-14T15:00:49.272577+0000","last_active":"2024-02-14T15:01:44.601840+0000","last_peered":"2024-02-14T15:01:44.601840+0000","last_clean":"2024-02-14T15:01:44.601840+0000","last_became_active":"2024-02-14T15:00:49.271670+0000","last_became_peered":"2024-02-14T15:00:49.271670+0000","last_unstale":"2024-02-14T15:01:44.601840+0000","last_undegraded":"2024-02-14T15:01:44.601840+0000","last_fullsized":"2024-02-14T15:01:44.601840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:19.968389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596942+0000","last_change":"2024-02-14T15:00:53.581212+0000","last_active":"2024-02-14T15:01:44.596942+0000","last_peered":"2024-02-14T15:01:44.596942+0000","last_clean":"2024-02-14T15:01:44.596942+0000","last_became_active":"2024-02-14T15:00:53.580782+0000","last_became_peered":"2024-02-14T15:00:53.580782+0000","last_unstale":"2024-02-14T15:01:44.596942+0000","last_undegraded":"2024-02-14T15:01:44.596942+0000","last_fullsized":"2024-02-14T15:01:44.596942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:14.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600793+0000","last_change":"2024-02-14T15:00:48.273572+0000","last_active":"2024-02-14T15:01:44.600793+0000","last_peered":"2024-02-14T15:01:44.600793+0000","last_clean":"2024-02-14T15:01:44.600793+0000","last_became_active":"2024-02-14T15:00:48.273138+0000","last_became_peered":"2024-02-14T15:00:48.273138+0000","last_unstale":"2024-02-14T15:01:44.600793+0000","last_undegraded":"2024-02-14T15:01:44.600793+0000","last_fullsized":"2024-02-14T15:01:44.600793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:04.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600031+0000","last_change":"2024-02-14T15:00:49.288226+0000","last_active":"2024-02-14T15:01:44.600031+0000","last_peered":"2024-02-14T15:01:44.600031+0000","last_clean":"2024-02-14T15:01:44.600031+0000","last_became_active":"2024-02-14T15:00:49.287874+0000","last_became_peered":"2024-02-14T15:00:49.287874+0000","last_unstale":"2024-02-14T15:01:44.600031+0000","last_undegraded":"2024-02-14T15:01:44.600031+0000","last_fullsized":"2024-02-14T15:01:44.600031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:24:02.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602056+0000","last_change":"2024-02-14T15:00:53.590543+0000","last_active":"2024-02-14T15:01:44.602056+0000","last_peered":"2024-02-14T15:01:44.602056+0000","last_clean":"2024-02-14T15:01:44.602056+0000","last_became_active":"2024-02-14T15:00:53.589072+0000","last_became_peered":"2024-02-14T15:00:53.589072+0000","last_unstale":"2024-02-14T15:01:44.602056+0000","last_undegraded":"2024-02-14T15:01:44.602056+0000","last_fullsized":"2024-02-14T15:01:44.602056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:13.685655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:21.656574+0000","last_change":"2024-02-14T15:00:48.272027+0000","last_active":"2024-02-14T15:02:21.656574+0000","last_peered":"2024-02-14T15:02:21.656574+0000","last_clean":"2024-02-14T15:02:21.656574+0000","last_became_active":"2024-02-14T15:00:48.271587+0000","last_became_peered":"2024-02-14T15:00:48.271587+0000","last_unstale":"2024-02-14T15:02:21.656574+0000","last_undegraded":"2024-02-14T15:02:21.656574+0000","last_fullsized":"2024-02-14T15:02:21.656574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:33:03.566189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600906+0000","last_change":"2024-02-14T15:00:49.278936+0000","last_active":"2024-02-14T15:01:44.600906+0000","last_peered":"2024-02-14T15:01:44.600906+0000","last_clean":"2024-02-14T15:01:44.600906+0000","last_became_active":"2024-02-14T15:00:49.278318+0000","last_became_peered":"2024-02-14T15:00:49.278318+0000","last_unstale":"2024-02-14T15:01:44.600906+0000","last_undegraded":"2024-02-14T15:01:44.600906+0000","last_fullsized":"2024-02-14T15:01:44.600906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:01.327303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599341+0000","last_change":"2024-02-14T15:00:53.580340+0000","last_active":"2024-02-14T15:01:44.599341+0000","last_peered":"2024-02-14T15:01:44.599341+0000","last_clean":"2024-02-14T15:01:44.599341+0000","last_became_active":"2024-02-14T15:00:53.579742+0000","last_became_peered":"2024-02-14T15:00:53.579742+0000","last_unstale":"2024-02-14T15:01:44.599341+0000","last_undegraded":"2024-02-14T15:01:44.599341+0000","last_fullsized":"2024-02-14T15:01:44.599341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:38:33.182328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605218+0000","last_change":"2024-02-14T15:00:48.271891+0000","last_active":"2024-02-14T15:01:44.605218+0000","last_peered":"2024-02-14T15:01:44.605218+0000","last_clean":"2024-02-14T15:01:44.605218+0000","last_became_active":"2024-02-14T15:00:48.271419+0000","last_became_peered":"2024-02-14T15:00:48.271419+0000","last_unstale":"2024-02-14T15:01:44.605218+0000","last_undegraded":"2024-02-14T15:01:44.605218+0000","last_fullsized":"2024-02-14T15:01:44.605218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:03:04.725647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596985+0000","last_change":"2024-02-14T15:00:49.277025+0000","last_active":"2024-02-14T15:01:44.596985+0000","last_peered":"2024-02-14T15:01:44.596985+0000","last_clean":"2024-02-14T15:01:44.596985+0000","last_became_active":"2024-02-14T15:00:49.276694+0000","last_became_peered":"2024-02-14T15:00:49.276694+0000","last_unstale":"2024-02-14T15:01:44.596985+0000","last_undegraded":"2024-02-14T15:01:44.596985+0000","last_fullsized":"2024-02-14T15:01:44.596985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:43:07.249407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.607053+0000","last_change":"2024-02-14T15:01:44.607053+0000","last_active":"2024-02-14T15:01:44.607053+0000","last_peered":"2024-02-14T15:01:44.607053+0000","last_clean":"2024-02-14T15:01:44.607053+0000","last_became_active":"2024-02-14T15:01:44.606596+0000","last_became_peered":"2024-02-14T15:01:44.606596+0000","last_unstale":"2024-02-14T15:01:44.607053+0000","last_undegraded":"2024-02-14T15:01:44.607053+0000","last_fullsized":"2024-02-14T15:01:44.607053+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:35.949953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602188+0000","last_change":"2024-02-14T15:00:48.272866+0000","last_active":"2024-02-14T15:01:44.602188+0000","last_peered":"2024-02-14T15:01:44.602188+0000","last_clean":"2024-02-14T15:01:44.602188+0000","last_became_active":"2024-02-14T15:00:48.272520+0000","last_became_peered":"2024-02-14T15:00:48.272520+0000","last_unstale":"2024-02-14T15:01:44.602188+0000","last_undegraded":"2024-02-14T15:01:44.602188+0000","last_fullsized":"2024-02-14T15:01:44.602188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:39.434577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600429+0000","last_change":"2024-02-14T15:00:49.280994+0000","last_active":"2024-02-14T15:01:44.600429+0000","last_peered":"2024-02-14T15:01:44.600429+0000","last_clean":"2024-02-14T15:01:44.600429+0000","last_became_active":"2024-02-14T15:00:49.280205+0000","last_became_peered":"2024-02-14T15:00:49.280205+0000","last_unstale":"2024-02-14T15:01:44.600429+0000","last_undegraded":"2024-02-14T15:01:44.600429+0000","last_fullsized":"2024-02-14T15:01:44.600429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:53:48.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599386+0000","last_change":"2024-02-14T15:00:53.582599+0000","last_active":"2024-02-14T15:01:44.599386+0000","last_peered":"2024-02-14T15:01:44.599386+0000","last_clean":"2024-02-14T15:01:44.599386+0000","last_became_active":"2024-02-14T15:00:53.582231+0000","last_became_peered":"2024-02-14T15:00:53.582231+0000","last_unstale":"2024-02-14T15:01:44.599386+0000","last_undegraded":"2024-02-14T15:01:44.599386+0000","last_fullsized":"2024-02-14T15:01:44.599386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:19.126264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":219,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:22.657760+0000","last_change":"2024-02-14T15:01:44.615393+0000","last_active":"2024-02-14T15:02:22.657760+0000","last_peered":"2024-02-14T15:02:22.657760+0000","last_clean":"2024-02-14T15:02:22.657760+0000","last_became_active":"2024-02-14T15:01:44.604294+0000","last_became_peered":"2024-02-14T15:01:44.604294+0000","last_unstale":"2024-02-14T15:02:22.657760+0000","last_undegraded":"2024-02-14T15:02:22.657760+0000","last_fullsized":"2024-02-14T15:02:22.657760+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:00.459860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602420+0000","last_change":"2024-02-14T15:00:49.270144+0000","last_active":"2024-02-14T15:01:44.602420+0000","last_peered":"2024-02-14T15:01:44.602420+0000","last_clean":"2024-02-14T15:01:44.602420+0000","last_became_active":"2024-02-14T15:00:49.269806+0000","last_became_peered":"2024-02-14T15:00:49.269806+0000","last_unstale":"2024-02-14T15:01:44.602420+0000","last_undegraded":"2024-02-14T15:01:44.602420+0000","last_fullsized":"2024-02-14T15:01:44.602420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:42:18.559226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603924+0000","last_change":"2024-02-14T15:00:53.582755+0000","last_active":"2024-02-14T15:01:44.603924+0000","last_peered":"2024-02-14T15:01:44.603924+0000","last_clean":"2024-02-14T15:01:44.603924+0000","last_became_active":"2024-02-14T15:00:53.582375+0000","last_became_peered":"2024-02-14T15:00:53.582375+0000","last_unstale":"2024-02-14T15:01:44.603924+0000","last_undegraded":"2024-02-14T15:01:44.603924+0000","last_fullsized":"2024-02-14T15:01:44.603924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:11:49.115136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597952+0000","last_change":"2024-02-14T15:00:48.268982+0000","last_active":"2024-02-14T15:01:44.597952+0000","last_peered":"2024-02-14T15:01:44.597952+0000","last_clean":"2024-02-14T15:01:44.597952+0000","last_became_active":"2024-02-14T15:00:48.267137+0000","last_became_peered":"2024-02-14T15:00:48.267137+0000","last_unstale":"2024-02-14T15:01:44.597952+0000","last_undegraded":"2024-02-14T15:01:44.597952+0000","last_fullsized":"2024-02-14T15:01:44.597952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:44.351561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605820+0000","last_change":"2024-02-14T15:00:49.267493+0000","last_active":"2024-02-14T15:01:44.605820+0000","last_peered":"2024-02-14T15:01:44.605820+0000","last_clean":"2024-02-14T15:01:44.605820+0000","last_became_active":"2024-02-14T15:00:49.267161+0000","last_became_peered":"2024-02-14T15:00:49.267161+0000","last_unstale":"2024-02-14T15:01:44.605820+0000","last_undegraded":"2024-02-14T15:01:44.605820+0000","last_fullsized":"2024-02-14T15:01:44.605820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:35:22.997999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600261+0000","last_change":"2024-02-14T15:00:53.591201+0000","last_active":"2024-02-14T15:01:44.600261+0000","last_peered":"2024-02-14T15:01:44.600261+0000","last_clean":"2024-02-14T15:01:44.600261+0000","last_became_active":"2024-02-14T15:00:53.590769+0000","last_became_peered":"2024-02-14T15:00:53.590769+0000","last_unstale":"2024-02-14T15:01:44.600261+0000","last_undegraded":"2024-02-14T15:01:44.600261+0000","last_fullsized":"2024-02-14T15:01:44.600261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.881271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601141+0000","last_change":"2024-02-14T15:00:48.282085+0000","last_active":"2024-02-14T15:01:44.601141+0000","last_peered":"2024-02-14T15:01:44.601141+0000","last_clean":"2024-02-14T15:01:44.601141+0000","last_became_active":"2024-02-14T15:00:48.281836+0000","last_became_peered":"2024-02-14T15:00:48.281836+0000","last_unstale":"2024-02-14T15:01:44.601141+0000","last_undegraded":"2024-02-14T15:01:44.601141+0000","last_fullsized":"2024-02-14T15:01:44.601141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:07:28.145074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604787+0000","last_change":"2024-02-14T15:00:49.280013+0000","last_active":"2024-02-14T15:01:44.604787+0000","last_peered":"2024-02-14T15:01:44.604787+0000","last_clean":"2024-02-14T15:01:44.604787+0000","last_became_active":"2024-02-14T15:00:49.279667+0000","last_became_peered":"2024-02-14T15:00:49.279667+0000","last_unstale":"2024-02-14T15:01:44.604787+0000","last_undegraded":"2024-02-14T15:01:44.604787+0000","last_fullsized":"2024-02-14T15:01:44.604787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:03:16.559295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599609+0000","last_change":"2024-02-14T15:00:53.580269+0000","last_active":"2024-02-14T15:01:44.599609+0000","last_peered":"2024-02-14T15:01:44.599609+0000","last_clean":"2024-02-14T15:01:44.599609+0000","last_became_active":"2024-02-14T15:00:53.579863+0000","last_became_peered":"2024-02-14T15:00:53.579863+0000","last_unstale":"2024-02-14T15:01:44.599609+0000","last_undegraded":"2024-02-14T15:01:44.599609+0000","last_fullsized":"2024-02-14T15:01:44.599609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:37:19.275654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:23.659763+0000","last_change":"2024-02-14T15:00:48.267310+0000","last_active":"2024-02-14T15:02:23.659763+0000","last_peered":"2024-02-14T15:02:23.659763+0000","last_clean":"2024-02-14T15:02:23.659763+0000","last_became_active":"2024-02-14T15:00:48.266864+0000","last_became_peered":"2024-02-14T15:00:48.266864+0000","last_unstale":"2024-02-14T15:02:23.659763+0000","last_undegraded":"2024-02-14T15:02:23.659763+0000","last_fullsized":"2024-02-14T15:02:23.659763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:07.703549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604511+0000","last_change":"2024-02-14T15:00:49.288568+0000","last_active":"2024-02-14T15:01:44.604511+0000","last_peered":"2024-02-14T15:01:44.604511+0000","last_clean":"2024-02-14T15:01:44.604511+0000","last_became_active":"2024-02-14T15:00:49.288222+0000","last_became_peered":"2024-02-14T15:00:49.288222+0000","last_unstale":"2024-02-14T15:01:44.604511+0000","last_undegraded":"2024-02-14T15:01:44.604511+0000","last_fullsized":"2024-02-14T15:01:44.604511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:38:38.909769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605643+0000","last_change":"2024-02-14T15:00:53.585673+0000","last_active":"2024-02-14T15:01:44.605643+0000","last_peered":"2024-02-14T15:01:44.605643+0000","last_clean":"2024-02-14T15:01:44.605643+0000","last_became_active":"2024-02-14T15:00:53.585238+0000","last_became_peered":"2024-02-14T15:00:53.585238+0000","last_unstale":"2024-02-14T15:01:44.605643+0000","last_undegraded":"2024-02-14T15:01:44.605643+0000","last_fullsized":"2024-02-14T15:01:44.605643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:48:11.845921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602092+0000","last_change":"2024-02-14T15:00:48.275833+0000","last_active":"2024-02-14T15:01:44.602092+0000","last_peered":"2024-02-14T15:01:44.602092+0000","last_clean":"2024-02-14T15:01:44.602092+0000","last_became_active":"2024-02-14T15:00:48.275499+0000","last_became_peered":"2024-02-14T15:00:48.275499+0000","last_unstale":"2024-02-14T15:01:44.602092+0000","last_undegraded":"2024-02-14T15:01:44.602092+0000","last_fullsized":"2024-02-14T15:01:44.602092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:35:10.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605394+0000","last_change":"2024-02-14T15:01:44.605394+0000","last_active":"2024-02-14T15:01:44.605394+0000","last_peered":"2024-02-14T15:01:44.605394+0000","last_clean":"2024-02-14T15:01:44.605394+0000","last_became_active":"2024-02-14T15:01:44.604874+0000","last_became_peered":"2024-02-14T15:01:44.604874+0000","last_unstale":"2024-02-14T15:01:44.605394+0000","last_undegraded":"2024-02-14T15:01:44.605394+0000","last_fullsized":"2024-02-14T15:01:44.605394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:41:47.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601458+0000","last_change":"2024-02-14T15:00:53.586406+0000","last_active":"2024-02-14T15:01:44.601458+0000","last_peered":"2024-02-14T15:01:44.601458+0000","last_clean":"2024-02-14T15:01:44.601458+0000","last_became_active":"2024-02-14T15:00:53.586034+0000","last_became_peered":"2024-02-14T15:00:53.586034+0000","last_unstale":"2024-02-14T15:01:44.601458+0000","last_undegraded":"2024-02-14T15:01:44.601458+0000","last_fullsized":"2024-02-14T15:01:44.601458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:46.886154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603426+0000","last_change":"2024-02-14T15:00:48.274305+0000","last_active":"2024-02-14T15:01:44.603426+0000","last_peered":"2024-02-14T15:01:44.603426+0000","last_clean":"2024-02-14T15:01:44.603426+0000","last_became_active":"2024-02-14T15:00:48.273966+0000","last_became_peered":"2024-02-14T15:00:48.273966+0000","last_unstale":"2024-02-14T15:01:44.603426+0000","last_undegraded":"2024-02-14T15:01:44.603426+0000","last_fullsized":"2024-02-14T15:01:44.603426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:49.198099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604582+0000","last_change":"2024-02-14T15:00:49.278473+0000","last_active":"2024-02-14T15:01:44.604582+0000","last_peered":"2024-02-14T15:01:44.604582+0000","last_clean":"2024-02-14T15:01:44.604582+0000","last_became_active":"2024-02-14T15:00:49.278145+0000","last_became_peered":"2024-02-14T15:00:49.278145+0000","last_unstale":"2024-02-14T15:01:44.604582+0000","last_undegraded":"2024-02-14T15:01:44.604582+0000","last_fullsized":"2024-02-14T15:01:44.604582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:49:11.718035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602337+0000","last_change":"2024-02-14T15:00:53.581164+0000","last_active":"2024-02-14T15:01:44.602337+0000","last_peered":"2024-02-14T15:01:44.602337+0000","last_clean":"2024-02-14T15:01:44.602337+0000","last_became_active":"2024-02-14T15:00:53.580796+0000","last_became_peered":"2024-02-14T15:00:53.580796+0000","last_unstale":"2024-02-14T15:01:44.602337+0000","last_undegraded":"2024-02-14T15:01:44.602337+0000","last_fullsized":"2024-02-14T15:01:44.602337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:37.245035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599775+0000","last_change":"2024-02-14T15:00:48.271427+0000","last_active":"2024-02-14T15:01:44.599775+0000","last_peered":"2024-02-14T15:01:44.599775+0000","last_clean":"2024-02-14T15:01:44.599775+0000","last_became_active":"2024-02-14T15:00:48.271115+0000","last_became_peered":"2024-02-14T15:00:48.271115+0000","last_unstale":"2024-02-14T15:01:44.599775+0000","last_undegraded":"2024-02-14T15:01:44.599775+0000","last_fullsized":"2024-02-14T15:01:44.599775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:56:01.008058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601171+0000","last_change":"2024-02-14T15:00:49.269964+0000","last_active":"2024-02-14T15:01:44.601171+0000","last_peered":"2024-02-14T15:01:44.601171+0000","last_clean":"2024-02-14T15:01:44.601171+0000","last_became_active":"2024-02-14T15:00:49.269628+0000","last_became_peered":"2024-02-14T15:00:49.269628+0000","last_unstale":"2024-02-14T15:01:44.601171+0000","last_undegraded":"2024-02-14T15:01:44.601171+0000","last_fullsized":"2024-02-14T15:01:44.601171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:04:26.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605769+0000","last_change":"2024-02-14T15:00:53.579843+0000","last_active":"2024-02-14T15:01:44.605769+0000","last_peered":"2024-02-14T15:01:44.605769+0000","last_clean":"2024-02-14T15:01:44.605769+0000","last_became_active":"2024-02-14T15:00:53.579457+0000","last_became_peered":"2024-02-14T15:00:53.579457+0000","last_unstale":"2024-02-14T15:01:44.605769+0000","last_undegraded":"2024-02-14T15:01:44.605769+0000","last_fullsized":"2024-02-14T15:01:44.605769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:34:49.438317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604529+0000","last_change":"2024-02-14T15:00:48.275016+0000","last_active":"2024-02-14T15:01:44.604529+0000","last_peered":"2024-02-14T15:01:44.604529+0000","last_clean":"2024-02-14T15:01:44.604529+0000","last_became_active":"2024-02-14T15:00:48.274636+0000","last_became_peered":"2024-02-14T15:00:48.274636+0000","last_unstale":"2024-02-14T15:01:44.604529+0000","last_undegraded":"2024-02-14T15:01:44.604529+0000","last_fullsized":"2024-02-14T15:01:44.604529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:12:12.961276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602904+0000","last_change":"2024-02-14T15:01:44.602904+0000","last_active":"2024-02-14T15:01:44.602904+0000","last_peered":"2024-02-14T15:01:44.602904+0000","last_clean":"2024-02-14T15:01:44.602904+0000","last_became_active":"2024-02-14T15:01:44.602480+0000","last_became_peered":"2024-02-14T15:01:44.602480+0000","last_unstale":"2024-02-14T15:01:44.602904+0000","last_undegraded":"2024-02-14T15:01:44.602904+0000","last_fullsized":"2024-02-14T15:01:44.602904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:27:33.451581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605376+0000","last_change":"2024-02-14T15:00:53.584068+0000","last_active":"2024-02-14T15:01:44.605376+0000","last_peered":"2024-02-14T15:01:44.605376+0000","last_clean":"2024-02-14T15:01:44.605376+0000","last_became_active":"2024-02-14T15:00:53.583263+0000","last_became_peered":"2024-02-14T15:00:53.583263+0000","last_unstale":"2024-02-14T15:01:44.605376+0000","last_undegraded":"2024-02-14T15:01:44.605376+0000","last_fullsized":"2024-02-14T15:01:44.605376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:15.726442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603928+0000","last_change":"2024-02-14T15:00:49.271265+0000","last_active":"2024-02-14T15:01:44.603928+0000","last_peered":"2024-02-14T15:01:44.603928+0000","last_clean":"2024-02-14T15:01:44.603928+0000","last_became_active":"2024-02-14T15:00:49.270931+0000","last_became_peered":"2024-02-14T15:00:49.270931+0000","last_unstale":"2024-02-14T15:01:44.603928+0000","last_undegraded":"2024-02-14T15:01:44.603928+0000","last_fullsized":"2024-02-14T15:01:44.603928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:15:40.570730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":100,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:23.659256+0000","last_change":"2024-02-14T15:00:48.286669+0000","last_active":"2024-02-14T15:02:23.659256+0000","last_peered":"2024-02-14T15:02:23.659256+0000","last_clean":"2024-02-14T15:02:23.659256+0000","last_became_active":"2024-02-14T15:00:48.286252+0000","last_became_peered":"2024-02-14T15:00:48.286252+0000","last_unstale":"2024-02-14T15:02:23.659256+0000","last_undegraded":"2024-02-14T15:02:23.659256+0000","last_fullsized":"2024-02-14T15:02:23.659256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:40.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604737+0000","last_change":"2024-02-14T15:00:53.583353+0000","last_active":"2024-02-14T15:01:44.604737+0000","last_peered":"2024-02-14T15:01:44.604737+0000","last_clean":"2024-02-14T15:01:44.604737+0000","last_became_active":"2024-02-14T15:00:53.582449+0000","last_became_peered":"2024-02-14T15:00:53.582449+0000","last_unstale":"2024-02-14T15:01:44.604737+0000","last_undegraded":"2024-02-14T15:01:44.604737+0000","last_fullsized":"2024-02-14T15:01:44.604737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:09:01.514603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602175+0000","last_change":"2024-02-14T15:00:49.285579+0000","last_active":"2024-02-14T15:01:44.602175+0000","last_peered":"2024-02-14T15:01:44.602175+0000","last_clean":"2024-02-14T15:01:44.602175+0000","last_became_active":"2024-02-14T15:00:49.283741+0000","last_became_peered":"2024-02-14T15:00:49.283741+0000","last_unstale":"2024-02-14T15:01:44.602175+0000","last_undegraded":"2024-02-14T15:01:44.602175+0000","last_fullsized":"2024-02-14T15:01:44.602175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:05.813218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598214+0000","last_change":"2024-02-14T15:00:48.264434+0000","last_active":"2024-02-14T15:01:44.598214+0000","last_peered":"2024-02-14T15:01:44.598214+0000","last_clean":"2024-02-14T15:01:44.598214+0000","last_became_active":"2024-02-14T15:00:48.264048+0000","last_became_peered":"2024-02-14T15:00:48.264048+0000","last_unstale":"2024-02-14T15:01:44.598214+0000","last_undegraded":"2024-02-14T15:01:44.598214+0000","last_fullsized":"2024-02-14T15:01:44.598214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:52:32.498128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604421+0000","last_change":"2024-02-14T14:59:36.539524+0000","last_active":"2024-02-14T15:01:44.604421+0000","last_peered":"2024-02-14T15:01:44.604421+0000","last_clean":"2024-02-14T15:01:44.604421+0000","last_became_active":"2024-02-14T14:59:36.538922+0000","last_became_peered":"2024-02-14T14:59:36.538922+0000","last_unstale":"2024-02-14T15:01:44.604421+0000","last_undegraded":"2024-02-14T15:01:44.604421+0000","last_fullsized":"2024-02-14T15:01:44.604421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_clean_scrub_stamp":"2024-02-14T14:56:09.567049+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:58.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603027+0000","last_change":"2024-02-14T15:00:53.585747+0000","last_active":"2024-02-14T15:01:44.603027+0000","last_peered":"2024-02-14T15:01:44.603027+0000","last_clean":"2024-02-14T15:01:44.603027+0000","last_became_active":"2024-02-14T15:00:53.583817+0000","last_became_peered":"2024-02-14T15:00:53.583817+0000","last_unstale":"2024-02-14T15:01:44.603027+0000","last_undegraded":"2024-02-14T15:01:44.603027+0000","last_fullsized":"2024-02-14T15:01:44.603027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:56:35.081078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605152+0000","last_change":"2024-02-14T15:00:49.272692+0000","last_active":"2024-02-14T15:01:44.605152+0000","last_peered":"2024-02-14T15:01:44.605152+0000","last_clean":"2024-02-14T15:01:44.605152+0000","last_became_active":"2024-02-14T15:00:49.272353+0000","last_became_peered":"2024-02-14T15:00:49.272353+0000","last_unstale":"2024-02-14T15:01:44.605152+0000","last_undegraded":"2024-02-14T15:01:44.605152+0000","last_fullsized":"2024-02-14T15:01:44.605152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:47.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604299+0000","last_change":"2024-02-14T15:00:48.272470+0000","last_active":"2024-02-14T15:01:44.604299+0000","last_peered":"2024-02-14T15:01:44.604299+0000","last_clean":"2024-02-14T15:01:44.604299+0000","last_became_active":"2024-02-14T15:00:48.272126+0000","last_became_peered":"2024-02-14T15:00:48.272126+0000","last_unstale":"2024-02-14T15:01:44.604299+0000","last_undegraded":"2024-02-14T15:01:44.604299+0000","last_fullsized":"2024-02-14T15:01:44.604299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:21:02.601411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600753+0000","last_change":"2024-02-14T15:00:53.585950+0000","last_active":"2024-02-14T15:01:44.600753+0000","last_peered":"2024-02-14T15:01:44.600753+0000","last_clean":"2024-02-14T15:01:44.600753+0000","last_became_active":"2024-02-14T15:00:53.585547+0000","last_became_peered":"2024-02-14T15:00:53.585547+0000","last_unstale":"2024-02-14T15:01:44.600753+0000","last_undegraded":"2024-02-14T15:01:44.600753+0000","last_fullsized":"2024-02-14T15:01:44.600753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:22.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602538+0000","last_change":"2024-02-14T15:00:48.274650+0000","last_active":"2024-02-14T15:01:44.602538+0000","last_peered":"2024-02-14T15:01:44.602538+0000","last_clean":"2024-02-14T15:01:44.602538+0000","last_became_active":"2024-02-14T15:00:48.274085+0000","last_became_peered":"2024-02-14T15:00:48.274085+0000","last_unstale":"2024-02-14T15:01:44.602538+0000","last_undegraded":"2024-02-14T15:01:44.602538+0000","last_fullsized":"2024-02-14T15:01:44.602538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:53.307596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605939+0000","last_change":"2024-02-14T15:00:49.268419+0000","last_active":"2024-02-14T15:01:44.605939+0000","last_peered":"2024-02-14T15:01:44.605939+0000","last_clean":"2024-02-14T15:01:44.605939+0000","last_became_active":"2024-02-14T15:00:49.268120+0000","last_became_peered":"2024-02-14T15:00:49.268120+0000","last_unstale":"2024-02-14T15:01:44.605939+0000","last_undegraded":"2024-02-14T15:01:44.605939+0000","last_fullsized":"2024-02-14T15:01:44.605939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:10:36.319337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601823+0000","last_change":"2024-02-14T15:00:53.586151+0000","last_active":"2024-02-14T15:01:44.601823+0000","last_peered":"2024-02-14T15:01:44.601823+0000","last_clean":"2024-02-14T15:01:44.601823+0000","last_became_active":"2024-02-14T15:00:53.585727+0000","last_became_peered":"2024-02-14T15:00:53.585727+0000","last_unstale":"2024-02-14T15:01:44.601823+0000","last_undegraded":"2024-02-14T15:01:44.601823+0000","last_fullsized":"2024-02-14T15:01:44.601823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:40:03.741063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.615028+0000","last_change":"2024-02-14T15:01:44.615028+0000","last_active":"2024-02-14T15:01:44.615028+0000","last_peered":"2024-02-14T15:01:44.615028+0000","last_clean":"2024-02-14T15:01:44.615028+0000","last_became_active":"2024-02-14T15:01:44.606281+0000","last_became_peered":"2024-02-14T15:01:44.606281+0000","last_unstale":"2024-02-14T15:01:44.615028+0000","last_undegraded":"2024-02-14T15:01:44.615028+0000","last_fullsized":"2024-02-14T15:01:44.615028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:37.706659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602519+0000","last_change":"2024-02-14T15:00:49.285726+0000","last_active":"2024-02-14T15:01:44.602519+0000","last_peered":"2024-02-14T15:01:44.602519+0000","last_clean":"2024-02-14T15:01:44.602519+0000","last_became_active":"2024-02-14T15:00:49.283728+0000","last_became_peered":"2024-02-14T15:00:49.283728+0000","last_unstale":"2024-02-14T15:01:44.602519+0000","last_undegraded":"2024-02-14T15:01:44.602519+0000","last_fullsized":"2024-02-14T15:01:44.602519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.764048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597397+0000","last_change":"2024-02-14T15:00:53.583644+0000","last_active":"2024-02-14T15:01:44.597397+0000","last_peered":"2024-02-14T15:01:44.597397+0000","last_clean":"2024-02-14T15:01:44.597397+0000","last_became_active":"2024-02-14T15:00:53.582801+0000","last_became_peered":"2024-02-14T15:00:53.582801+0000","last_unstale":"2024-02-14T15:01:44.597397+0000","last_undegraded":"2024-02-14T15:01:44.597397+0000","last_fullsized":"2024-02-14T15:01:44.597397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:41:40.622091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599344+0000","last_change":"2024-02-14T15:00:48.270231+0000","last_active":"2024-02-14T15:01:44.599344+0000","last_peered":"2024-02-14T15:01:44.599344+0000","last_clean":"2024-02-14T15:01:44.599344+0000","last_became_active":"2024-02-14T15:00:48.269880+0000","last_became_peered":"2024-02-14T15:00:48.269880+0000","last_unstale":"2024-02-14T15:01:44.599344+0000","last_undegraded":"2024-02-14T15:01:44.599344+0000","last_fullsized":"2024-02-14T15:01:44.599344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:08:47.376417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597044+0000","last_change":"2024-02-14T15:00:49.277227+0000","last_active":"2024-02-14T15:01:44.597044+0000","last_peered":"2024-02-14T15:01:44.597044+0000","last_clean":"2024-02-14T15:01:44.597044+0000","last_became_active":"2024-02-14T15:00:49.276904+0000","last_became_peered":"2024-02-14T15:00:49.276904+0000","last_unstale":"2024-02-14T15:01:44.597044+0000","last_undegraded":"2024-02-14T15:01:44.597044+0000","last_fullsized":"2024-02-14T15:01:44.597044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:01.091327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598683+0000","last_change":"2024-02-14T15:00:53.587855+0000","last_active":"2024-02-14T15:01:44.598683+0000","last_peered":"2024-02-14T15:01:44.598683+0000","last_clean":"2024-02-14T15:01:44.598683+0000","last_became_active":"2024-02-14T15:00:53.586313+0000","last_became_peered":"2024-02-14T15:00:53.586313+0000","last_unstale":"2024-02-14T15:01:44.598683+0000","last_undegraded":"2024-02-14T15:01:44.598683+0000","last_fullsized":"2024-02-14T15:01:44.598683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:36:41.279582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600029+0000","last_change":"2024-02-14T15:00:48.267385+0000","last_active":"2024-02-14T15:01:44.600029+0000","last_peered":"2024-02-14T15:01:44.600029+0000","last_clean":"2024-02-14T15:01:44.600029+0000","last_became_active":"2024-02-14T15:00:48.267036+0000","last_became_peered":"2024-02-14T15:00:48.267036+0000","last_unstale":"2024-02-14T15:01:44.600029+0000","last_undegraded":"2024-02-14T15:01:44.600029+0000","last_fullsized":"2024-02-14T15:01:44.600029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:29:24.840030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601999+0000","last_change":"2024-02-14T15:00:49.272233+0000","last_active":"2024-02-14T15:01:44.601999+0000","last_peered":"2024-02-14T15:01:44.601999+0000","last_clean":"2024-02-14T15:01:44.601999+0000","last_became_active":"2024-02-14T15:00:49.271813+0000","last_became_peered":"2024-02-14T15:00:49.271813+0000","last_unstale":"2024-02-14T15:01:44.601999+0000","last_undegraded":"2024-02-14T15:01:44.601999+0000","last_fullsized":"2024-02-14T15:01:44.601999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:30:32.521443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599873+0000","last_change":"2024-02-14T15:00:53.583385+0000","last_active":"2024-02-14T15:01:44.599873+0000","last_peered":"2024-02-14T15:01:44.599873+0000","last_clean":"2024-02-14T15:01:44.599873+0000","last_became_active":"2024-02-14T15:00:53.583023+0000","last_became_peered":"2024-02-14T15:00:53.583023+0000","last_unstale":"2024-02-14T15:01:44.599873+0000","last_undegraded":"2024-02-14T15:01:44.599873+0000","last_fullsized":"2024-02-14T15:01:44.599873+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:40:53.500939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601855+0000","last_change":"2024-02-14T15:00:48.261376+0000","last_active":"2024-02-14T15:01:44.601855+0000","last_peered":"2024-02-14T15:01:44.601855+0000","last_clean":"2024-02-14T15:01:44.601855+0000","last_became_active":"2024-02-14T15:00:48.260461+0000","last_became_peered":"2024-02-14T15:00:48.260461+0000","last_unstale":"2024-02-14T15:01:44.601855+0000","last_undegraded":"2024-02-14T15:01:44.601855+0000","last_fullsized":"2024-02-14T15:01:44.601855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:35.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599874+0000","last_change":"2024-02-14T15:00:49.273117+0000","last_active":"2024-02-14T15:01:44.599874+0000","last_peered":"2024-02-14T15:01:44.599874+0000","last_clean":"2024-02-14T15:01:44.599874+0000","last_became_active":"2024-02-14T15:00:49.272810+0000","last_became_peered":"2024-02-14T15:00:49.272810+0000","last_unstale":"2024-02-14T15:01:44.599874+0000","last_undegraded":"2024-02-14T15:01:44.599874+0000","last_fullsized":"2024-02-14T15:01:44.599874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:37:00.992212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599218+0000","last_change":"2024-02-14T15:00:53.583972+0000","last_active":"2024-02-14T15:01:44.599218+0000","last_peered":"2024-02-14T15:01:44.599218+0000","last_clean":"2024-02-14T15:01:44.599218+0000","last_became_active":"2024-02-14T15:00:53.583563+0000","last_became_peered":"2024-02-14T15:00:53.583563+0000","last_unstale":"2024-02-14T15:01:44.599218+0000","last_undegraded":"2024-02-14T15:01:44.599218+0000","last_fullsized":"2024-02-14T15:01:44.599218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:08:55.811082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597557+0000","last_change":"2024-02-14T15:00:48.270363+0000","last_active":"2024-02-14T15:01:44.597557+0000","last_peered":"2024-02-14T15:01:44.597557+0000","last_clean":"2024-02-14T15:01:44.597557+0000","last_became_active":"2024-02-14T15:00:48.270030+0000","last_became_peered":"2024-02-14T15:00:48.270030+0000","last_unstale":"2024-02-14T15:01:44.597557+0000","last_undegraded":"2024-02-14T15:01:44.597557+0000","last_fullsized":"2024-02-14T15:01:44.597557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:28.885451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602276+0000","last_change":"2024-02-14T15:00:49.285647+0000","last_active":"2024-02-14T15:01:44.602276+0000","last_peered":"2024-02-14T15:01:44.602276+0000","last_clean":"2024-02-14T15:01:44.602276+0000","last_became_active":"2024-02-14T15:00:49.285267+0000","last_became_peered":"2024-02-14T15:00:49.285267+0000","last_unstale":"2024-02-14T15:01:44.602276+0000","last_undegraded":"2024-02-14T15:01:44.602276+0000","last_fullsized":"2024-02-14T15:01:44.602276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:03.143177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599006+0000","last_change":"2024-02-14T15:00:53.591029+0000","last_active":"2024-02-14T15:01:44.599006+0000","last_peered":"2024-02-14T15:01:44.599006+0000","last_clean":"2024-02-14T15:01:44.599006+0000","last_became_active":"2024-02-14T15:00:53.590570+0000","last_became_peered":"2024-02-14T15:00:53.590570+0000","last_unstale":"2024-02-14T15:01:44.599006+0000","last_undegraded":"2024-02-14T15:01:44.599006+0000","last_fullsized":"2024-02-14T15:01:44.599006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:41.809227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.611339+0000","last_change":"2024-02-14T15:01:44.611339+0000","last_active":"2024-02-14T15:01:44.611339+0000","last_peered":"2024-02-14T15:01:44.611339+0000","last_clean":"2024-02-14T15:01:44.611339+0000","last_became_active":"2024-02-14T15:01:44.605631+0000","last_became_peered":"2024-02-14T15:01:44.605631+0000","last_unstale":"2024-02-14T15:01:44.611339+0000","last_undegraded":"2024-02-14T15:01:44.611339+0000","last_fullsized":"2024-02-14T15:01:44.611339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:24:19.853345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604654+0000","last_change":"2024-02-14T15:00:49.276346+0000","last_active":"2024-02-14T15:01:44.604654+0000","last_peered":"2024-02-14T15:01:44.604654+0000","last_clean":"2024-02-14T15:01:44.604654+0000","last_became_active":"2024-02-14T15:00:49.276000+0000","last_became_peered":"2024-02-14T15:00:49.276000+0000","last_unstale":"2024-02-14T15:01:44.604654+0000","last_undegraded":"2024-02-14T15:01:44.604654+0000","last_fullsized":"2024-02-14T15:01:44.604654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:14:03.811981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599368+0000","last_change":"2024-02-14T15:00:53.587021+0000","last_active":"2024-02-14T15:01:44.599368+0000","last_peered":"2024-02-14T15:01:44.599368+0000","last_clean":"2024-02-14T15:01:44.599368+0000","last_became_active":"2024-02-14T15:00:53.586492+0000","last_became_peered":"2024-02-14T15:00:53.586492+0000","last_unstale":"2024-02-14T15:01:44.599368+0000","last_undegraded":"2024-02-14T15:01:44.599368+0000","last_fullsized":"2024-02-14T15:01:44.599368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:51:02.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.612410+0000","last_change":"2024-02-14T15:01:44.611916+0000","last_active":"2024-02-14T15:01:44.612410+0000","last_peered":"2024-02-14T15:01:44.612410+0000","last_clean":"2024-02-14T15:01:44.612410+0000","last_became_active":"2024-02-14T15:01:44.605791+0000","last_became_peered":"2024-02-14T15:01:44.605791+0000","last_unstale":"2024-02-14T15:01:44.612410+0000","last_undegraded":"2024-02-14T15:01:44.612410+0000","last_fullsized":"2024-02-14T15:01:44.612410+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:50:54.066205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605087+0000","last_change":"2024-02-14T15:00:49.276110+0000","last_active":"2024-02-14T15:01:44.605087+0000","last_peered":"2024-02-14T15:01:44.605087+0000","last_clean":"2024-02-14T15:01:44.605087+0000","last_became_active":"2024-02-14T15:00:49.275729+0000","last_became_peered":"2024-02-14T15:00:49.275729+0000","last_unstale":"2024-02-14T15:01:44.605087+0000","last_undegraded":"2024-02-14T15:01:44.605087+0000","last_fullsized":"2024-02-14T15:01:44.605087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:07:29.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596658+0000","last_change":"2024-02-14T15:00:53.582061+0000","last_active":"2024-02-14T15:01:44.596658+0000","last_peered":"2024-02-14T15:01:44.596658+0000","last_clean":"2024-02-14T15:01:44.596658+0000","last_became_active":"2024-02-14T15:00:53.581659+0000","last_became_peered":"2024-02-14T15:00:53.581659+0000","last_unstale":"2024-02-14T15:01:44.596658+0000","last_undegraded":"2024-02-14T15:01:44.596658+0000","last_fullsized":"2024-02-14T15:01:44.596658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:13.814496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603397+0000","last_change":"2024-02-14T15:00:48.283300+0000","last_active":"2024-02-14T15:01:44.603397+0000","last_peered":"2024-02-14T15:01:44.603397+0000","last_clean":"2024-02-14T15:01:44.603397+0000","last_became_active":"2024-02-14T15:00:48.282939+0000","last_became_peered":"2024-02-14T15:00:48.282939+0000","last_unstale":"2024-02-14T15:01:44.603397+0000","last_undegraded":"2024-02-14T15:01:44.603397+0000","last_fullsized":"2024-02-14T15:01:44.603397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:56:52.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604622+0000","last_change":"2024-02-14T15:00:49.273435+0000","last_active":"2024-02-14T15:01:44.604622+0000","last_peered":"2024-02-14T15:01:44.604622+0000","last_clean":"2024-02-14T15:01:44.604622+0000","last_became_active":"2024-02-14T15:00:49.272980+0000","last_became_peered":"2024-02-14T15:00:49.272980+0000","last_unstale":"2024-02-14T15:01:44.604622+0000","last_undegraded":"2024-02-14T15:01:44.604622+0000","last_fullsized":"2024-02-14T15:01:44.604622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:19:05.294943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600573+0000","last_change":"2024-02-14T15:00:53.583509+0000","last_active":"2024-02-14T15:01:44.600573+0000","last_peered":"2024-02-14T15:01:44.600573+0000","last_clean":"2024-02-14T15:01:44.600573+0000","last_became_active":"2024-02-14T15:00:53.583148+0000","last_became_peered":"2024-02-14T15:00:53.583148+0000","last_unstale":"2024-02-14T15:01:44.600573+0000","last_undegraded":"2024-02-14T15:01:44.600573+0000","last_fullsized":"2024-02-14T15:01:44.600573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:06:47.059643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605520+0000","last_change":"2024-02-14T15:00:48.286707+0000","last_active":"2024-02-14T15:01:44.605520+0000","last_peered":"2024-02-14T15:01:44.605520+0000","last_clean":"2024-02-14T15:01:44.605520+0000","last_became_active":"2024-02-14T15:00:48.286355+0000","last_became_peered":"2024-02-14T15:00:48.286355+0000","last_unstale":"2024-02-14T15:01:44.605520+0000","last_undegraded":"2024-02-14T15:01:44.605520+0000","last_fullsized":"2024-02-14T15:01:44.605520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:48:31.024464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605460+0000","last_change":"2024-02-14T15:00:49.271598+0000","last_active":"2024-02-14T15:01:44.605460+0000","last_peered":"2024-02-14T15:01:44.605460+0000","last_clean":"2024-02-14T15:01:44.605460+0000","last_became_active":"2024-02-14T15:00:49.270572+0000","last_became_peered":"2024-02-14T15:00:49.270572+0000","last_unstale":"2024-02-14T15:01:44.605460+0000","last_undegraded":"2024-02-14T15:01:44.605460+0000","last_fullsized":"2024-02-14T15:01:44.605460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:12.666557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599923+0000","last_change":"2024-02-14T15:00:49.276697+0000","last_active":"2024-02-14T15:01:44.599923+0000","last_peered":"2024-02-14T15:01:44.599923+0000","last_clean":"2024-02-14T15:01:44.599923+0000","last_became_active":"2024-02-14T15:00:49.276279+0000","last_became_peered":"2024-02-14T15:00:49.276279+0000","last_unstale":"2024-02-14T15:01:44.599923+0000","last_undegraded":"2024-02-14T15:01:44.599923+0000","last_fullsized":"2024-02-14T15:01:44.599923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:11:31.824974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.607605+0000","last_change":"2024-02-14T15:01:44.607605+0000","last_active":"2024-02-14T15:01:44.607605+0000","last_peered":"2024-02-14T15:01:44.607605+0000","last_clean":"2024-02-14T15:01:44.607605+0000","last_became_active":"2024-02-14T15:01:44.607171+0000","last_became_peered":"2024-02-14T15:01:44.607171+0000","last_unstale":"2024-02-14T15:01:44.607605+0000","last_undegraded":"2024-02-14T15:01:44.607605+0000","last_fullsized":"2024-02-14T15:01:44.607605+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:30:20.503814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597556+0000","last_change":"2024-02-14T15:00:49.274219+0000","last_active":"2024-02-14T15:01:44.597556+0000","last_peered":"2024-02-14T15:01:44.597556+0000","last_clean":"2024-02-14T15:01:44.597556+0000","last_became_active":"2024-02-14T15:00:49.273833+0000","last_became_peered":"2024-02-14T15:00:49.273833+0000","last_unstale":"2024-02-14T15:01:44.597556+0000","last_undegraded":"2024-02-14T15:01:44.597556+0000","last_fullsized":"2024-02-14T15:01:44.597556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:06:02.720273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596758+0000","last_change":"2024-02-14T15:00:49.273576+0000","last_active":"2024-02-14T15:01:44.596758+0000","last_peered":"2024-02-14T15:01:44.596758+0000","last_clean":"2024-02-14T15:01:44.596758+0000","last_became_active":"2024-02-14T15:00:49.272920+0000","last_became_peered":"2024-02-14T15:00:49.272920+0000","last_unstale":"2024-02-14T15:01:44.596758+0000","last_undegraded":"2024-02-14T15:01:44.596758+0000","last_fullsized":"2024-02-14T15:01:44.596758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:22:03.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605886+0000","last_change":"2024-02-14T15:00:49.274549+0000","last_active":"2024-02-14T15:01:44.605886+0000","last_peered":"2024-02-14T15:01:44.605886+0000","last_clean":"2024-02-14T15:01:44.605886+0000","last_became_active":"2024-02-14T15:00:49.274097+0000","last_became_peered":"2024-02-14T15:00:49.274097+0000","last_unstale":"2024-02-14T15:01:44.605886+0000","last_undegraded":"2024-02-14T15:01:44.605886+0000","last_fullsized":"2024-02-14T15:01:44.605886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:47:44.017607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600676+0000","last_change":"2024-02-14T15:00:49.275699+0000","last_active":"2024-02-14T15:01:44.600676+0000","last_peered":"2024-02-14T15:01:44.600676+0000","last_clean":"2024-02-14T15:01:44.600676+0000","last_became_active":"2024-02-14T15:00:49.275361+0000","last_became_peered":"2024-02-14T15:00:49.275361+0000","last_unstale":"2024-02-14T15:01:44.600676+0000","last_undegraded":"2024-02-14T15:01:44.600676+0000","last_fullsized":"2024-02-14T15:01:44.600676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:34:02.638758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598315+0000","last_change":"2024-02-14T15:00:49.275024+0000","last_active":"2024-02-14T15:01:44.598315+0000","last_peered":"2024-02-14T15:01:44.598315+0000","last_clean":"2024-02-14T15:01:44.598315+0000","last_became_active":"2024-02-14T15:00:49.274710+0000","last_became_peered":"2024-02-14T15:00:49.274710+0000","last_unstale":"2024-02-14T15:01:44.598315+0000","last_undegraded":"2024-02-14T15:01:44.598315+0000","last_fullsized":"2024-02-14T15:01:44.598315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:42.471112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602642+0000","last_change":"2024-02-14T15:00:49.282638+0000","last_active":"2024-02-14T15:01:44.602642+0000","last_peered":"2024-02-14T15:01:44.602642+0000","last_clean":"2024-02-14T15:01:44.602642+0000","last_became_active":"2024-02-14T15:00:49.280223+0000","last_became_peered":"2024-02-14T15:00:49.280223+0000","last_unstale":"2024-02-14T15:01:44.602642+0000","last_undegraded":"2024-02-14T15:01:44.602642+0000","last_fullsized":"2024-02-14T15:01:44.602642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:40.250242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600914+0000","last_change":"2024-02-14T15:00:49.273025+0000","last_active":"2024-02-14T15:01:44.600914+0000","last_peered":"2024-02-14T15:01:44.600914+0000","last_clean":"2024-02-14T15:01:44.600914+0000","last_became_active":"2024-02-14T15:00:49.272700+0000","last_became_peered":"2024-02-14T15:00:49.272700+0000","last_unstale":"2024-02-14T15:01:44.600914+0000","last_undegraded":"2024-02-14T15:01:44.600914+0000","last_fullsized":"2024-02-14T15:01:44.600914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:08.677428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.606377+0000","last_change":"2024-02-14T15:01:44.605957+0000","last_active":"2024-02-14T15:01:44.606377+0000","last_peered":"2024-02-14T15:01:44.606377+0000","last_clean":"2024-02-14T15:01:44.606377+0000","last_became_active":"2024-02-14T15:01:44.605398+0000","last_became_peered":"2024-02-14T15:01:44.605398+0000","last_unstale":"2024-02-14T15:01:44.606377+0000","last_undegraded":"2024-02-14T15:01:44.606377+0000","last_fullsized":"2024-02-14T15:01:44.606377+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:50:43.667496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601312+0000","last_change":"2024-02-14T15:00:49.282178+0000","last_active":"2024-02-14T15:01:44.601312+0000","last_peered":"2024-02-14T15:01:44.601312+0000","last_clean":"2024-02-14T15:01:44.601312+0000","last_became_active":"2024-02-14T15:00:49.281804+0000","last_became_peered":"2024-02-14T15:00:49.281804+0000","last_unstale":"2024-02-14T15:01:44.601312+0000","last_undegraded":"2024-02-14T15:01:44.601312+0000","last_fullsized":"2024-02-14T15:01:44.601312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:58.672808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605635+0000","last_change":"2024-02-14T15:00:49.269882+0000","last_active":"2024-02-14T15:01:44.605635+0000","last_peered":"2024-02-14T15:01:44.605635+0000","last_clean":"2024-02-14T15:01:44.605635+0000","last_became_active":"2024-02-14T15:00:49.269489+0000","last_became_peered":"2024-02-14T15:00:49.269489+0000","last_unstale":"2024-02-14T15:01:44.605635+0000","last_undegraded":"2024-02-14T15:01:44.605635+0000","last_fullsized":"2024-02-14T15:01:44.605635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:57:40.465044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600399+0000","last_change":"2024-02-14T15:00:49.277688+0000","last_active":"2024-02-14T15:01:44.600399+0000","last_peered":"2024-02-14T15:01:44.600399+0000","last_clean":"2024-02-14T15:01:44.600399+0000","last_became_active":"2024-02-14T15:00:49.277348+0000","last_became_peered":"2024-02-14T15:00:49.277348+0000","last_unstale":"2024-02-14T15:01:44.600399+0000","last_undegraded":"2024-02-14T15:01:44.600399+0000","last_fullsized":"2024-02-14T15:01:44.600399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:33.946319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67357,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":436,"num_read_kb":287,"num_write":161,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":63938,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":499976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.436},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.61499999999999999,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.57299999999999995}]},{"osd":1,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.56699999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91200000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.53700000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83699999999999997,"5min":1.155,"15min":1.155},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80300000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.504,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91300000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59899999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78600000000000003,"15min":0.79100000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.745},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":1.121,"15min":1.121},"last":0.79300000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.624}]},{"osd":9,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75800000000000001,"15min":0.75600000000000001},"min":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97999999999999998,"5min":1.002,"15min":1.002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77600000000000002,"15min":0.77500000000000002},"min":{"1min":0.57699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.69299999999999995}]},{"osd":10,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.92200000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":0.81499999999999995},"min":{"1min":0.65600000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.72899999999999998}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1150991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82299999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69199999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82699999999999996,"5min":0.876,"15min":0.876},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.69199999999999995,"15min":0.69999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81699999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72099999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":1.054,"15min":1.054},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.629,"15min":0.65200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.61299999999999999,"5min":1.145,"15min":1.145},"last":0.48499999999999999}]},{"osd":5,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88400000000000001,"5min":1.032,"15min":1.032},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.50600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95799999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.626,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.90300000000000002,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":0.85299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95299999999999996,"5min":1.129,"15min":1.129},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.83999999999999997,"15min":0.85899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.872,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74199999999999999}]},{"osd":9,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.86599999999999999,"15min":0.874},"min":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.169},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.85799999999999998,"15min":0.88500000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.91200000000000003,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.81599999999999995}]},{"osd":11,"last update":"Wed Feb 14 15:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86899999999999999,"15min":0.875},"min":{"1min":0.63900000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.032,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.90600000000000003},"min":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0109999999999999,"5min":1.141,"15min":1.141},"last":0.91000000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":504308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44900000000000001,"15min":0.46200000000000002},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":1.22,"15min":1.22},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.48299999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.54200000000000004,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.374}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0629999999999999,"5min":1.141,"15min":1.141},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.69999999999999996,"15min":0.73699999999999999},"min":{"1min":0.45600000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97799999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.90600000000000003,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4370000000000001,"5min":19.152000000000001,"15min":19.152000000000001},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.31},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.94199999999999995,"15min":0.86299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":18.734999999999999,"15min":18.734999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.93999999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5069999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.45800000000000002}]},{"osd":8,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.94799999999999995,"15min":0.85999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.165,"5min":19.135999999999999,"15min":19.135999999999999},"last":0.622},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0049999999999999,"5min":1.23,"15min":1.23},"last":0.72099999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.79500000000000004,"15min":0.80400000000000005},"min":{"1min":0.58299999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.436},{"interface":"front","average":{"1min":0.748,"5min":0.82799999999999996,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0369999999999999,"5min":1.264,"15min":1.264},"last":0.57299999999999995}]},{"osd":11,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.53500000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":500132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":0.68500000000000005,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":13.997,"5min":13.997,"15min":13.997},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83499999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66100000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.435},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.32500000000000001}]},{"osd":3,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35799999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.48799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.628,"15min":0.622},"min":{"1min":0.436,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.373},{"interface":"front","average":{"1min":0.628,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.40799999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45500000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78200000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":1.046,"15min":1.212},"min":{"1min":0.55900000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":13.618,"15min":13.618},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.59499999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.58499999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70699999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.59599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.68400000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61699999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.59699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.70899999999999996,"15min":0.69099999999999995},"min":{"1min":0.60499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67600000000000005}]},{"osd":11,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.749}]}]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1096041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71599999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.504},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":0.877,"15min":0.877},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.63}]},{"osd":3,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.68899999999999995,"15min":0.65000000000000002},"min":{"1min":0.52900000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83199999999999996,"5min":0.872,"15min":0.872},"last":0.83199999999999996}]},{"osd":5,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.58199999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91000000000000003,"5min":0.997,"15min":0.997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85499999999999998,"5min":1.095,"15min":1.095},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.78400000000000003,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79400000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.59299999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97799999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80300000000000005,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.68300000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.052,"15min":1.052},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.67300000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.0339999999999998,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":0.76900000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.05,"5min":1.179,"15min":1.179},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82599999999999996,"15min":0.82699999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93600000000000005}]},{"osd":11,"last update":"Wed Feb 14 15:01:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.877}]}]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":495388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.37},"min":{"1min":0.32500000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.39300000000000002},"min":{"1min":0.34200000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.44900000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.464,"5min":0.747,"15min":0.60999999999999999},"min":{"1min":0.45700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":19.181000000000001,"5min":19.181000000000001,"15min":19.181000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61099999999999999,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.442,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.501}]},{"osd":3,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45800000000000002,"15min":0.44400000000000001},"min":{"1min":0.372,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57899999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80200000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":0.76700000000000002,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":19.167000000000002,"5min":19.167000000000002,"15min":19.167000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71399999999999997}]},{"osd":5,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.875,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64900000000000002,"15min":0.63200000000000001},"min":{"1min":0.47499999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60399999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.56499999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.99199999999999999,"5min":1.107,"15min":1.107},"last":0.76100000000000001}]},{"osd":8,"last update":"Wed Feb 14 15:01:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.55300000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84699999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92200000000000004,"5min":1.266,"15min":1.266},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68899999999999995,"15min":0.72499999999999998},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.92300000000000004,"5min":0.996,"15min":0.996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.73899999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:01:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.008,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94999999999999996,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.85199999999999998}]},{"osd":11,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":559236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.32800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.56599999999999995,"15min":0.51300000000000001},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.309},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48599999999999999}]},{"osd":2,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.53600000000000003},"min":{"1min":0.47099999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.80800000000000005,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":18.332000000000001,"15min":18.332000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.871,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.53400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.76400000000000001}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.746}]},{"osd":5,"last update":"Wed Feb 14 15:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.63600000000000001,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.66100000000000003,"15min":0.60499999999999998},"min":{"1min":0.56399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0089999999999999,"5min":1.016,"15min":1.016},"last":0.79000000000000004}]},{"osd":6,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59999999999999998,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.69399999999999995,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.89800000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65500000000000003,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0209999999999999,"5min":1.077,"15min":1.077},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71499999999999997}]},{"osd":8,"last update":"Wed Feb 14 15:01:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.51700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.96699999999999997,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.73499999999999999},"min":{"1min":0.60499999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.96899999999999997}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.0069999999999999,"5min":1.044,"15min":1.044},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.88500000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84199999999999997,"15min":0.84099999999999997},"min":{"1min":0.61399999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":1.002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98699999999999999,"5min":1.107,"15min":1.107},"last":0.95199999999999996}]},{"osd":11,"last update":"Wed Feb 14 15:01:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.026},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.98499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":496776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.51000000000000001},"min":{"1min":0.40899999999999997,"5min":0.25900000000000001,"15min":0.253},"max":{"1min":0.93200000000000005,"5min":1.002,"15min":1.002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54300000000000004,"15min":0.51100000000000001},"min":{"1min":0.40000000000000002,"5min":0.30599999999999999,"15min":0.28199999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.53000000000000003},"min":{"1min":0.48999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.58699999999999997,"15min":0.52500000000000002},"min":{"1min":0.47399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.61399999999999999}]},{"osd":5,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83399999999999996}]},{"osd":6,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.34499999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81299999999999994,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.123,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.5,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.55300000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79900000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.69299999999999995},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.109,"5min":1.153,"15min":1.153},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0649999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.48799999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0880000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.53900000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.873}]},{"osd":11,"last update":"Wed Feb 14 15:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.751,"15min":0.748},"min":{"1min":0.59499999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0980000000000001,"5min":1.802,"15min":1.802},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.73999999999999999,"15min":0.72599999999999998},"min":{"1min":0.67600000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1659999999999999,"5min":1.242,"15min":1.242},"last":0.89600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1032192,"data_stored":496066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77400000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.99399999999999999}]},{"osd":1,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58099999999999996,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.46200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79500000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.95299999999999996}]},{"osd":4,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Feb 14 15:01:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.32600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.441,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89600000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.46600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76900000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80200000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.46700000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93000000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88500000000000001}]},{"osd":9,"last update":"Wed Feb 14 15:02:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.498,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":1.01,"15min":1.01},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.875}]},{"osd":10,"last update":"Wed Feb 14 15:01:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.627,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.66900000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.84499999999999997}]},{"osd":11,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.92300000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":495582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.35599999999999998,"15min":0.32100000000000001},"min":{"1min":0.26900000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.57599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.747,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.224,"5min":19.896999999999998,"15min":19.896999999999998},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.753,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.874,"5min":19.911999999999999,"15min":19.911999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.60099999999999998,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2729999999999999,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.54200000000000004}]},{"osd":4,"last update":"Wed Feb 14 15:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.45600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93999999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.0309999999999999,"15min":1.4670000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":19.879999999999999,"15min":19.879999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.45100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78600000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.48499999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.42499999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:01:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.628},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.78700000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70499999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64300000000000002,"15min":0.67200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.64500000000000002,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52600000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.748},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.68100000000000005}]},{"osd":11,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76200000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1101824,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.42199999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72299999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.376}]},{"osd":2,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.755},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.56100000000000005,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.91900000000000004,"5min":0.997,"15min":0.997},"last":0.61099999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89500000000000002,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89100000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:01:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61299999999999999,"15min":0.57099999999999995},"min":{"1min":0.50900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.874,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.64700000000000002,"15min":0.60999999999999999},"min":{"1min":0.48599999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89300000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.503,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72499999999999998,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92600000000000005,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.012,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.79500000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.60399999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98499999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.006,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88400000000000001}]},{"osd":9,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":1.04,"15min":1.04},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0609999999999999,"5min":1.105,"15min":1.105},"last":0.81499999999999995}]},{"osd":10,"last update":"Wed Feb 14 15:01:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96499999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.57099999999999995}]},{"osd":11,"last update":"Wed Feb 14 15:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.86599999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1638400,"data_stored":1091238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.47399999999999998},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77400000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.82999999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.54900000000000004,"15min":0.52000000000000002},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52500000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.35699999999999998},"min":{"1min":0.32800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.379},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.38900000000000001},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.41299999999999998}]},{"osd":3,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89900000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.76200000000000001}]},{"osd":4,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.877,"5min":0.878,"15min":0.878},"last":0.64100000000000001}]},{"osd":6,"last update":"Wed Feb 14 15:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58399999999999996,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.57599999999999996},"min":{"1min":0.45200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89200000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72299999999999998}]},{"osd":8,"last update":"Wed Feb 14 15:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.42499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93000000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.745},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57999999999999996,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47299999999999998}]},{"osd":9,"last update":"Wed Feb 14 15:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95799999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.038,"15min":1.038},"last":0.77700000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:01:36 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54300000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0229999999999999,"5min":1.032,"15min":1.032},"last":0.80300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T15:02:28.634 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T15:02:28.883 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:28 smithi076 ceph-mon[37057]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:02:29.238 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:28 smithi017 ceph-mon[32549]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:02:29.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:28 smithi050 ceph-mon[36512]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:02:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:29 smithi076 ceph-mon[37057]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:30.256 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:29 smithi017 ceph-mon[32549]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:30.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:29 smithi050 ceph-mon[36512]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 -- 172.21.15.17:0/886791446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 msgr2=0x7ffbac10ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 --2- 172.21.15.17:0/886791446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac10ebe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b0d0 tx=0x7ffba400b3e0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 -- 172.21.15.17:0/886791446 shutdown_connections 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 --2- 172.21.15.17:0/886791446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac10ebe0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 --2- 172.21.15.17:0/886791446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac107da0 0x7ffbac108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 --2- 172.21.15.17:0/886791446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbac06a420 0x7ffbac06a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 -- 172.21.15.17:0/886791446 >> 172.21.15.17:0/886791446 conn(0x7ffbac076c40 msgr2=0x7ffbac077050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 -- 172.21.15.17:0/886791446 shutdown_connections 2024-02-14T15:02:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.484+0000 7ffbb2254700 1 -- 172.21.15.17:0/886791446 wait complete. 2024-02-14T15:02:30.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 Processor -- start 2024-02-14T15:02:30.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 -- start start 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbac06a420 0x7ffbac198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac107da0 0x7ffbac19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac19d1b0 con 0x7ffbac06adc0 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac19d320 con 0x7ffbac06a420 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbb2254700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac19d4e0 con 0x7ffbac107da0 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbaaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbaaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46846/0 (socket says 172.21.15.17:46846) 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbaaffd700 1 -- 172.21.15.17:0/2164262890 learned_addr learned my addr 172.21.15.17:0/2164262890 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbab7fe700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbac06a420 0x7ffbac198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.485+0000 7ffbabfff700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac107da0 0x7ffbac19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbaaffd700 1 -- 172.21.15.17:0/2164262890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac107da0 msgr2=0x7ffbac19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbaaffd700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac107da0 0x7ffbac19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbaaffd700 1 -- 172.21.15.17:0/2164262890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbac06a420 msgr2=0x7ffbac198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbaaffd700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbac06a420 0x7ffbac198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbaaffd700 1 -- 172.21.15.17:0/2164262890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4009d00 con 0x7ffbac06adc0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbaaffd700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac1988b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffba000d950 tx=0x7ffba000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffba8ff9700 1 -- 172.21.15.17:0/2164262890 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0009940 con 0x7ffbac06adc0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffba8ff9700 1 -- 172.21.15.17:0/2164262890 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba0010460 con 0x7ffbac06adc0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac19d7c0 con 0x7ffbac06adc0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffba8ff9700 1 -- 172.21.15.17:0/2164262890 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000f620 con 0x7ffbac06adc0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.486+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac1a5020 con 0x7ffbac06adc0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.487+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac10a2c0 con 0x7ffbac06adc0 2024-02-14T15:02:30.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.489+0000 7ffba8ff9700 1 -- 172.21.15.17:0/2164262890 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ffba00105d0 con 0x7ffbac06adc0 2024-02-14T15:02:30.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.489+0000 7ffba8ff9700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb940a2260 0x7ffb940a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:30.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.490+0000 7ffba8ff9700 1 -- 172.21.15.17:0/2164262890 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ffba00cdff0 con 0x7ffbac06adc0 2024-02-14T15:02:30.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.490+0000 7ffbab7fe700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb940a2260 0x7ffb940a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:30.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.490+0000 7ffbab7fe700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb940a2260 0x7ffb940a4720 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c005950 tx=0x7ffb9c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:30.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.493+0000 7ffba8ff9700 1 -- 172.21.15.17:0/2164262890 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffba009a630 con 0x7ffbac06adc0 2024-02-14T15:02:30.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.695+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbac111d80 con 0x7ffb940a2260 2024-02-14T15:02:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.707+0000 7ffba8ff9700 1 -- 172.21.15.17:0/2164262890 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427757 (secure 0 0 0) 0x7ffbac111d80 con 0x7ffb940a2260 2024-02-14T15:02:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.711+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb940a2260 msgr2=0x7ffb940a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.711+0000 7ffbb2254700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb940a2260 0x7ffb940a4720 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c005950 tx=0x7ffb9c00b410 comp rx=0 tx=0).stop 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.711+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 msgr2=0x7ffbac1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.711+0000 7ffbb2254700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac1988b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffba000d950 tx=0x7ffba000dc60 comp rx=0 tx=0).stop 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 shutdown_connections 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb940a2260 0x7ffb940a4720 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac107da0 0x7ffbac19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbac06a420 0x7ffbac198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 --2- 172.21.15.17:0/2164262890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbac06adc0 0x7ffbac1988b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 >> 172.21.15.17:0/2164262890 conn(0x7ffbac076c40 msgr2=0x7ffbac0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 shutdown_connections 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:30.712+0000 7ffbb2254700 1 -- 172.21.15.17:0/2164262890 wait complete. 2024-02-14T15:02:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-02-14T15:02:31.450 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-02-14T15:02:30.679512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657959,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":602,"num_read_kb":421,"num_write":344,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1244674,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388304,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553520,"statfs":{"total":1151940427776,"available":1151542804480,"internally_reserved":0,"allocated":14385152,"data_stored":7949519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002895"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604662+0000","last_change":"2024-02-14T15:00:49.281057+0000","last_active":"2024-02-14T15:01:44.604662+0000","last_peered":"2024-02-14T15:01:44.604662+0000","last_clean":"2024-02-14T15:01:44.604662+0000","last_became_active":"2024-02-14T15:00:49.280642+0000","last_became_peered":"2024-02-14T15:00:49.280642+0000","last_unstale":"2024-02-14T15:01:44.604662+0000","last_undegraded":"2024-02-14T15:01:44.604662+0000","last_fullsized":"2024-02-14T15:01:44.604662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:03.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597007+0000","last_change":"2024-02-14T15:00:49.282794+0000","last_active":"2024-02-14T15:01:44.597007+0000","last_peered":"2024-02-14T15:01:44.597007+0000","last_clean":"2024-02-14T15:01:44.597007+0000","last_became_active":"2024-02-14T15:00:49.281441+0000","last_became_peered":"2024-02-14T15:00:49.281441+0000","last_unstale":"2024-02-14T15:01:44.597007+0000","last_undegraded":"2024-02-14T15:01:44.597007+0000","last_fullsized":"2024-02-14T15:01:44.597007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:57:36.577281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596895+0000","last_change":"2024-02-14T15:00:49.286253+0000","last_active":"2024-02-14T15:01:44.596895+0000","last_peered":"2024-02-14T15:01:44.596895+0000","last_clean":"2024-02-14T15:01:44.596895+0000","last_became_active":"2024-02-14T15:00:49.285950+0000","last_became_peered":"2024-02-14T15:00:49.285950+0000","last_unstale":"2024-02-14T15:01:44.596895+0000","last_undegraded":"2024-02-14T15:01:44.596895+0000","last_fullsized":"2024-02-14T15:01:44.596895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:53.395339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598104+0000","last_change":"2024-02-14T15:00:49.272764+0000","last_active":"2024-02-14T15:01:44.598104+0000","last_peered":"2024-02-14T15:01:44.598104+0000","last_clean":"2024-02-14T15:01:44.598104+0000","last_became_active":"2024-02-14T15:00:49.272430+0000","last_became_peered":"2024-02-14T15:00:49.272430+0000","last_unstale":"2024-02-14T15:01:44.598104+0000","last_undegraded":"2024-02-14T15:01:44.598104+0000","last_fullsized":"2024-02-14T15:01:44.598104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:36.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603685+0000","last_change":"2024-02-14T15:00:49.275028+0000","last_active":"2024-02-14T15:01:44.603685+0000","last_peered":"2024-02-14T15:01:44.603685+0000","last_clean":"2024-02-14T15:01:44.603685+0000","last_became_active":"2024-02-14T15:00:49.274686+0000","last_became_peered":"2024-02-14T15:00:49.274686+0000","last_unstale":"2024-02-14T15:01:44.603685+0000","last_undegraded":"2024-02-14T15:01:44.603685+0000","last_fullsized":"2024-02-14T15:01:44.603685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:21:23.465321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601040+0000","last_change":"2024-02-14T15:00:49.287144+0000","last_active":"2024-02-14T15:01:44.601040+0000","last_peered":"2024-02-14T15:01:44.601040+0000","last_clean":"2024-02-14T15:01:44.601040+0000","last_became_active":"2024-02-14T15:00:49.286767+0000","last_became_peered":"2024-02-14T15:00:49.286767+0000","last_unstale":"2024-02-14T15:01:44.601040+0000","last_undegraded":"2024-02-14T15:01:44.601040+0000","last_fullsized":"2024-02-14T15:01:44.601040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:45:09.391595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604308+0000","last_change":"2024-02-14T15:00:49.274442+0000","last_active":"2024-02-14T15:01:44.604308+0000","last_peered":"2024-02-14T15:01:44.604308+0000","last_clean":"2024-02-14T15:01:44.604308+0000","last_became_active":"2024-02-14T15:00:49.274122+0000","last_became_peered":"2024-02-14T15:00:49.274122+0000","last_unstale":"2024-02-14T15:01:44.604308+0000","last_undegraded":"2024-02-14T15:01:44.604308+0000","last_fullsized":"2024-02-14T15:01:44.604308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:55.974889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601679+0000","last_change":"2024-02-14T15:00:49.285432+0000","last_active":"2024-02-14T15:01:44.601679+0000","last_peered":"2024-02-14T15:01:44.601679+0000","last_clean":"2024-02-14T15:01:44.601679+0000","last_became_active":"2024-02-14T15:00:49.283703+0000","last_became_peered":"2024-02-14T15:00:49.283703+0000","last_unstale":"2024-02-14T15:01:44.601679+0000","last_undegraded":"2024-02-14T15:01:44.601679+0000","last_fullsized":"2024-02-14T15:01:44.601679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:14.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600470+0000","last_change":"2024-02-14T15:00:49.283908+0000","last_active":"2024-02-14T15:01:44.600470+0000","last_peered":"2024-02-14T15:01:44.600470+0000","last_clean":"2024-02-14T15:01:44.600470+0000","last_became_active":"2024-02-14T15:00:49.283131+0000","last_became_peered":"2024-02-14T15:00:49.283131+0000","last_unstale":"2024-02-14T15:01:44.600470+0000","last_undegraded":"2024-02-14T15:01:44.600470+0000","last_fullsized":"2024-02-14T15:01:44.600470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:42.247624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600593+0000","last_change":"2024-02-14T15:00:49.287229+0000","last_active":"2024-02-14T15:01:44.600593+0000","last_peered":"2024-02-14T15:01:44.600593+0000","last_clean":"2024-02-14T15:01:44.600593+0000","last_became_active":"2024-02-14T15:00:49.286791+0000","last_became_peered":"2024-02-14T15:00:49.286791+0000","last_unstale":"2024-02-14T15:01:44.600593+0000","last_undegraded":"2024-02-14T15:01:44.600593+0000","last_fullsized":"2024-02-14T15:01:44.600593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:33:01.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605011+0000","last_change":"2024-02-14T15:00:49.281669+0000","last_active":"2024-02-14T15:01:44.605011+0000","last_peered":"2024-02-14T15:01:44.605011+0000","last_clean":"2024-02-14T15:01:44.605011+0000","last_became_active":"2024-02-14T15:00:49.281298+0000","last_became_peered":"2024-02-14T15:00:49.281298+0000","last_unstale":"2024-02-14T15:01:44.605011+0000","last_undegraded":"2024-02-14T15:01:44.605011+0000","last_fullsized":"2024-02-14T15:01:44.605011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:54:40.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598239+0000","last_change":"2024-02-14T15:00:49.289679+0000","last_active":"2024-02-14T15:01:44.598239+0000","last_peered":"2024-02-14T15:01:44.598239+0000","last_clean":"2024-02-14T15:01:44.598239+0000","last_became_active":"2024-02-14T15:00:49.288767+0000","last_became_peered":"2024-02-14T15:00:49.288767+0000","last_unstale":"2024-02-14T15:01:44.598239+0000","last_undegraded":"2024-02-14T15:01:44.598239+0000","last_fullsized":"2024-02-14T15:01:44.598239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:16:20.181517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605558+0000","last_change":"2024-02-14T15:00:49.257471+0000","last_active":"2024-02-14T15:01:44.605558+0000","last_peered":"2024-02-14T15:01:44.605558+0000","last_clean":"2024-02-14T15:01:44.605558+0000","last_became_active":"2024-02-14T15:00:49.256766+0000","last_became_peered":"2024-02-14T15:00:49.256766+0000","last_unstale":"2024-02-14T15:01:44.605558+0000","last_undegraded":"2024-02-14T15:01:44.605558+0000","last_fullsized":"2024-02-14T15:01:44.605558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:14:00.525621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596566+0000","last_change":"2024-02-14T15:00:49.278462+0000","last_active":"2024-02-14T15:01:44.596566+0000","last_peered":"2024-02-14T15:01:44.596566+0000","last_clean":"2024-02-14T15:01:44.596566+0000","last_became_active":"2024-02-14T15:00:49.278114+0000","last_became_peered":"2024-02-14T15:00:49.278114+0000","last_unstale":"2024-02-14T15:01:44.596566+0000","last_undegraded":"2024-02-14T15:01:44.596566+0000","last_fullsized":"2024-02-14T15:01:44.596566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:02:16.601359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596566+0000","last_change":"2024-02-14T15:00:49.270653+0000","last_active":"2024-02-14T15:01:44.596566+0000","last_peered":"2024-02-14T15:01:44.596566+0000","last_clean":"2024-02-14T15:01:44.596566+0000","last_became_active":"2024-02-14T15:00:49.270334+0000","last_became_peered":"2024-02-14T15:00:49.270334+0000","last_unstale":"2024-02-14T15:01:44.596566+0000","last_undegraded":"2024-02-14T15:01:44.596566+0000","last_fullsized":"2024-02-14T15:01:44.596566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:13:53.840909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599429+0000","last_change":"2024-02-14T15:00:49.280597+0000","last_active":"2024-02-14T15:01:44.599429+0000","last_peered":"2024-02-14T15:01:44.599429+0000","last_clean":"2024-02-14T15:01:44.599429+0000","last_became_active":"2024-02-14T15:00:49.279612+0000","last_became_peered":"2024-02-14T15:00:49.279612+0000","last_unstale":"2024-02-14T15:01:44.599429+0000","last_undegraded":"2024-02-14T15:01:44.599429+0000","last_fullsized":"2024-02-14T15:01:44.599429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:11.676608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604531+0000","last_change":"2024-02-14T15:00:49.281422+0000","last_active":"2024-02-14T15:01:44.604531+0000","last_peered":"2024-02-14T15:01:44.604531+0000","last_clean":"2024-02-14T15:01:44.604531+0000","last_became_active":"2024-02-14T15:00:49.281048+0000","last_became_peered":"2024-02-14T15:00:49.281048+0000","last_unstale":"2024-02-14T15:01:44.604531+0000","last_undegraded":"2024-02-14T15:01:44.604531+0000","last_fullsized":"2024-02-14T15:01:44.604531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:04.994390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599774+0000","last_change":"2024-02-14T15:00:49.288245+0000","last_active":"2024-02-14T15:01:44.599774+0000","last_peered":"2024-02-14T15:01:44.599774+0000","last_clean":"2024-02-14T15:01:44.599774+0000","last_became_active":"2024-02-14T15:00:49.287908+0000","last_became_peered":"2024-02-14T15:00:49.287908+0000","last_unstale":"2024-02-14T15:01:44.599774+0000","last_undegraded":"2024-02-14T15:01:44.599774+0000","last_fullsized":"2024-02-14T15:01:44.599774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:52.397826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598096+0000","last_change":"2024-02-14T15:00:49.275428+0000","last_active":"2024-02-14T15:01:44.598096+0000","last_peered":"2024-02-14T15:01:44.598096+0000","last_clean":"2024-02-14T15:01:44.598096+0000","last_became_active":"2024-02-14T15:00:49.273464+0000","last_became_peered":"2024-02-14T15:00:49.273464+0000","last_unstale":"2024-02-14T15:01:44.598096+0000","last_undegraded":"2024-02-14T15:01:44.598096+0000","last_fullsized":"2024-02-14T15:01:44.598096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:34.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597554+0000","last_change":"2024-02-14T15:00:53.587219+0000","last_active":"2024-02-14T15:01:44.597554+0000","last_peered":"2024-02-14T15:01:44.597554+0000","last_clean":"2024-02-14T15:01:44.597554+0000","last_became_active":"2024-02-14T15:00:53.586917+0000","last_became_peered":"2024-02-14T15:00:53.586917+0000","last_unstale":"2024-02-14T15:01:44.597554+0000","last_undegraded":"2024-02-14T15:01:44.597554+0000","last_fullsized":"2024-02-14T15:01:44.597554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:39:38.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599865+0000","last_change":"2024-02-14T15:00:48.269243+0000","last_active":"2024-02-14T15:01:44.599865+0000","last_peered":"2024-02-14T15:01:44.599865+0000","last_clean":"2024-02-14T15:01:44.599865+0000","last_became_active":"2024-02-14T15:00:48.268883+0000","last_became_peered":"2024-02-14T15:00:48.268883+0000","last_unstale":"2024-02-14T15:01:44.599865+0000","last_undegraded":"2024-02-14T15:01:44.599865+0000","last_fullsized":"2024-02-14T15:01:44.599865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:08.530092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602527+0000","last_change":"2024-02-14T15:00:49.282458+0000","last_active":"2024-02-14T15:01:44.602527+0000","last_peered":"2024-02-14T15:01:44.602527+0000","last_clean":"2024-02-14T15:01:44.602527+0000","last_became_active":"2024-02-14T15:00:49.281982+0000","last_became_peered":"2024-02-14T15:00:49.281982+0000","last_unstale":"2024-02-14T15:01:44.602527+0000","last_undegraded":"2024-02-14T15:01:44.602527+0000","last_fullsized":"2024-02-14T15:01:44.602527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:51:33.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597726+0000","last_change":"2024-02-14T15:00:53.586578+0000","last_active":"2024-02-14T15:01:44.597726+0000","last_peered":"2024-02-14T15:01:44.597726+0000","last_clean":"2024-02-14T15:01:44.597726+0000","last_became_active":"2024-02-14T15:00:53.586109+0000","last_became_peered":"2024-02-14T15:00:53.586109+0000","last_unstale":"2024-02-14T15:01:44.597726+0000","last_undegraded":"2024-02-14T15:01:44.597726+0000","last_fullsized":"2024-02-14T15:01:44.597726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:07.332435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602661+0000","last_change":"2024-02-14T15:00:48.281953+0000","last_active":"2024-02-14T15:01:44.602661+0000","last_peered":"2024-02-14T15:01:44.602661+0000","last_clean":"2024-02-14T15:01:44.602661+0000","last_became_active":"2024-02-14T15:00:48.281590+0000","last_became_peered":"2024-02-14T15:00:48.281590+0000","last_unstale":"2024-02-14T15:01:44.602661+0000","last_undegraded":"2024-02-14T15:01:44.602661+0000","last_fullsized":"2024-02-14T15:01:44.602661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:22.991037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597012+0000","last_change":"2024-02-14T15:00:49.284729+0000","last_active":"2024-02-14T15:01:44.597012+0000","last_peered":"2024-02-14T15:01:44.597012+0000","last_clean":"2024-02-14T15:01:44.597012+0000","last_became_active":"2024-02-14T15:00:49.284013+0000","last_became_peered":"2024-02-14T15:00:49.284013+0000","last_unstale":"2024-02-14T15:01:44.597012+0000","last_undegraded":"2024-02-14T15:01:44.597012+0000","last_fullsized":"2024-02-14T15:01:44.597012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:03:31.313032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598204+0000","last_change":"2024-02-14T15:00:53.589825+0000","last_active":"2024-02-14T15:01:44.598204+0000","last_peered":"2024-02-14T15:01:44.598204+0000","last_clean":"2024-02-14T15:01:44.598204+0000","last_became_active":"2024-02-14T15:00:53.589424+0000","last_became_peered":"2024-02-14T15:00:53.589424+0000","last_unstale":"2024-02-14T15:01:44.598204+0000","last_undegraded":"2024-02-14T15:01:44.598204+0000","last_fullsized":"2024-02-14T15:01:44.598204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:24.522152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602765+0000","last_change":"2024-02-14T15:00:48.276884+0000","last_active":"2024-02-14T15:01:44.602765+0000","last_peered":"2024-02-14T15:01:44.602765+0000","last_clean":"2024-02-14T15:01:44.602765+0000","last_became_active":"2024-02-14T15:00:48.276576+0000","last_became_peered":"2024-02-14T15:00:48.276576+0000","last_unstale":"2024-02-14T15:01:44.602765+0000","last_undegraded":"2024-02-14T15:01:44.602765+0000","last_fullsized":"2024-02-14T15:01:44.602765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:25.793368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600320+0000","last_change":"2024-02-14T15:00:49.277602+0000","last_active":"2024-02-14T15:01:44.600320+0000","last_peered":"2024-02-14T15:01:44.600320+0000","last_clean":"2024-02-14T15:01:44.600320+0000","last_became_active":"2024-02-14T15:00:49.277249+0000","last_became_peered":"2024-02-14T15:00:49.277249+0000","last_unstale":"2024-02-14T15:01:44.600320+0000","last_undegraded":"2024-02-14T15:01:44.600320+0000","last_fullsized":"2024-02-14T15:01:44.600320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.347962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597955+0000","last_change":"2024-02-14T15:00:53.587838+0000","last_active":"2024-02-14T15:01:44.597955+0000","last_peered":"2024-02-14T15:01:44.597955+0000","last_clean":"2024-02-14T15:01:44.597955+0000","last_became_active":"2024-02-14T15:00:53.587012+0000","last_became_peered":"2024-02-14T15:00:53.587012+0000","last_unstale":"2024-02-14T15:01:44.597955+0000","last_undegraded":"2024-02-14T15:01:44.597955+0000","last_fullsized":"2024-02-14T15:01:44.597955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:32:12.276578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602772+0000","last_change":"2024-02-14T15:00:48.273529+0000","last_active":"2024-02-14T15:01:44.602772+0000","last_peered":"2024-02-14T15:01:44.602772+0000","last_clean":"2024-02-14T15:01:44.602772+0000","last_became_active":"2024-02-14T15:00:48.272555+0000","last_became_peered":"2024-02-14T15:00:48.272555+0000","last_unstale":"2024-02-14T15:01:44.602772+0000","last_undegraded":"2024-02-14T15:01:44.602772+0000","last_fullsized":"2024-02-14T15:01:44.602772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:19.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602297+0000","last_change":"2024-02-14T15:00:49.278834+0000","last_active":"2024-02-14T15:01:44.602297+0000","last_peered":"2024-02-14T15:01:44.602297+0000","last_clean":"2024-02-14T15:01:44.602297+0000","last_became_active":"2024-02-14T15:00:49.277095+0000","last_became_peered":"2024-02-14T15:00:49.277095+0000","last_unstale":"2024-02-14T15:01:44.602297+0000","last_undegraded":"2024-02-14T15:01:44.602297+0000","last_fullsized":"2024-02-14T15:01:44.602297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:27.291974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597933+0000","last_change":"2024-02-14T15:00:53.577162+0000","last_active":"2024-02-14T15:01:44.597933+0000","last_peered":"2024-02-14T15:01:44.597933+0000","last_clean":"2024-02-14T15:01:44.597933+0000","last_became_active":"2024-02-14T15:00:53.576809+0000","last_became_peered":"2024-02-14T15:00:53.576809+0000","last_unstale":"2024-02-14T15:01:44.597933+0000","last_undegraded":"2024-02-14T15:01:44.597933+0000","last_fullsized":"2024-02-14T15:01:44.597933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:37.698842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:28.666458+0000","last_change":"2024-02-14T15:00:48.276146+0000","last_active":"2024-02-14T15:02:28.666458+0000","last_peered":"2024-02-14T15:02:28.666458+0000","last_clean":"2024-02-14T15:02:28.666458+0000","last_became_active":"2024-02-14T15:00:48.275833+0000","last_became_peered":"2024-02-14T15:00:48.275833+0000","last_unstale":"2024-02-14T15:02:28.666458+0000","last_undegraded":"2024-02-14T15:02:28.666458+0000","last_fullsized":"2024-02-14T15:02:28.666458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:46.649486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602443+0000","last_change":"2024-02-14T15:00:49.267285+0000","last_active":"2024-02-14T15:01:44.602443+0000","last_peered":"2024-02-14T15:01:44.602443+0000","last_clean":"2024-02-14T15:01:44.602443+0000","last_became_active":"2024-02-14T15:00:49.265592+0000","last_became_peered":"2024-02-14T15:00:49.265592+0000","last_unstale":"2024-02-14T15:01:44.602443+0000","last_undegraded":"2024-02-14T15:01:44.602443+0000","last_fullsized":"2024-02-14T15:01:44.602443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:18.469835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597929+0000","last_change":"2024-02-14T15:00:53.582085+0000","last_active":"2024-02-14T15:01:44.597929+0000","last_peered":"2024-02-14T15:01:44.597929+0000","last_clean":"2024-02-14T15:01:44.597929+0000","last_became_active":"2024-02-14T15:00:53.581706+0000","last_became_peered":"2024-02-14T15:00:53.581706+0000","last_unstale":"2024-02-14T15:01:44.597929+0000","last_undegraded":"2024-02-14T15:01:44.597929+0000","last_fullsized":"2024-02-14T15:01:44.597929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:32.481142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601026+0000","last_change":"2024-02-14T15:00:48.268975+0000","last_active":"2024-02-14T15:01:44.601026+0000","last_peered":"2024-02-14T15:01:44.601026+0000","last_clean":"2024-02-14T15:01:44.601026+0000","last_became_active":"2024-02-14T15:00:48.268547+0000","last_became_peered":"2024-02-14T15:00:48.268547+0000","last_unstale":"2024-02-14T15:01:44.601026+0000","last_undegraded":"2024-02-14T15:01:44.601026+0000","last_fullsized":"2024-02-14T15:01:44.601026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:13.937917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601675+0000","last_change":"2024-02-14T15:00:49.276909+0000","last_active":"2024-02-14T15:01:44.601675+0000","last_peered":"2024-02-14T15:01:44.601675+0000","last_clean":"2024-02-14T15:01:44.601675+0000","last_became_active":"2024-02-14T15:00:49.276533+0000","last_became_peered":"2024-02-14T15:00:49.276533+0000","last_unstale":"2024-02-14T15:01:44.601675+0000","last_undegraded":"2024-02-14T15:01:44.601675+0000","last_fullsized":"2024-02-14T15:01:44.601675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:35:31.321455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604508+0000","last_change":"2024-02-14T15:00:53.592223+0000","last_active":"2024-02-14T15:01:44.604508+0000","last_peered":"2024-02-14T15:01:44.604508+0000","last_clean":"2024-02-14T15:01:44.604508+0000","last_became_active":"2024-02-14T15:00:53.591883+0000","last_became_peered":"2024-02-14T15:00:53.591883+0000","last_unstale":"2024-02-14T15:01:44.604508+0000","last_undegraded":"2024-02-14T15:01:44.604508+0000","last_fullsized":"2024-02-14T15:01:44.604508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:10.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605326+0000","last_change":"2024-02-14T15:00:48.274485+0000","last_active":"2024-02-14T15:01:44.605326+0000","last_peered":"2024-02-14T15:01:44.605326+0000","last_clean":"2024-02-14T15:01:44.605326+0000","last_became_active":"2024-02-14T15:00:48.274160+0000","last_became_peered":"2024-02-14T15:00:48.274160+0000","last_unstale":"2024-02-14T15:01:44.605326+0000","last_undegraded":"2024-02-14T15:01:44.605326+0000","last_fullsized":"2024-02-14T15:01:44.605326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:00:47.105069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601840+0000","last_change":"2024-02-14T15:00:49.272577+0000","last_active":"2024-02-14T15:01:44.601840+0000","last_peered":"2024-02-14T15:01:44.601840+0000","last_clean":"2024-02-14T15:01:44.601840+0000","last_became_active":"2024-02-14T15:00:49.271670+0000","last_became_peered":"2024-02-14T15:00:49.271670+0000","last_unstale":"2024-02-14T15:01:44.601840+0000","last_undegraded":"2024-02-14T15:01:44.601840+0000","last_fullsized":"2024-02-14T15:01:44.601840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:19.968389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596942+0000","last_change":"2024-02-14T15:00:53.581212+0000","last_active":"2024-02-14T15:01:44.596942+0000","last_peered":"2024-02-14T15:01:44.596942+0000","last_clean":"2024-02-14T15:01:44.596942+0000","last_became_active":"2024-02-14T15:00:53.580782+0000","last_became_peered":"2024-02-14T15:00:53.580782+0000","last_unstale":"2024-02-14T15:01:44.596942+0000","last_undegraded":"2024-02-14T15:01:44.596942+0000","last_fullsized":"2024-02-14T15:01:44.596942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:14.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600793+0000","last_change":"2024-02-14T15:00:48.273572+0000","last_active":"2024-02-14T15:01:44.600793+0000","last_peered":"2024-02-14T15:01:44.600793+0000","last_clean":"2024-02-14T15:01:44.600793+0000","last_became_active":"2024-02-14T15:00:48.273138+0000","last_became_peered":"2024-02-14T15:00:48.273138+0000","last_unstale":"2024-02-14T15:01:44.600793+0000","last_undegraded":"2024-02-14T15:01:44.600793+0000","last_fullsized":"2024-02-14T15:01:44.600793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:04.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600031+0000","last_change":"2024-02-14T15:00:49.288226+0000","last_active":"2024-02-14T15:01:44.600031+0000","last_peered":"2024-02-14T15:01:44.600031+0000","last_clean":"2024-02-14T15:01:44.600031+0000","last_became_active":"2024-02-14T15:00:49.287874+0000","last_became_peered":"2024-02-14T15:00:49.287874+0000","last_unstale":"2024-02-14T15:01:44.600031+0000","last_undegraded":"2024-02-14T15:01:44.600031+0000","last_fullsized":"2024-02-14T15:01:44.600031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:24:02.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602056+0000","last_change":"2024-02-14T15:00:53.590543+0000","last_active":"2024-02-14T15:01:44.602056+0000","last_peered":"2024-02-14T15:01:44.602056+0000","last_clean":"2024-02-14T15:01:44.602056+0000","last_became_active":"2024-02-14T15:00:53.589072+0000","last_became_peered":"2024-02-14T15:00:53.589072+0000","last_unstale":"2024-02-14T15:01:44.602056+0000","last_undegraded":"2024-02-14T15:01:44.602056+0000","last_fullsized":"2024-02-14T15:01:44.602056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:13.685655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:26.663045+0000","last_change":"2024-02-14T15:00:48.272027+0000","last_active":"2024-02-14T15:02:26.663045+0000","last_peered":"2024-02-14T15:02:26.663045+0000","last_clean":"2024-02-14T15:02:26.663045+0000","last_became_active":"2024-02-14T15:00:48.271587+0000","last_became_peered":"2024-02-14T15:00:48.271587+0000","last_unstale":"2024-02-14T15:02:26.663045+0000","last_undegraded":"2024-02-14T15:02:26.663045+0000","last_fullsized":"2024-02-14T15:02:26.663045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:33:03.566189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600906+0000","last_change":"2024-02-14T15:00:49.278936+0000","last_active":"2024-02-14T15:01:44.600906+0000","last_peered":"2024-02-14T15:01:44.600906+0000","last_clean":"2024-02-14T15:01:44.600906+0000","last_became_active":"2024-02-14T15:00:49.278318+0000","last_became_peered":"2024-02-14T15:00:49.278318+0000","last_unstale":"2024-02-14T15:01:44.600906+0000","last_undegraded":"2024-02-14T15:01:44.600906+0000","last_fullsized":"2024-02-14T15:01:44.600906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:01.327303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599341+0000","last_change":"2024-02-14T15:00:53.580340+0000","last_active":"2024-02-14T15:01:44.599341+0000","last_peered":"2024-02-14T15:01:44.599341+0000","last_clean":"2024-02-14T15:01:44.599341+0000","last_became_active":"2024-02-14T15:00:53.579742+0000","last_became_peered":"2024-02-14T15:00:53.579742+0000","last_unstale":"2024-02-14T15:01:44.599341+0000","last_undegraded":"2024-02-14T15:01:44.599341+0000","last_fullsized":"2024-02-14T15:01:44.599341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:38:33.182328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605218+0000","last_change":"2024-02-14T15:00:48.271891+0000","last_active":"2024-02-14T15:01:44.605218+0000","last_peered":"2024-02-14T15:01:44.605218+0000","last_clean":"2024-02-14T15:01:44.605218+0000","last_became_active":"2024-02-14T15:00:48.271419+0000","last_became_peered":"2024-02-14T15:00:48.271419+0000","last_unstale":"2024-02-14T15:01:44.605218+0000","last_undegraded":"2024-02-14T15:01:44.605218+0000","last_fullsized":"2024-02-14T15:01:44.605218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:03:04.725647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596985+0000","last_change":"2024-02-14T15:00:49.277025+0000","last_active":"2024-02-14T15:01:44.596985+0000","last_peered":"2024-02-14T15:01:44.596985+0000","last_clean":"2024-02-14T15:01:44.596985+0000","last_became_active":"2024-02-14T15:00:49.276694+0000","last_became_peered":"2024-02-14T15:00:49.276694+0000","last_unstale":"2024-02-14T15:01:44.596985+0000","last_undegraded":"2024-02-14T15:01:44.596985+0000","last_fullsized":"2024-02-14T15:01:44.596985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:43:07.249407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.607053+0000","last_change":"2024-02-14T15:01:44.607053+0000","last_active":"2024-02-14T15:01:44.607053+0000","last_peered":"2024-02-14T15:01:44.607053+0000","last_clean":"2024-02-14T15:01:44.607053+0000","last_became_active":"2024-02-14T15:01:44.606596+0000","last_became_peered":"2024-02-14T15:01:44.606596+0000","last_unstale":"2024-02-14T15:01:44.607053+0000","last_undegraded":"2024-02-14T15:01:44.607053+0000","last_fullsized":"2024-02-14T15:01:44.607053+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:35.949953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602188+0000","last_change":"2024-02-14T15:00:48.272866+0000","last_active":"2024-02-14T15:01:44.602188+0000","last_peered":"2024-02-14T15:01:44.602188+0000","last_clean":"2024-02-14T15:01:44.602188+0000","last_became_active":"2024-02-14T15:00:48.272520+0000","last_became_peered":"2024-02-14T15:00:48.272520+0000","last_unstale":"2024-02-14T15:01:44.602188+0000","last_undegraded":"2024-02-14T15:01:44.602188+0000","last_fullsized":"2024-02-14T15:01:44.602188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:39.434577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600429+0000","last_change":"2024-02-14T15:00:49.280994+0000","last_active":"2024-02-14T15:01:44.600429+0000","last_peered":"2024-02-14T15:01:44.600429+0000","last_clean":"2024-02-14T15:01:44.600429+0000","last_became_active":"2024-02-14T15:00:49.280205+0000","last_became_peered":"2024-02-14T15:00:49.280205+0000","last_unstale":"2024-02-14T15:01:44.600429+0000","last_undegraded":"2024-02-14T15:01:44.600429+0000","last_fullsized":"2024-02-14T15:01:44.600429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:53:48.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599386+0000","last_change":"2024-02-14T15:00:53.582599+0000","last_active":"2024-02-14T15:01:44.599386+0000","last_peered":"2024-02-14T15:01:44.599386+0000","last_clean":"2024-02-14T15:01:44.599386+0000","last_became_active":"2024-02-14T15:00:53.582231+0000","last_became_peered":"2024-02-14T15:00:53.582231+0000","last_unstale":"2024-02-14T15:01:44.599386+0000","last_undegraded":"2024-02-14T15:01:44.599386+0000","last_fullsized":"2024-02-14T15:01:44.599386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:19.126264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":229,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:27.664395+0000","last_change":"2024-02-14T15:01:44.615393+0000","last_active":"2024-02-14T15:02:27.664395+0000","last_peered":"2024-02-14T15:02:27.664395+0000","last_clean":"2024-02-14T15:02:27.664395+0000","last_became_active":"2024-02-14T15:01:44.604294+0000","last_became_peered":"2024-02-14T15:01:44.604294+0000","last_unstale":"2024-02-14T15:02:27.664395+0000","last_undegraded":"2024-02-14T15:02:27.664395+0000","last_fullsized":"2024-02-14T15:02:27.664395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:00.459860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":152,"num_read_kb":138,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602420+0000","last_change":"2024-02-14T15:00:49.270144+0000","last_active":"2024-02-14T15:01:44.602420+0000","last_peered":"2024-02-14T15:01:44.602420+0000","last_clean":"2024-02-14T15:01:44.602420+0000","last_became_active":"2024-02-14T15:00:49.269806+0000","last_became_peered":"2024-02-14T15:00:49.269806+0000","last_unstale":"2024-02-14T15:01:44.602420+0000","last_undegraded":"2024-02-14T15:01:44.602420+0000","last_fullsized":"2024-02-14T15:01:44.602420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:42:18.559226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603924+0000","last_change":"2024-02-14T15:00:53.582755+0000","last_active":"2024-02-14T15:01:44.603924+0000","last_peered":"2024-02-14T15:01:44.603924+0000","last_clean":"2024-02-14T15:01:44.603924+0000","last_became_active":"2024-02-14T15:00:53.582375+0000","last_became_peered":"2024-02-14T15:00:53.582375+0000","last_unstale":"2024-02-14T15:01:44.603924+0000","last_undegraded":"2024-02-14T15:01:44.603924+0000","last_fullsized":"2024-02-14T15:01:44.603924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:11:49.115136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597952+0000","last_change":"2024-02-14T15:00:48.268982+0000","last_active":"2024-02-14T15:01:44.597952+0000","last_peered":"2024-02-14T15:01:44.597952+0000","last_clean":"2024-02-14T15:01:44.597952+0000","last_became_active":"2024-02-14T15:00:48.267137+0000","last_became_peered":"2024-02-14T15:00:48.267137+0000","last_unstale":"2024-02-14T15:01:44.597952+0000","last_undegraded":"2024-02-14T15:01:44.597952+0000","last_fullsized":"2024-02-14T15:01:44.597952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:44.351561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605820+0000","last_change":"2024-02-14T15:00:49.267493+0000","last_active":"2024-02-14T15:01:44.605820+0000","last_peered":"2024-02-14T15:01:44.605820+0000","last_clean":"2024-02-14T15:01:44.605820+0000","last_became_active":"2024-02-14T15:00:49.267161+0000","last_became_peered":"2024-02-14T15:00:49.267161+0000","last_unstale":"2024-02-14T15:01:44.605820+0000","last_undegraded":"2024-02-14T15:01:44.605820+0000","last_fullsized":"2024-02-14T15:01:44.605820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:35:22.997999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600261+0000","last_change":"2024-02-14T15:00:53.591201+0000","last_active":"2024-02-14T15:01:44.600261+0000","last_peered":"2024-02-14T15:01:44.600261+0000","last_clean":"2024-02-14T15:01:44.600261+0000","last_became_active":"2024-02-14T15:00:53.590769+0000","last_became_peered":"2024-02-14T15:00:53.590769+0000","last_unstale":"2024-02-14T15:01:44.600261+0000","last_undegraded":"2024-02-14T15:01:44.600261+0000","last_fullsized":"2024-02-14T15:01:44.600261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.881271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601141+0000","last_change":"2024-02-14T15:00:48.282085+0000","last_active":"2024-02-14T15:01:44.601141+0000","last_peered":"2024-02-14T15:01:44.601141+0000","last_clean":"2024-02-14T15:01:44.601141+0000","last_became_active":"2024-02-14T15:00:48.281836+0000","last_became_peered":"2024-02-14T15:00:48.281836+0000","last_unstale":"2024-02-14T15:01:44.601141+0000","last_undegraded":"2024-02-14T15:01:44.601141+0000","last_fullsized":"2024-02-14T15:01:44.601141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:07:28.145074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604787+0000","last_change":"2024-02-14T15:00:49.280013+0000","last_active":"2024-02-14T15:01:44.604787+0000","last_peered":"2024-02-14T15:01:44.604787+0000","last_clean":"2024-02-14T15:01:44.604787+0000","last_became_active":"2024-02-14T15:00:49.279667+0000","last_became_peered":"2024-02-14T15:00:49.279667+0000","last_unstale":"2024-02-14T15:01:44.604787+0000","last_undegraded":"2024-02-14T15:01:44.604787+0000","last_fullsized":"2024-02-14T15:01:44.604787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:03:16.559295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599609+0000","last_change":"2024-02-14T15:00:53.580269+0000","last_active":"2024-02-14T15:01:44.599609+0000","last_peered":"2024-02-14T15:01:44.599609+0000","last_clean":"2024-02-14T15:01:44.599609+0000","last_became_active":"2024-02-14T15:00:53.579863+0000","last_became_peered":"2024-02-14T15:00:53.579863+0000","last_unstale":"2024-02-14T15:01:44.599609+0000","last_undegraded":"2024-02-14T15:01:44.599609+0000","last_fullsized":"2024-02-14T15:01:44.599609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:37:19.275654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:28.666086+0000","last_change":"2024-02-14T15:00:48.267310+0000","last_active":"2024-02-14T15:02:28.666086+0000","last_peered":"2024-02-14T15:02:28.666086+0000","last_clean":"2024-02-14T15:02:28.666086+0000","last_became_active":"2024-02-14T15:00:48.266864+0000","last_became_peered":"2024-02-14T15:00:48.266864+0000","last_unstale":"2024-02-14T15:02:28.666086+0000","last_undegraded":"2024-02-14T15:02:28.666086+0000","last_fullsized":"2024-02-14T15:02:28.666086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:07.703549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604511+0000","last_change":"2024-02-14T15:00:49.288568+0000","last_active":"2024-02-14T15:01:44.604511+0000","last_peered":"2024-02-14T15:01:44.604511+0000","last_clean":"2024-02-14T15:01:44.604511+0000","last_became_active":"2024-02-14T15:00:49.288222+0000","last_became_peered":"2024-02-14T15:00:49.288222+0000","last_unstale":"2024-02-14T15:01:44.604511+0000","last_undegraded":"2024-02-14T15:01:44.604511+0000","last_fullsized":"2024-02-14T15:01:44.604511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:38:38.909769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605643+0000","last_change":"2024-02-14T15:00:53.585673+0000","last_active":"2024-02-14T15:01:44.605643+0000","last_peered":"2024-02-14T15:01:44.605643+0000","last_clean":"2024-02-14T15:01:44.605643+0000","last_became_active":"2024-02-14T15:00:53.585238+0000","last_became_peered":"2024-02-14T15:00:53.585238+0000","last_unstale":"2024-02-14T15:01:44.605643+0000","last_undegraded":"2024-02-14T15:01:44.605643+0000","last_fullsized":"2024-02-14T15:01:44.605643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:48:11.845921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602092+0000","last_change":"2024-02-14T15:00:48.275833+0000","last_active":"2024-02-14T15:01:44.602092+0000","last_peered":"2024-02-14T15:01:44.602092+0000","last_clean":"2024-02-14T15:01:44.602092+0000","last_became_active":"2024-02-14T15:00:48.275499+0000","last_became_peered":"2024-02-14T15:00:48.275499+0000","last_unstale":"2024-02-14T15:01:44.602092+0000","last_undegraded":"2024-02-14T15:01:44.602092+0000","last_fullsized":"2024-02-14T15:01:44.602092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:35:10.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605394+0000","last_change":"2024-02-14T15:01:44.605394+0000","last_active":"2024-02-14T15:01:44.605394+0000","last_peered":"2024-02-14T15:01:44.605394+0000","last_clean":"2024-02-14T15:01:44.605394+0000","last_became_active":"2024-02-14T15:01:44.604874+0000","last_became_peered":"2024-02-14T15:01:44.604874+0000","last_unstale":"2024-02-14T15:01:44.605394+0000","last_undegraded":"2024-02-14T15:01:44.605394+0000","last_fullsized":"2024-02-14T15:01:44.605394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:41:47.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601458+0000","last_change":"2024-02-14T15:00:53.586406+0000","last_active":"2024-02-14T15:01:44.601458+0000","last_peered":"2024-02-14T15:01:44.601458+0000","last_clean":"2024-02-14T15:01:44.601458+0000","last_became_active":"2024-02-14T15:00:53.586034+0000","last_became_peered":"2024-02-14T15:00:53.586034+0000","last_unstale":"2024-02-14T15:01:44.601458+0000","last_undegraded":"2024-02-14T15:01:44.601458+0000","last_fullsized":"2024-02-14T15:01:44.601458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:46.886154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603426+0000","last_change":"2024-02-14T15:00:48.274305+0000","last_active":"2024-02-14T15:01:44.603426+0000","last_peered":"2024-02-14T15:01:44.603426+0000","last_clean":"2024-02-14T15:01:44.603426+0000","last_became_active":"2024-02-14T15:00:48.273966+0000","last_became_peered":"2024-02-14T15:00:48.273966+0000","last_unstale":"2024-02-14T15:01:44.603426+0000","last_undegraded":"2024-02-14T15:01:44.603426+0000","last_fullsized":"2024-02-14T15:01:44.603426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:49.198099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604582+0000","last_change":"2024-02-14T15:00:49.278473+0000","last_active":"2024-02-14T15:01:44.604582+0000","last_peered":"2024-02-14T15:01:44.604582+0000","last_clean":"2024-02-14T15:01:44.604582+0000","last_became_active":"2024-02-14T15:00:49.278145+0000","last_became_peered":"2024-02-14T15:00:49.278145+0000","last_unstale":"2024-02-14T15:01:44.604582+0000","last_undegraded":"2024-02-14T15:01:44.604582+0000","last_fullsized":"2024-02-14T15:01:44.604582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:49:11.718035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602337+0000","last_change":"2024-02-14T15:00:53.581164+0000","last_active":"2024-02-14T15:01:44.602337+0000","last_peered":"2024-02-14T15:01:44.602337+0000","last_clean":"2024-02-14T15:01:44.602337+0000","last_became_active":"2024-02-14T15:00:53.580796+0000","last_became_peered":"2024-02-14T15:00:53.580796+0000","last_unstale":"2024-02-14T15:01:44.602337+0000","last_undegraded":"2024-02-14T15:01:44.602337+0000","last_fullsized":"2024-02-14T15:01:44.602337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:37.245035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599775+0000","last_change":"2024-02-14T15:00:48.271427+0000","last_active":"2024-02-14T15:01:44.599775+0000","last_peered":"2024-02-14T15:01:44.599775+0000","last_clean":"2024-02-14T15:01:44.599775+0000","last_became_active":"2024-02-14T15:00:48.271115+0000","last_became_peered":"2024-02-14T15:00:48.271115+0000","last_unstale":"2024-02-14T15:01:44.599775+0000","last_undegraded":"2024-02-14T15:01:44.599775+0000","last_fullsized":"2024-02-14T15:01:44.599775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:56:01.008058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601171+0000","last_change":"2024-02-14T15:00:49.269964+0000","last_active":"2024-02-14T15:01:44.601171+0000","last_peered":"2024-02-14T15:01:44.601171+0000","last_clean":"2024-02-14T15:01:44.601171+0000","last_became_active":"2024-02-14T15:00:49.269628+0000","last_became_peered":"2024-02-14T15:00:49.269628+0000","last_unstale":"2024-02-14T15:01:44.601171+0000","last_undegraded":"2024-02-14T15:01:44.601171+0000","last_fullsized":"2024-02-14T15:01:44.601171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:04:26.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605769+0000","last_change":"2024-02-14T15:00:53.579843+0000","last_active":"2024-02-14T15:01:44.605769+0000","last_peered":"2024-02-14T15:01:44.605769+0000","last_clean":"2024-02-14T15:01:44.605769+0000","last_became_active":"2024-02-14T15:00:53.579457+0000","last_became_peered":"2024-02-14T15:00:53.579457+0000","last_unstale":"2024-02-14T15:01:44.605769+0000","last_undegraded":"2024-02-14T15:01:44.605769+0000","last_fullsized":"2024-02-14T15:01:44.605769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:34:49.438317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604529+0000","last_change":"2024-02-14T15:00:48.275016+0000","last_active":"2024-02-14T15:01:44.604529+0000","last_peered":"2024-02-14T15:01:44.604529+0000","last_clean":"2024-02-14T15:01:44.604529+0000","last_became_active":"2024-02-14T15:00:48.274636+0000","last_became_peered":"2024-02-14T15:00:48.274636+0000","last_unstale":"2024-02-14T15:01:44.604529+0000","last_undegraded":"2024-02-14T15:01:44.604529+0000","last_fullsized":"2024-02-14T15:01:44.604529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:12:12.961276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602904+0000","last_change":"2024-02-14T15:01:44.602904+0000","last_active":"2024-02-14T15:01:44.602904+0000","last_peered":"2024-02-14T15:01:44.602904+0000","last_clean":"2024-02-14T15:01:44.602904+0000","last_became_active":"2024-02-14T15:01:44.602480+0000","last_became_peered":"2024-02-14T15:01:44.602480+0000","last_unstale":"2024-02-14T15:01:44.602904+0000","last_undegraded":"2024-02-14T15:01:44.602904+0000","last_fullsized":"2024-02-14T15:01:44.602904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:27:33.451581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605376+0000","last_change":"2024-02-14T15:00:53.584068+0000","last_active":"2024-02-14T15:01:44.605376+0000","last_peered":"2024-02-14T15:01:44.605376+0000","last_clean":"2024-02-14T15:01:44.605376+0000","last_became_active":"2024-02-14T15:00:53.583263+0000","last_became_peered":"2024-02-14T15:00:53.583263+0000","last_unstale":"2024-02-14T15:01:44.605376+0000","last_undegraded":"2024-02-14T15:01:44.605376+0000","last_fullsized":"2024-02-14T15:01:44.605376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:15.726442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603928+0000","last_change":"2024-02-14T15:00:49.271265+0000","last_active":"2024-02-14T15:01:44.603928+0000","last_peered":"2024-02-14T15:01:44.603928+0000","last_clean":"2024-02-14T15:01:44.603928+0000","last_became_active":"2024-02-14T15:00:49.270931+0000","last_became_peered":"2024-02-14T15:00:49.270931+0000","last_unstale":"2024-02-14T15:01:44.603928+0000","last_undegraded":"2024-02-14T15:01:44.603928+0000","last_fullsized":"2024-02-14T15:01:44.603928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:15:40.570730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:02:28.665524+0000","last_change":"2024-02-14T15:00:48.286669+0000","last_active":"2024-02-14T15:02:28.665524+0000","last_peered":"2024-02-14T15:02:28.665524+0000","last_clean":"2024-02-14T15:02:28.665524+0000","last_became_active":"2024-02-14T15:00:48.286252+0000","last_became_peered":"2024-02-14T15:00:48.286252+0000","last_unstale":"2024-02-14T15:02:28.665524+0000","last_undegraded":"2024-02-14T15:02:28.665524+0000","last_fullsized":"2024-02-14T15:02:28.665524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:40.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604737+0000","last_change":"2024-02-14T15:00:53.583353+0000","last_active":"2024-02-14T15:01:44.604737+0000","last_peered":"2024-02-14T15:01:44.604737+0000","last_clean":"2024-02-14T15:01:44.604737+0000","last_became_active":"2024-02-14T15:00:53.582449+0000","last_became_peered":"2024-02-14T15:00:53.582449+0000","last_unstale":"2024-02-14T15:01:44.604737+0000","last_undegraded":"2024-02-14T15:01:44.604737+0000","last_fullsized":"2024-02-14T15:01:44.604737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:09:01.514603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602175+0000","last_change":"2024-02-14T15:00:49.285579+0000","last_active":"2024-02-14T15:01:44.602175+0000","last_peered":"2024-02-14T15:01:44.602175+0000","last_clean":"2024-02-14T15:01:44.602175+0000","last_became_active":"2024-02-14T15:00:49.283741+0000","last_became_peered":"2024-02-14T15:00:49.283741+0000","last_unstale":"2024-02-14T15:01:44.602175+0000","last_undegraded":"2024-02-14T15:01:44.602175+0000","last_fullsized":"2024-02-14T15:01:44.602175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:05.813218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598214+0000","last_change":"2024-02-14T15:00:48.264434+0000","last_active":"2024-02-14T15:01:44.598214+0000","last_peered":"2024-02-14T15:01:44.598214+0000","last_clean":"2024-02-14T15:01:44.598214+0000","last_became_active":"2024-02-14T15:00:48.264048+0000","last_became_peered":"2024-02-14T15:00:48.264048+0000","last_unstale":"2024-02-14T15:01:44.598214+0000","last_undegraded":"2024-02-14T15:01:44.598214+0000","last_fullsized":"2024-02-14T15:01:44.598214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:52:32.498128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604421+0000","last_change":"2024-02-14T14:59:36.539524+0000","last_active":"2024-02-14T15:01:44.604421+0000","last_peered":"2024-02-14T15:01:44.604421+0000","last_clean":"2024-02-14T15:01:44.604421+0000","last_became_active":"2024-02-14T14:59:36.538922+0000","last_became_peered":"2024-02-14T14:59:36.538922+0000","last_unstale":"2024-02-14T15:01:44.604421+0000","last_undegraded":"2024-02-14T15:01:44.604421+0000","last_fullsized":"2024-02-14T15:01:44.604421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_clean_scrub_stamp":"2024-02-14T14:56:09.567049+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:58.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603027+0000","last_change":"2024-02-14T15:00:53.585747+0000","last_active":"2024-02-14T15:01:44.603027+0000","last_peered":"2024-02-14T15:01:44.603027+0000","last_clean":"2024-02-14T15:01:44.603027+0000","last_became_active":"2024-02-14T15:00:53.583817+0000","last_became_peered":"2024-02-14T15:00:53.583817+0000","last_unstale":"2024-02-14T15:01:44.603027+0000","last_undegraded":"2024-02-14T15:01:44.603027+0000","last_fullsized":"2024-02-14T15:01:44.603027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:56:35.081078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605152+0000","last_change":"2024-02-14T15:00:49.272692+0000","last_active":"2024-02-14T15:01:44.605152+0000","last_peered":"2024-02-14T15:01:44.605152+0000","last_clean":"2024-02-14T15:01:44.605152+0000","last_became_active":"2024-02-14T15:00:49.272353+0000","last_became_peered":"2024-02-14T15:00:49.272353+0000","last_unstale":"2024-02-14T15:01:44.605152+0000","last_undegraded":"2024-02-14T15:01:44.605152+0000","last_fullsized":"2024-02-14T15:01:44.605152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:47.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604299+0000","last_change":"2024-02-14T15:00:48.272470+0000","last_active":"2024-02-14T15:01:44.604299+0000","last_peered":"2024-02-14T15:01:44.604299+0000","last_clean":"2024-02-14T15:01:44.604299+0000","last_became_active":"2024-02-14T15:00:48.272126+0000","last_became_peered":"2024-02-14T15:00:48.272126+0000","last_unstale":"2024-02-14T15:01:44.604299+0000","last_undegraded":"2024-02-14T15:01:44.604299+0000","last_fullsized":"2024-02-14T15:01:44.604299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:21:02.601411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600753+0000","last_change":"2024-02-14T15:00:53.585950+0000","last_active":"2024-02-14T15:01:44.600753+0000","last_peered":"2024-02-14T15:01:44.600753+0000","last_clean":"2024-02-14T15:01:44.600753+0000","last_became_active":"2024-02-14T15:00:53.585547+0000","last_became_peered":"2024-02-14T15:00:53.585547+0000","last_unstale":"2024-02-14T15:01:44.600753+0000","last_undegraded":"2024-02-14T15:01:44.600753+0000","last_fullsized":"2024-02-14T15:01:44.600753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:22.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602538+0000","last_change":"2024-02-14T15:00:48.274650+0000","last_active":"2024-02-14T15:01:44.602538+0000","last_peered":"2024-02-14T15:01:44.602538+0000","last_clean":"2024-02-14T15:01:44.602538+0000","last_became_active":"2024-02-14T15:00:48.274085+0000","last_became_peered":"2024-02-14T15:00:48.274085+0000","last_unstale":"2024-02-14T15:01:44.602538+0000","last_undegraded":"2024-02-14T15:01:44.602538+0000","last_fullsized":"2024-02-14T15:01:44.602538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:53.307596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605939+0000","last_change":"2024-02-14T15:00:49.268419+0000","last_active":"2024-02-14T15:01:44.605939+0000","last_peered":"2024-02-14T15:01:44.605939+0000","last_clean":"2024-02-14T15:01:44.605939+0000","last_became_active":"2024-02-14T15:00:49.268120+0000","last_became_peered":"2024-02-14T15:00:49.268120+0000","last_unstale":"2024-02-14T15:01:44.605939+0000","last_undegraded":"2024-02-14T15:01:44.605939+0000","last_fullsized":"2024-02-14T15:01:44.605939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:10:36.319337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601823+0000","last_change":"2024-02-14T15:00:53.586151+0000","last_active":"2024-02-14T15:01:44.601823+0000","last_peered":"2024-02-14T15:01:44.601823+0000","last_clean":"2024-02-14T15:01:44.601823+0000","last_became_active":"2024-02-14T15:00:53.585727+0000","last_became_peered":"2024-02-14T15:00:53.585727+0000","last_unstale":"2024-02-14T15:01:44.601823+0000","last_undegraded":"2024-02-14T15:01:44.601823+0000","last_fullsized":"2024-02-14T15:01:44.601823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:40:03.741063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.615028+0000","last_change":"2024-02-14T15:01:44.615028+0000","last_active":"2024-02-14T15:01:44.615028+0000","last_peered":"2024-02-14T15:01:44.615028+0000","last_clean":"2024-02-14T15:01:44.615028+0000","last_became_active":"2024-02-14T15:01:44.606281+0000","last_became_peered":"2024-02-14T15:01:44.606281+0000","last_unstale":"2024-02-14T15:01:44.615028+0000","last_undegraded":"2024-02-14T15:01:44.615028+0000","last_fullsized":"2024-02-14T15:01:44.615028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:37.706659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602519+0000","last_change":"2024-02-14T15:00:49.285726+0000","last_active":"2024-02-14T15:01:44.602519+0000","last_peered":"2024-02-14T15:01:44.602519+0000","last_clean":"2024-02-14T15:01:44.602519+0000","last_became_active":"2024-02-14T15:00:49.283728+0000","last_became_peered":"2024-02-14T15:00:49.283728+0000","last_unstale":"2024-02-14T15:01:44.602519+0000","last_undegraded":"2024-02-14T15:01:44.602519+0000","last_fullsized":"2024-02-14T15:01:44.602519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.764048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597397+0000","last_change":"2024-02-14T15:00:53.583644+0000","last_active":"2024-02-14T15:01:44.597397+0000","last_peered":"2024-02-14T15:01:44.597397+0000","last_clean":"2024-02-14T15:01:44.597397+0000","last_became_active":"2024-02-14T15:00:53.582801+0000","last_became_peered":"2024-02-14T15:00:53.582801+0000","last_unstale":"2024-02-14T15:01:44.597397+0000","last_undegraded":"2024-02-14T15:01:44.597397+0000","last_fullsized":"2024-02-14T15:01:44.597397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:41:40.622091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599344+0000","last_change":"2024-02-14T15:00:48.270231+0000","last_active":"2024-02-14T15:01:44.599344+0000","last_peered":"2024-02-14T15:01:44.599344+0000","last_clean":"2024-02-14T15:01:44.599344+0000","last_became_active":"2024-02-14T15:00:48.269880+0000","last_became_peered":"2024-02-14T15:00:48.269880+0000","last_unstale":"2024-02-14T15:01:44.599344+0000","last_undegraded":"2024-02-14T15:01:44.599344+0000","last_fullsized":"2024-02-14T15:01:44.599344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:08:47.376417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597044+0000","last_change":"2024-02-14T15:00:49.277227+0000","last_active":"2024-02-14T15:01:44.597044+0000","last_peered":"2024-02-14T15:01:44.597044+0000","last_clean":"2024-02-14T15:01:44.597044+0000","last_became_active":"2024-02-14T15:00:49.276904+0000","last_became_peered":"2024-02-14T15:00:49.276904+0000","last_unstale":"2024-02-14T15:01:44.597044+0000","last_undegraded":"2024-02-14T15:01:44.597044+0000","last_fullsized":"2024-02-14T15:01:44.597044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:01.091327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598683+0000","last_change":"2024-02-14T15:00:53.587855+0000","last_active":"2024-02-14T15:01:44.598683+0000","last_peered":"2024-02-14T15:01:44.598683+0000","last_clean":"2024-02-14T15:01:44.598683+0000","last_became_active":"2024-02-14T15:00:53.586313+0000","last_became_peered":"2024-02-14T15:00:53.586313+0000","last_unstale":"2024-02-14T15:01:44.598683+0000","last_undegraded":"2024-02-14T15:01:44.598683+0000","last_fullsized":"2024-02-14T15:01:44.598683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:36:41.279582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600029+0000","last_change":"2024-02-14T15:00:48.267385+0000","last_active":"2024-02-14T15:01:44.600029+0000","last_peered":"2024-02-14T15:01:44.600029+0000","last_clean":"2024-02-14T15:01:44.600029+0000","last_became_active":"2024-02-14T15:00:48.267036+0000","last_became_peered":"2024-02-14T15:00:48.267036+0000","last_unstale":"2024-02-14T15:01:44.600029+0000","last_undegraded":"2024-02-14T15:01:44.600029+0000","last_fullsized":"2024-02-14T15:01:44.600029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:29:24.840030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601999+0000","last_change":"2024-02-14T15:00:49.272233+0000","last_active":"2024-02-14T15:01:44.601999+0000","last_peered":"2024-02-14T15:01:44.601999+0000","last_clean":"2024-02-14T15:01:44.601999+0000","last_became_active":"2024-02-14T15:00:49.271813+0000","last_became_peered":"2024-02-14T15:00:49.271813+0000","last_unstale":"2024-02-14T15:01:44.601999+0000","last_undegraded":"2024-02-14T15:01:44.601999+0000","last_fullsized":"2024-02-14T15:01:44.601999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:30:32.521443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599873+0000","last_change":"2024-02-14T15:00:53.583385+0000","last_active":"2024-02-14T15:01:44.599873+0000","last_peered":"2024-02-14T15:01:44.599873+0000","last_clean":"2024-02-14T15:01:44.599873+0000","last_became_active":"2024-02-14T15:00:53.583023+0000","last_became_peered":"2024-02-14T15:00:53.583023+0000","last_unstale":"2024-02-14T15:01:44.599873+0000","last_undegraded":"2024-02-14T15:01:44.599873+0000","last_fullsized":"2024-02-14T15:01:44.599873+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:40:53.500939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601855+0000","last_change":"2024-02-14T15:00:48.261376+0000","last_active":"2024-02-14T15:01:44.601855+0000","last_peered":"2024-02-14T15:01:44.601855+0000","last_clean":"2024-02-14T15:01:44.601855+0000","last_became_active":"2024-02-14T15:00:48.260461+0000","last_became_peered":"2024-02-14T15:00:48.260461+0000","last_unstale":"2024-02-14T15:01:44.601855+0000","last_undegraded":"2024-02-14T15:01:44.601855+0000","last_fullsized":"2024-02-14T15:01:44.601855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:35.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599874+0000","last_change":"2024-02-14T15:00:49.273117+0000","last_active":"2024-02-14T15:01:44.599874+0000","last_peered":"2024-02-14T15:01:44.599874+0000","last_clean":"2024-02-14T15:01:44.599874+0000","last_became_active":"2024-02-14T15:00:49.272810+0000","last_became_peered":"2024-02-14T15:00:49.272810+0000","last_unstale":"2024-02-14T15:01:44.599874+0000","last_undegraded":"2024-02-14T15:01:44.599874+0000","last_fullsized":"2024-02-14T15:01:44.599874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:37:00.992212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599218+0000","last_change":"2024-02-14T15:00:53.583972+0000","last_active":"2024-02-14T15:01:44.599218+0000","last_peered":"2024-02-14T15:01:44.599218+0000","last_clean":"2024-02-14T15:01:44.599218+0000","last_became_active":"2024-02-14T15:00:53.583563+0000","last_became_peered":"2024-02-14T15:00:53.583563+0000","last_unstale":"2024-02-14T15:01:44.599218+0000","last_undegraded":"2024-02-14T15:01:44.599218+0000","last_fullsized":"2024-02-14T15:01:44.599218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:08:55.811082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597557+0000","last_change":"2024-02-14T15:00:48.270363+0000","last_active":"2024-02-14T15:01:44.597557+0000","last_peered":"2024-02-14T15:01:44.597557+0000","last_clean":"2024-02-14T15:01:44.597557+0000","last_became_active":"2024-02-14T15:00:48.270030+0000","last_became_peered":"2024-02-14T15:00:48.270030+0000","last_unstale":"2024-02-14T15:01:44.597557+0000","last_undegraded":"2024-02-14T15:01:44.597557+0000","last_fullsized":"2024-02-14T15:01:44.597557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:28.885451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602276+0000","last_change":"2024-02-14T15:00:49.285647+0000","last_active":"2024-02-14T15:01:44.602276+0000","last_peered":"2024-02-14T15:01:44.602276+0000","last_clean":"2024-02-14T15:01:44.602276+0000","last_became_active":"2024-02-14T15:00:49.285267+0000","last_became_peered":"2024-02-14T15:00:49.285267+0000","last_unstale":"2024-02-14T15:01:44.602276+0000","last_undegraded":"2024-02-14T15:01:44.602276+0000","last_fullsized":"2024-02-14T15:01:44.602276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:03.143177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599006+0000","last_change":"2024-02-14T15:00:53.591029+0000","last_active":"2024-02-14T15:01:44.599006+0000","last_peered":"2024-02-14T15:01:44.599006+0000","last_clean":"2024-02-14T15:01:44.599006+0000","last_became_active":"2024-02-14T15:00:53.590570+0000","last_became_peered":"2024-02-14T15:00:53.590570+0000","last_unstale":"2024-02-14T15:01:44.599006+0000","last_undegraded":"2024-02-14T15:01:44.599006+0000","last_fullsized":"2024-02-14T15:01:44.599006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:41.809227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.611339+0000","last_change":"2024-02-14T15:01:44.611339+0000","last_active":"2024-02-14T15:01:44.611339+0000","last_peered":"2024-02-14T15:01:44.611339+0000","last_clean":"2024-02-14T15:01:44.611339+0000","last_became_active":"2024-02-14T15:01:44.605631+0000","last_became_peered":"2024-02-14T15:01:44.605631+0000","last_unstale":"2024-02-14T15:01:44.611339+0000","last_undegraded":"2024-02-14T15:01:44.611339+0000","last_fullsized":"2024-02-14T15:01:44.611339+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:24:19.853345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604654+0000","last_change":"2024-02-14T15:00:49.276346+0000","last_active":"2024-02-14T15:01:44.604654+0000","last_peered":"2024-02-14T15:01:44.604654+0000","last_clean":"2024-02-14T15:01:44.604654+0000","last_became_active":"2024-02-14T15:00:49.276000+0000","last_became_peered":"2024-02-14T15:00:49.276000+0000","last_unstale":"2024-02-14T15:01:44.604654+0000","last_undegraded":"2024-02-14T15:01:44.604654+0000","last_fullsized":"2024-02-14T15:01:44.604654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:14:03.811981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599368+0000","last_change":"2024-02-14T15:00:53.587021+0000","last_active":"2024-02-14T15:01:44.599368+0000","last_peered":"2024-02-14T15:01:44.599368+0000","last_clean":"2024-02-14T15:01:44.599368+0000","last_became_active":"2024-02-14T15:00:53.586492+0000","last_became_peered":"2024-02-14T15:00:53.586492+0000","last_unstale":"2024-02-14T15:01:44.599368+0000","last_undegraded":"2024-02-14T15:01:44.599368+0000","last_fullsized":"2024-02-14T15:01:44.599368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:51:02.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.612410+0000","last_change":"2024-02-14T15:01:44.611916+0000","last_active":"2024-02-14T15:01:44.612410+0000","last_peered":"2024-02-14T15:01:44.612410+0000","last_clean":"2024-02-14T15:01:44.612410+0000","last_became_active":"2024-02-14T15:01:44.605791+0000","last_became_peered":"2024-02-14T15:01:44.605791+0000","last_unstale":"2024-02-14T15:01:44.612410+0000","last_undegraded":"2024-02-14T15:01:44.612410+0000","last_fullsized":"2024-02-14T15:01:44.612410+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:50:54.066205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605087+0000","last_change":"2024-02-14T15:00:49.276110+0000","last_active":"2024-02-14T15:01:44.605087+0000","last_peered":"2024-02-14T15:01:44.605087+0000","last_clean":"2024-02-14T15:01:44.605087+0000","last_became_active":"2024-02-14T15:00:49.275729+0000","last_became_peered":"2024-02-14T15:00:49.275729+0000","last_unstale":"2024-02-14T15:01:44.605087+0000","last_undegraded":"2024-02-14T15:01:44.605087+0000","last_fullsized":"2024-02-14T15:01:44.605087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:07:29.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596658+0000","last_change":"2024-02-14T15:00:53.582061+0000","last_active":"2024-02-14T15:01:44.596658+0000","last_peered":"2024-02-14T15:01:44.596658+0000","last_clean":"2024-02-14T15:01:44.596658+0000","last_became_active":"2024-02-14T15:00:53.581659+0000","last_became_peered":"2024-02-14T15:00:53.581659+0000","last_unstale":"2024-02-14T15:01:44.596658+0000","last_undegraded":"2024-02-14T15:01:44.596658+0000","last_fullsized":"2024-02-14T15:01:44.596658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:13.814496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.603397+0000","last_change":"2024-02-14T15:00:48.283300+0000","last_active":"2024-02-14T15:01:44.603397+0000","last_peered":"2024-02-14T15:01:44.603397+0000","last_clean":"2024-02-14T15:01:44.603397+0000","last_became_active":"2024-02-14T15:00:48.282939+0000","last_became_peered":"2024-02-14T15:00:48.282939+0000","last_unstale":"2024-02-14T15:01:44.603397+0000","last_undegraded":"2024-02-14T15:01:44.603397+0000","last_fullsized":"2024-02-14T15:01:44.603397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:56:52.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.604622+0000","last_change":"2024-02-14T15:00:49.273435+0000","last_active":"2024-02-14T15:01:44.604622+0000","last_peered":"2024-02-14T15:01:44.604622+0000","last_clean":"2024-02-14T15:01:44.604622+0000","last_became_active":"2024-02-14T15:00:49.272980+0000","last_became_peered":"2024-02-14T15:00:49.272980+0000","last_unstale":"2024-02-14T15:01:44.604622+0000","last_undegraded":"2024-02-14T15:01:44.604622+0000","last_fullsized":"2024-02-14T15:01:44.604622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:19:05.294943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600573+0000","last_change":"2024-02-14T15:00:53.583509+0000","last_active":"2024-02-14T15:01:44.600573+0000","last_peered":"2024-02-14T15:01:44.600573+0000","last_clean":"2024-02-14T15:01:44.600573+0000","last_became_active":"2024-02-14T15:00:53.583148+0000","last_became_peered":"2024-02-14T15:00:53.583148+0000","last_unstale":"2024-02-14T15:01:44.600573+0000","last_undegraded":"2024-02-14T15:01:44.600573+0000","last_fullsized":"2024-02-14T15:01:44.600573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:06:47.059643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605520+0000","last_change":"2024-02-14T15:00:48.286707+0000","last_active":"2024-02-14T15:01:44.605520+0000","last_peered":"2024-02-14T15:01:44.605520+0000","last_clean":"2024-02-14T15:01:44.605520+0000","last_became_active":"2024-02-14T15:00:48.286355+0000","last_became_peered":"2024-02-14T15:00:48.286355+0000","last_unstale":"2024-02-14T15:01:44.605520+0000","last_undegraded":"2024-02-14T15:01:44.605520+0000","last_fullsized":"2024-02-14T15:01:44.605520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:48:31.024464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605460+0000","last_change":"2024-02-14T15:00:49.271598+0000","last_active":"2024-02-14T15:01:44.605460+0000","last_peered":"2024-02-14T15:01:44.605460+0000","last_clean":"2024-02-14T15:01:44.605460+0000","last_became_active":"2024-02-14T15:00:49.270572+0000","last_became_peered":"2024-02-14T15:00:49.270572+0000","last_unstale":"2024-02-14T15:01:44.605460+0000","last_undegraded":"2024-02-14T15:01:44.605460+0000","last_fullsized":"2024-02-14T15:01:44.605460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:12.666557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.599923+0000","last_change":"2024-02-14T15:00:49.276697+0000","last_active":"2024-02-14T15:01:44.599923+0000","last_peered":"2024-02-14T15:01:44.599923+0000","last_clean":"2024-02-14T15:01:44.599923+0000","last_became_active":"2024-02-14T15:00:49.276279+0000","last_became_peered":"2024-02-14T15:00:49.276279+0000","last_unstale":"2024-02-14T15:01:44.599923+0000","last_undegraded":"2024-02-14T15:01:44.599923+0000","last_fullsized":"2024-02-14T15:01:44.599923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:11:31.824974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.607605+0000","last_change":"2024-02-14T15:01:44.607605+0000","last_active":"2024-02-14T15:01:44.607605+0000","last_peered":"2024-02-14T15:01:44.607605+0000","last_clean":"2024-02-14T15:01:44.607605+0000","last_became_active":"2024-02-14T15:01:44.607171+0000","last_became_peered":"2024-02-14T15:01:44.607171+0000","last_unstale":"2024-02-14T15:01:44.607605+0000","last_undegraded":"2024-02-14T15:01:44.607605+0000","last_fullsized":"2024-02-14T15:01:44.607605+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:30:20.503814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.597556+0000","last_change":"2024-02-14T15:00:49.274219+0000","last_active":"2024-02-14T15:01:44.597556+0000","last_peered":"2024-02-14T15:01:44.597556+0000","last_clean":"2024-02-14T15:01:44.597556+0000","last_became_active":"2024-02-14T15:00:49.273833+0000","last_became_peered":"2024-02-14T15:00:49.273833+0000","last_unstale":"2024-02-14T15:01:44.597556+0000","last_undegraded":"2024-02-14T15:01:44.597556+0000","last_fullsized":"2024-02-14T15:01:44.597556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:06:02.720273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.596758+0000","last_change":"2024-02-14T15:00:49.273576+0000","last_active":"2024-02-14T15:01:44.596758+0000","last_peered":"2024-02-14T15:01:44.596758+0000","last_clean":"2024-02-14T15:01:44.596758+0000","last_became_active":"2024-02-14T15:00:49.272920+0000","last_became_peered":"2024-02-14T15:00:49.272920+0000","last_unstale":"2024-02-14T15:01:44.596758+0000","last_undegraded":"2024-02-14T15:01:44.596758+0000","last_fullsized":"2024-02-14T15:01:44.596758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:22:03.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605886+0000","last_change":"2024-02-14T15:00:49.274549+0000","last_active":"2024-02-14T15:01:44.605886+0000","last_peered":"2024-02-14T15:01:44.605886+0000","last_clean":"2024-02-14T15:01:44.605886+0000","last_became_active":"2024-02-14T15:00:49.274097+0000","last_became_peered":"2024-02-14T15:00:49.274097+0000","last_unstale":"2024-02-14T15:01:44.605886+0000","last_undegraded":"2024-02-14T15:01:44.605886+0000","last_fullsized":"2024-02-14T15:01:44.605886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:47:44.017607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600676+0000","last_change":"2024-02-14T15:00:49.275699+0000","last_active":"2024-02-14T15:01:44.600676+0000","last_peered":"2024-02-14T15:01:44.600676+0000","last_clean":"2024-02-14T15:01:44.600676+0000","last_became_active":"2024-02-14T15:00:49.275361+0000","last_became_peered":"2024-02-14T15:00:49.275361+0000","last_unstale":"2024-02-14T15:01:44.600676+0000","last_undegraded":"2024-02-14T15:01:44.600676+0000","last_fullsized":"2024-02-14T15:01:44.600676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:34:02.638758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.598315+0000","last_change":"2024-02-14T15:00:49.275024+0000","last_active":"2024-02-14T15:01:44.598315+0000","last_peered":"2024-02-14T15:01:44.598315+0000","last_clean":"2024-02-14T15:01:44.598315+0000","last_became_active":"2024-02-14T15:00:49.274710+0000","last_became_peered":"2024-02-14T15:00:49.274710+0000","last_unstale":"2024-02-14T15:01:44.598315+0000","last_undegraded":"2024-02-14T15:01:44.598315+0000","last_fullsized":"2024-02-14T15:01:44.598315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:42.471112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.602642+0000","last_change":"2024-02-14T15:00:49.282638+0000","last_active":"2024-02-14T15:01:44.602642+0000","last_peered":"2024-02-14T15:01:44.602642+0000","last_clean":"2024-02-14T15:01:44.602642+0000","last_became_active":"2024-02-14T15:00:49.280223+0000","last_became_peered":"2024-02-14T15:00:49.280223+0000","last_unstale":"2024-02-14T15:01:44.602642+0000","last_undegraded":"2024-02-14T15:01:44.602642+0000","last_fullsized":"2024-02-14T15:01:44.602642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:40.250242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600914+0000","last_change":"2024-02-14T15:00:49.273025+0000","last_active":"2024-02-14T15:01:44.600914+0000","last_peered":"2024-02-14T15:01:44.600914+0000","last_clean":"2024-02-14T15:01:44.600914+0000","last_became_active":"2024-02-14T15:00:49.272700+0000","last_became_peered":"2024-02-14T15:00:49.272700+0000","last_unstale":"2024-02-14T15:01:44.600914+0000","last_undegraded":"2024-02-14T15:01:44.600914+0000","last_fullsized":"2024-02-14T15:01:44.600914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:08.677428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.606377+0000","last_change":"2024-02-14T15:01:44.605957+0000","last_active":"2024-02-14T15:01:44.606377+0000","last_peered":"2024-02-14T15:01:44.606377+0000","last_clean":"2024-02-14T15:01:44.606377+0000","last_became_active":"2024-02-14T15:01:44.605398+0000","last_became_peered":"2024-02-14T15:01:44.605398+0000","last_unstale":"2024-02-14T15:01:44.606377+0000","last_undegraded":"2024-02-14T15:01:44.606377+0000","last_fullsized":"2024-02-14T15:01:44.606377+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:50:43.667496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.601312+0000","last_change":"2024-02-14T15:00:49.282178+0000","last_active":"2024-02-14T15:01:44.601312+0000","last_peered":"2024-02-14T15:01:44.601312+0000","last_clean":"2024-02-14T15:01:44.601312+0000","last_became_active":"2024-02-14T15:00:49.281804+0000","last_became_peered":"2024-02-14T15:00:49.281804+0000","last_unstale":"2024-02-14T15:01:44.601312+0000","last_undegraded":"2024-02-14T15:01:44.601312+0000","last_fullsized":"2024-02-14T15:01:44.601312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:58.672808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.605635+0000","last_change":"2024-02-14T15:00:49.269882+0000","last_active":"2024-02-14T15:01:44.605635+0000","last_peered":"2024-02-14T15:01:44.605635+0000","last_clean":"2024-02-14T15:01:44.605635+0000","last_became_active":"2024-02-14T15:00:49.269489+0000","last_became_peered":"2024-02-14T15:00:49.269489+0000","last_unstale":"2024-02-14T15:01:44.605635+0000","last_undegraded":"2024-02-14T15:01:44.605635+0000","last_fullsized":"2024-02-14T15:01:44.605635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:57:40.465044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-14T15:01:44.600399+0000","last_change":"2024-02-14T15:00:49.277688+0000","last_active":"2024-02-14T15:01:44.600399+0000","last_peered":"2024-02-14T15:01:44.600399+0000","last_clean":"2024-02-14T15:01:44.600399+0000","last_became_active":"2024-02-14T15:00:49.277348+0000","last_became_peered":"2024-02-14T15:00:49.277348+0000","last_unstale":"2024-02-14T15:01:44.600399+0000","last_undegraded":"2024-02-14T15:01:44.600399+0000","last_fullsized":"2024-02-14T15:01:44.600399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:33.946319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67357,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":466,"num_read_kb":302,"num_write":161,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":63938,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":499976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.61499999999999999,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.49099999999999999}]},{"osd":1,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.56699999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91200000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.53700000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83699999999999997,"5min":1.155,"15min":1.155},"last":0.627}]},{"osd":5,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80300000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77300000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.504,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91300000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78600000000000003,"15min":0.79100000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.97599999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":1.121,"15min":1.121},"last":0.81000000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75800000000000001,"15min":0.75600000000000001},"min":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97999999999999998,"5min":1.002,"15min":1.002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77600000000000002,"15min":0.77500000000000002},"min":{"1min":0.57699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.88400000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.92200000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":0.81499999999999995},"min":{"1min":0.65600000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.86799999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1150991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.72999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82299999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82699999999999996,"5min":0.876,"15min":0.876},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.69199999999999995,"15min":0.69999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81699999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.82399999999999995}]},{"osd":4,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":1.054,"15min":1.054},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.629,"15min":0.65200000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.61299999999999999,"5min":1.145,"15min":1.145},"last":0.78700000000000003}]},{"osd":5,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73399999999999999,"15min":0.74199999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88400000000000001,"5min":1.032,"15min":1.032},"last":0.76100000000000001}]},{"osd":6,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.50600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95799999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.626,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.90300000000000002,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94299999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":0.85299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95299999999999996,"5min":1.129,"15min":1.129},"last":0.872},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.83999999999999997,"15min":0.85899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.872,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84699999999999998}]},{"osd":9,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.86599999999999999,"15min":0.874},"min":{"1min":0.67800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.169},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.85799999999999998,"15min":0.88500000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.91200000000000003,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.88700000000000001}]},{"osd":11,"last update":"Wed Feb 14 15:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86899999999999999,"15min":0.875},"min":{"1min":0.63900000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.032,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.90600000000000003},"min":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0109999999999999,"5min":1.141,"15min":1.141},"last":0.96499999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":504308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44900000000000001,"15min":0.46200000000000002},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":1.22,"15min":1.22},"last":0.30599999999999999}]},{"osd":1,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.443}]},{"osd":3,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.54200000000000004,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0629999999999999,"5min":1.141,"15min":1.141},"last":0.59699999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.69999999999999996,"15min":0.73699999999999999},"min":{"1min":0.45600000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97799999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.90600000000000003,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4370000000000001,"5min":19.152000000000001,"15min":19.152000000000001},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.31},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.94199999999999995,"15min":0.86299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":18.734999999999999,"15min":18.734999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.93999999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5069999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.60599999999999998}]},{"osd":8,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.94799999999999995,"15min":0.85999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.165,"5min":19.135999999999999,"15min":19.135999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0049999999999999,"5min":1.23,"15min":1.23},"last":0.54500000000000004}]},{"osd":10,"last update":"Wed Feb 14 15:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.79500000000000004,"15min":0.80400000000000005},"min":{"1min":0.58299999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.82799999999999996,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0369999999999999,"5min":1.264,"15min":1.264},"last":0.69299999999999995}]},{"osd":11,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.627},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.52900000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":500132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.3400000000000001,"5min":0.68500000000000005,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":13.997,"5min":13.997,"15min":13.997},"last":0.42499999999999999}]},{"osd":1,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83499999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51300000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.371},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.55200000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.48799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.628,"15min":0.622},"min":{"1min":0.436,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.40799999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78200000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":1.046,"15min":1.212},"min":{"1min":0.55900000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":13.618,"15min":13.618},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.69099999999999995,"15min":0.67800000000000005},"min":{"1min":0.59499999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.75},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.58499999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76000000000000001}]},{"osd":9,"last update":"Wed Feb 14 15:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.59599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.68400000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80100000000000005}]},{"osd":10,"last update":"Wed Feb 14 15:01:37 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.59699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.70899999999999996,"15min":0.69099999999999995},"min":{"1min":0.60499999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73799999999999999}]},{"osd":11,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78800000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1096041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71599999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.435},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":0.877,"15min":0.877},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.68899999999999995,"15min":0.65000000000000002},"min":{"1min":0.52900000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83199999999999996,"5min":0.872,"15min":0.872},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.58199999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91000000000000003,"5min":0.997,"15min":0.997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85499999999999998,"5min":1.095,"15min":1.095},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.78400000000000003,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71999999999999997}]},{"osd":8,"last update":"Wed Feb 14 15:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.59299999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97799999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80300000000000005,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.88900000000000001}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.052,"15min":1.052},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.67300000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.0339999999999998,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":0.74099999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.05,"5min":1.179,"15min":1.179},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82599999999999996,"15min":0.82699999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.82699999999999996}]},{"osd":11,"last update":"Wed Feb 14 15:01:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84099999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":495388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.37},"min":{"1min":0.32500000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.39300000000000002},"min":{"1min":0.34200000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.44900000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.75},{"interface":"front","average":{"1min":1.464,"5min":0.747,"15min":0.60999999999999999},"min":{"1min":0.45700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":19.181000000000001,"5min":19.181000000000001,"15min":19.181000000000001},"last":0.85299999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61099999999999999,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.442,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45800000000000002,"15min":0.44400000000000001},"min":{"1min":0.372,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57899999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.37},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.379,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80200000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":0.76700000000000002,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":19.167000000000002,"5min":19.167000000000002,"15min":19.167000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.629}]},{"osd":5,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.875,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64900000000000002,"15min":0.63200000000000001},"min":{"1min":0.47499999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.56499999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.99199999999999999,"5min":1.107,"15min":1.107},"last":0.71499999999999997}]},{"osd":8,"last update":"Wed Feb 14 15:01:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.55300000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84699999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92200000000000004,"5min":1.266,"15min":1.266},"last":0.81999999999999995}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68899999999999995,"15min":0.72499999999999998},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.92300000000000004,"5min":0.996,"15min":0.996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.58399999999999996}]},{"osd":10,"last update":"Wed Feb 14 15:01:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.008,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94999999999999996,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.91000000000000003}]},{"osd":11,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.872},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":559236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.32800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.56599999999999995,"15min":0.51300000000000001},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.309},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72499999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.53600000000000003},"min":{"1min":0.47099999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.80800000000000005,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":18.332000000000001,"15min":18.332000000000001},"last":0.81899999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.871,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.53400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.63600000000000001,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.66100000000000003,"15min":0.60499999999999998},"min":{"1min":0.56399999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0089999999999999,"5min":1.016,"15min":1.016},"last":0.86599999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59999999999999998,"15min":0.57799999999999996},"min":{"1min":0.42699999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.69399999999999995,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.84199999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65500000000000003,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0209999999999999,"5min":1.077,"15min":1.077},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.753}]},{"osd":8,"last update":"Wed Feb 14 15:01:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.51700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.96699999999999997,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.73499999999999999},"min":{"1min":0.60499999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.93500000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.0069999999999999,"5min":1.044,"15min":1.044},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67000000000000004}]},{"osd":10,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84199999999999997,"15min":0.84099999999999997},"min":{"1min":0.61399999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98699999999999999,"5min":1.107,"15min":1.107},"last":0.999}]},{"osd":11,"last update":"Wed Feb 14 15:01:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.016},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.0409999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":496776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.51000000000000001},"min":{"1min":0.40899999999999997,"5min":0.25900000000000001,"15min":0.253},"max":{"1min":0.93200000000000005,"5min":1.002,"15min":1.002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54300000000000004,"15min":0.51100000000000001},"min":{"1min":0.40000000000000002,"5min":0.30599999999999999,"15min":0.28199999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.53000000000000003},"min":{"1min":0.48999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.58699999999999997,"15min":0.52500000000000002},"min":{"1min":0.47399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.61399999999999999}]},{"osd":5,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83399999999999996}]},{"osd":6,"last update":"Wed Feb 14 15:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.34499999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81299999999999994,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.123,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.5,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.55300000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79900000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.69299999999999995},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.109,"5min":1.153,"15min":1.153},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0649999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.48799999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0880000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.53900000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.873}]},{"osd":11,"last update":"Wed Feb 14 15:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.751,"15min":0.748},"min":{"1min":0.59499999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0980000000000001,"5min":1.802,"15min":1.802},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.73999999999999999,"15min":0.72599999999999998},"min":{"1min":0.67600000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1659999999999999,"5min":1.242,"15min":1.242},"last":0.89600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1032192,"data_stored":496066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77400000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.443},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58099999999999996,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.46200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79500000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.751},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68100000000000005}]},{"osd":5,"last update":"Wed Feb 14 15:01:34 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.32600000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.495},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.441,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Feb 14 15:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89600000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.46600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76900000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.79500000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.46700000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93000000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.78000000000000003}]},{"osd":9,"last update":"Wed Feb 14 15:02:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.498,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":1.01,"15min":1.01},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Wed Feb 14 15:01:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.627,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.875},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.66900000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.86499999999999999}]},{"osd":11,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.84499999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":495582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.35599999999999998,"15min":0.32100000000000001},"min":{"1min":0.26900000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.57599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.443}]},{"osd":1,"last update":"Wed Feb 14 15:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.747,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.224,"5min":19.896999999999998,"15min":19.896999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.753,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.874,"5min":19.911999999999999,"15min":19.911999999999999},"last":0.99399999999999999}]},{"osd":2,"last update":"Wed Feb 14 15:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60099999999999998,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2729999999999999,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.39700000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.45600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93999999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":1.0309999999999999,"15min":1.4670000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":19.879999999999999,"15min":19.879999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.45100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78600000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.48699999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.48499999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.2090000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70499999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64300000000000002,"15min":0.67200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.64500000000000002,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":1.0249999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.754},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.66600000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.86199999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1101824,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.42199999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72299999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41899999999999998}]},{"osd":2,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.56100000000000005,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.91900000000000004,"5min":0.997,"15min":0.997},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89500000000000002,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89100000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.45000000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:01:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61299999999999999,"15min":0.57099999999999995},"min":{"1min":0.50900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.874,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.64700000000000002,"15min":0.60999999999999999},"min":{"1min":0.48599999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89300000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.503,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72499999999999998,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92600000000000005,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.012,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67200000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.60399999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98499999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.754},{"interface":"front","average":{"1min":0.876,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.006,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.57599999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":1.04,"15min":1.04},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0609999999999999,"5min":1.105,"15min":1.105},"last":0.65700000000000003}]},{"osd":10,"last update":"Wed Feb 14 15:01:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96499999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.79900000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.77400000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1638400,"data_stored":1091238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.47399999999999998},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77400000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.82999999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.753},{"interface":"front","average":{"1min":0.62,"5min":0.54900000000000004,"15min":0.52000000000000002},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.78300000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.35699999999999998},"min":{"1min":0.32800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.38900000000000001},"min":{"1min":0.29099999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.36599999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89900000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed Feb 14 15:01:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.877,"5min":0.878,"15min":0.878},"last":0.76800000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58399999999999996,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84099999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.57599999999999996},"min":{"1min":0.45200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.626}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89200000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66000000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.42499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93000000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57999999999999996,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64300000000000002}]},{"osd":9,"last update":"Wed Feb 14 15:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95799999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.038,"15min":1.038},"last":0.73999999999999999}]},{"osd":10,"last update":"Wed Feb 14 15:01:36 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54300000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0229999999999999,"5min":1.032,"15min":1.032},"last":0.80300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T15:02:31.461 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-14T15:02:31.461 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-14T15:02:31.461 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-14T15:02:31.461 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph health --format=json 2024-02-14T15:02:31.715 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:32.816 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:32 smithi017 ceph-mon[32549]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:32.816 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:32 smithi017 ceph-mon[32549]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:02:32.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:32 smithi050 ceph-mon[36512]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:32.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:32 smithi050 ceph-mon[36512]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:02:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:32 smithi076 ceph-mon[37057]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:32 smithi076 ceph-mon[37057]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 -- 172.21.15.17:0/1498415574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 msgr2=0x7ff21c105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 --2- 172.21.15.17:0/1498415574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c105820 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff210009990 tx=0x7ff210009ca0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 -- 172.21.15.17:0/1498415574 shutdown_connections 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 --2- 172.21.15.17:0/1498415574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c105f00 0x7ff21c10dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 --2- 172.21.15.17:0/1498415574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff21c100eb0 0x7ff21c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 --2- 172.21.15.17:0/1498415574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c105820 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 -- 172.21.15.17:0/1498415574 >> 172.21.15.17:0/1498415574 conn(0x7ff21c076c70 msgr2=0x7ff21c077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 -- 172.21.15.17:0/1498415574 shutdown_connections 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.497+0000 7ff222062700 1 -- 172.21.15.17:0/1498415574 wait complete. 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 Processor -- start 2024-02-14T15:02:33.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 -- start start 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff21c100eb0 0x7ff21c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c105f00 0x7ff21c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c19d170 con 0x7ff21c101860 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c19d2e0 con 0x7ff21c105f00 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff222062700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c19d4a0 con 0x7ff21c100eb0 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff21affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff21affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c1987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46870/0 (socket says 172.21.15.17:46870) 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff21affd700 1 -- 172.21.15.17:0/4286057698 learned_addr learned my addr 172.21.15.17:0/4286057698 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff21b7fe700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff21c100eb0 0x7ff21c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.498+0000 7ff21bfff700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c105f00 0x7ff21c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff21affd700 1 -- 172.21.15.17:0/4286057698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff21c100eb0 msgr2=0x7ff21c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff21affd700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff21c100eb0 0x7ff21c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff21affd700 1 -- 172.21.15.17:0/4286057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c105f00 msgr2=0x7ff21c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:33.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff21affd700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c105f00 0x7ff21c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff21affd700 1 -- 172.21.15.17:0/4286057698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2100095f0 con 0x7ff21c101860 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff21bfff700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c105f00 0x7ff21c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff21affd700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c1987d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff2100095c0 tx=0x7ff2100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff218ff9700 1 -- 172.21.15.17:0/4286057698 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21001d070 con 0x7ff21c101860 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff218ff9700 1 -- 172.21.15.17:0/4286057698 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff210003680 con 0x7ff21c101860 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c19d720 con 0x7ff21c101860 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff218ff9700 1 -- 172.21.15.17:0/4286057698 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2100173f0 con 0x7ff21c101860 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.499+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c1a4fe0 con 0x7ff21c101860 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.500+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c10a230 con 0x7ff21c101860 2024-02-14T15:02:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.501+0000 7ff218ff9700 1 -- 172.21.15.17:0/4286057698 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff21000fb10 con 0x7ff21c101860 2024-02-14T15:02:33.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.502+0000 7ff218ff9700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff2040a21e0 0x7ff2040a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:33.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.502+0000 7ff21b7fe700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff2040a21e0 0x7ff2040a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:33.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.502+0000 7ff218ff9700 1 -- 172.21.15.17:0/4286057698 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff2100cf2e0 con 0x7ff21c101860 2024-02-14T15:02:33.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.502+0000 7ff21b7fe700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff2040a21e0 0x7ff2040a46a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005950 tx=0x7ff20c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:33.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.506+0000 7ff218ff9700 1 -- 172.21.15.17:0/4286057698 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2100980a0 con 0x7ff21c101860 2024-02-14T15:02:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.756+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff21c102300 con 0x7ff21c101860 2024-02-14T15:02:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.757+0000 7ff218ff9700 1 -- 172.21.15.17:0/4286057698 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff21009b760 con 0x7ff21c101860 2024-02-14T15:02:33.759 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:33.759 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-14T15:02:33.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff2040a21e0 msgr2=0x7ff2040a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff2040a21e0 0x7ff2040a46a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005950 tx=0x7ff20c00b410 comp rx=0 tx=0).stop 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 msgr2=0x7ff21c1987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c1987d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff2100095c0 tx=0x7ff2100047c0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 shutdown_connections 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff2040a21e0 0x7ff2040a46a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff21c105f00 0x7ff21c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff21c100eb0 0x7ff21c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 --2- 172.21.15.17:0/4286057698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c101860 0x7ff21c1987d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 >> 172.21.15.17:0/4286057698 conn(0x7ff21c076c70 msgr2=0x7ff21c06a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 shutdown_connections 2024-02-14T15:02:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:33.759+0000 7ff222062700 1 -- 172.21.15.17:0/4286057698 wait complete. 2024-02-14T15:02:34.537 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-14T15:02:34.538 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:02:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 -- 172.21.15.17:0/403822663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0101d10 msgr2=0x7ffbf01085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/403822663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0101d10 0x7ffbf01085a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009900 tx=0x7ffbe802bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 -- 172.21.15.17:0/403822663 shutdown_connections 2024-02-14T15:02:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/403822663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0101d10 0x7ffbf01085a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/403822663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbf0101350 0x7ffbf01017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/403822663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf0100160 0x7ffbf0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 -- 172.21.15.17:0/403822663 >> 172.21.15.17:0/403822663 conn(0x7ffbf00fb6f0 msgr2=0x7ffbf00fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 -- 172.21.15.17:0/403822663 shutdown_connections 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.647+0000 7ffbf7a15640 1 -- 172.21.15.17:0/403822663 wait complete. 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 Processor -- start 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 -- start start 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0100160 0x7ffbf01971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbf0101350 0x7ffbf0197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf0101d10 0x7ffbf019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf019ece0 con 0x7ffbf0100160 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf019ee50 con 0x7ffbf0101350 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf7a15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf019f010 con 0x7ffbf0101d10 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0100160 0x7ffbf01971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0100160 0x7ffbf01971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46892/0 (socket says 172.21.15.17:46892) 2024-02-14T15:02:34.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 -- 172.21.15.17:0/2354788989 learned_addr learned my addr 172.21.15.17:0/2354788989 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf5fb2640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf0101d10 0x7ffbf019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf4fb0640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbf0101350 0x7ffbf0197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 -- 172.21.15.17:0/2354788989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf0101d10 msgr2=0x7ffbf019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf0101d10 0x7ffbf019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 -- 172.21.15.17:0/2354788989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbf0101350 msgr2=0x7ffbf0197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbf0101350 0x7ffbf0197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.648+0000 7ffbf57b1640 1 -- 172.21.15.17:0/2354788989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe8009560 con 0x7ffbf0100160 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.649+0000 7ffbf57b1640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0100160 0x7ffbf01971f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000eaa0 tx=0x7ffbe000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.649+0000 7ffbee7fc640 1 -- 172.21.15.17:0/2354788989 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000ce10 con 0x7ffbf0100160 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.649+0000 7ffbee7fc640 1 -- 172.21.15.17:0/2354788989 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbe0003e50 con 0x7ffbf0100160 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.649+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbf019f2f0 con 0x7ffbf0100160 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.649+0000 7ffbee7fc640 1 -- 172.21.15.17:0/2354788989 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00116b0 con 0x7ffbf0100160 2024-02-14T15:02:34.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.649+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf019f800 con 0x7ffbf0100160 2024-02-14T15:02:34.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.650+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbf010bb70 con 0x7ffbf0100160 2024-02-14T15:02:34.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.651+0000 7ffbee7fc640 1 -- 172.21.15.17:0/2354788989 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ffbe00042c0 con 0x7ffbf0100160 2024-02-14T15:02:34.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.652+0000 7ffbee7fc640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffbd80a21d0 0x7ffbd80a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.652+0000 7ffbf4fb0640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffbd80a21d0 0x7ffbd80a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.652+0000 7ffbee7fc640 1 -- 172.21.15.17:0/2354788989 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ffbe00ce4a0 con 0x7ffbf0100160 2024-02-14T15:02:34.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.652+0000 7ffbf4fb0640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffbd80a21d0 0x7ffbd80a4690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4005e10 tx=0x7ffbe4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:34.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.656+0000 7ffbee7fc640 1 -- 172.21.15.17:0/2354788989 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffbe0097010 con 0x7ffbf0100160 2024-02-14T15:02:34.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:34 smithi017 ceph-mon[32549]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:34.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:34 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4286057698' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T15:02:34.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.839+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbf010bd50 con 0x7ffbf0100160 2024-02-14T15:02:34.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.841+0000 7ffbee7fc640 1 -- 172.21.15.17:0/2354788989 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24451 (secure 0 0 0) 0x7ffbe009a6d0 con 0x7ffbf0100160 2024-02-14T15:02:34.843 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:34.843 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":91,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:01:44.589457+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffbd80a21d0 msgr2=0x7ffbd80a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffbd80a21d0 0x7ffbd80a4690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4005e10 tx=0x7ffbe4005da0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0100160 msgr2=0x7ffbf01971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0100160 0x7ffbf01971f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000eaa0 tx=0x7ffbe000ef60 comp rx=0 tx=0).stop 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 shutdown_connections 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffbd80a21d0 0x7ffbd80a4690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf0101d10 0x7ffbf019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbf0101350 0x7ffbf0197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 --2- 172.21.15.17:0/2354788989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbf0100160 0x7ffbf01971f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 >> 172.21.15.17:0/2354788989 conn(0x7ffbf00fb6f0 msgr2=0x7ffbf0069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:34.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 shutdown_connections 2024-02-14T15:02:34.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.843+0000 7ffbf7a15640 1 -- 172.21.15.17:0/2354788989 wait complete. 2024-02-14T15:02:34.856 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:02:34.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:02:34.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:34 smithi050 ceph-mon[36512]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:34.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:34 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4286057698' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/1490911844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 msgr2=0x7f7a04100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/1490911844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04100570 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009900 tx=0x7f79f802bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/1490911844 shutdown_connections 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/1490911844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a04101d00 0x7f7a04108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/1490911844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a04101340 0x7f7a041017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/1490911844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04100570 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/1490911844 >> 172.21.15.17:0/1490911844 conn(0x7f7a040fb6c0 msgr2=0x7f7a040fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/1490911844 shutdown_connections 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/1490911844 wait complete. 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.961+0000 7f7a0a1d0640 1 Processor -- start 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a0a1d0640 1 -- start start 2024-02-14T15:02:34.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a0a1d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a0a1d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a04101340 0x7f7a041978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a0a1d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a04101d00 0x7f7a0419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a0a1d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0419ee60 con 0x7f7a04100150 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a0a1d0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0419efd0 con 0x7f7a04101340 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a0a1d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0419f190 con 0x7f7a04101d00 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46916/0 (socket says 172.21.15.17:46916) 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 -- 172.21.15.17:0/2201794861 learned_addr learned my addr 172.21.15.17:0/2201794861 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a02ffd640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a04101340 0x7f7a041978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a03fff640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a04101d00 0x7f7a0419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 -- 172.21.15.17:0/2201794861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a04101d00 msgr2=0x7f7a0419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a04101d00 0x7f7a0419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 -- 172.21.15.17:0/2201794861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a04101340 msgr2=0x7f7a041978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:34.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a04101340 0x7f7a041978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 -- 172.21.15.17:0/2201794861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f8009560 con 0x7f7a04100150 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a02ffd640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a04101340 0x7f7a041978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a03fff640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a04101d00 0x7f7a0419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a037fe640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04197370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79f8036040 tx=0x7f79f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.962+0000 7f7a00ff9640 1 -- 172.21.15.17:0/2201794861 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80044e0 con 0x7f7a04100150 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.963+0000 7f7a00ff9640 1 -- 172.21.15.17:0/2201794861 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79f80054e0 con 0x7f7a04100150 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.963+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0419f410 con 0x7f7a04100150 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.963+0000 7f7a00ff9640 1 -- 172.21.15.17:0/2201794861 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f803e450 con 0x7f7a04100150 2024-02-14T15:02:34.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.963+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0419f920 con 0x7f7a04100150 2024-02-14T15:02:34.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.964+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a041074e0 con 0x7f7a04100150 2024-02-14T15:02:34.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.965+0000 7f7a00ff9640 1 -- 172.21.15.17:0/2201794861 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f79f8003710 con 0x7f7a04100150 2024-02-14T15:02:34.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.965+0000 7f7a00ff9640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79ec0a2150 0x7f79ec0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:34.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.965+0000 7f7a02ffd640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79ec0a2150 0x7f79ec0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:34.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.965+0000 7f7a00ff9640 1 -- 172.21.15.17:0/2201794861 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f79f80f0ae0 con 0x7f7a04100150 2024-02-14T15:02:34.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.966+0000 7f7a02ffd640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79ec0a2150 0x7f79ec0a4610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a04198880 tx=0x7f79f4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:34.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:34.969+0000 7f7a00ff9640 1 -- 172.21.15.17:0/2201794861 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79f80b9760 con 0x7f7a04100150 2024-02-14T15:02:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:34 smithi076 ceph-mon[37057]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:34 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4286057698' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T15:02:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.150+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a0410cc60 con 0x7f7a04100150 2024-02-14T15:02:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.151+0000 7f7a00ff9640 1 -- 172.21.15.17:0/2201794861 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79f80bce20 con 0x7f7a04100150 2024-02-14T15:02:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:02:35.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.152+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79ec0a2150 msgr2=0x7f79ec0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.152+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79ec0a2150 0x7f79ec0a4610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a04198880 tx=0x7f79f4005ec0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.152+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 msgr2=0x7f7a04197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.152+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04197370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79f8036040 tx=0x7f79f80042f0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.152+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 shutdown_connections 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.153+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f79ec0a2150 0x7f79ec0a4610 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.153+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a04101d00 0x7f7a0419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.153+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a04101340 0x7f7a041978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.153+0000 7f7a0a1d0640 1 --2- 172.21.15.17:0/2201794861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a04100150 0x7f7a04197370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.153+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 >> 172.21.15.17:0/2201794861 conn(0x7f7a040fb6c0 msgr2=0x7f7a041046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.153+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 shutdown_connections 2024-02-14T15:02:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.153+0000 7f7a0a1d0640 1 -- 172.21.15.17:0/2201794861 wait complete. 2024-02-14T15:02:35.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:02:35.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 -- 172.21.15.17:0/3970157600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654102df0 msgr2=0x7fc654109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 --2- 172.21.15.17:0/3970157600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654102df0 0x7fc654109680 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009ca0 tx=0x7fc64c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 -- 172.21.15.17:0/3970157600 shutdown_connections 2024-02-14T15:02:35.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 --2- 172.21.15.17:0/3970157600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654102df0 0x7fc654109680 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 --2- 172.21.15.17:0/3970157600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654102430 0x7fc6541028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 --2- 172.21.15.17:0/3970157600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654101240 0x7fc654101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 -- 172.21.15.17:0/3970157600 >> 172.21.15.17:0/3970157600 conn(0x7fc6540fc7b0 msgr2=0x7fc6540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 -- 172.21.15.17:0/3970157600 shutdown_connections 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.269+0000 7fc65bc40640 1 -- 172.21.15.17:0/3970157600 wait complete. 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 Processor -- start 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 -- start start 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654101240 0x7fc654197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654102430 0x7fc6541977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654102df0 0x7fc65419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65419ed70 con 0x7fc654101240 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65419eee0 con 0x7fc654102df0 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc65bc40640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65419f0a0 con 0x7fc654102430 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc6599dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654101240 0x7fc654197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc6599dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654101240 0x7fc654197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46936/0 (socket says 172.21.15.17:46936) 2024-02-14T15:02:35.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc6599dc640 1 -- 172.21.15.17:0/1005488943 learned_addr learned my addr 172.21.15.17:0/1005488943 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.270+0000 7fc6591db640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654102430 0x7fc6541977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc65a1dd640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654102df0 0x7fc65419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc6599dc640 1 -- 172.21.15.17:0/1005488943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654102430 msgr2=0x7fc6541977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc6599dc640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654102430 0x7fc6541977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc6599dc640 1 -- 172.21.15.17:0/1005488943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654102df0 msgr2=0x7fc65419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc6599dc640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654102df0 0x7fc65419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc6599dc640 1 -- 172.21.15.17:0/1005488943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c009900 con 0x7fc654101240 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc65a1dd640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654102df0 0x7fc65419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc6599dc640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654101240 0x7fc654197280 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d950 tx=0x7fc65000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc64affd640 1 -- 172.21.15.17:0/1005488943 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650009c30 con 0x7fc654101240 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc64affd640 1 -- 172.21.15.17:0/1005488943 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc650009d90 con 0x7fc654101240 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65419f380 con 0x7fc654101240 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc64affd640 1 -- 172.21.15.17:0/1005488943 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000f720 con 0x7fc654101240 2024-02-14T15:02:35.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.271+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65419f840 con 0x7fc654101240 2024-02-14T15:02:35.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.272+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc654108500 con 0x7fc654101240 2024-02-14T15:02:35.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.273+0000 7fc64affd640 1 -- 172.21.15.17:0/1005488943 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc650010a60 con 0x7fc654101240 2024-02-14T15:02:35.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.273+0000 7fc64affd640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6400a2150 0x7fc6400a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.274+0000 7fc64affd640 1 -- 172.21.15.17:0/1005488943 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc6500cde40 con 0x7fc654101240 2024-02-14T15:02:35.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.274+0000 7fc6591db640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6400a2150 0x7fc6400a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.274+0000 7fc6591db640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6400a2150 0x7fc6400a4610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc654198790 tx=0x7fc64400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:35.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.277+0000 7fc64affd640 1 -- 172.21.15.17:0/1005488943 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6500969b0 con 0x7fc654101240 2024-02-14T15:02:35.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.457+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc65402ce50 con 0x7fc654101240 2024-02-14T15:02:35.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.457+0000 7fc64affd640 1 -- 172.21.15.17:0/1005488943 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc650017020 con 0x7fc654101240 2024-02-14T15:02:35.459 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:02:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6400a2150 msgr2=0x7fc6400a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6400a2150 0x7fc6400a4610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc654198790 tx=0x7fc64400a2d0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654101240 msgr2=0x7fc654197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654101240 0x7fc654197280 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d950 tx=0x7fc65000de10 comp rx=0 tx=0).stop 2024-02-14T15:02:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 shutdown_connections 2024-02-14T15:02:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc6400a2150 0x7fc6400a4610 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc654102df0 0x7fc65419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654102430 0x7fc6541977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 --2- 172.21.15.17:0/1005488943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654101240 0x7fc654197280 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 >> 172.21.15.17:0/1005488943 conn(0x7fc6540fc7b0 msgr2=0x7fc6541056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 shutdown_connections 2024-02-14T15:02:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.459+0000 7fc65bc40640 1 -- 172.21.15.17:0/1005488943 wait complete. 2024-02-14T15:02:35.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:02:35.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.576+0000 7f70bf2b5640 1 -- 172.21.15.17:0/172000593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 msgr2=0x7f70b8100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.576+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/172000593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8100570 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009900 tx=0x7f70b002bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:35.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.576+0000 7f70bf2b5640 1 -- 172.21.15.17:0/172000593 shutdown_connections 2024-02-14T15:02:35.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.576+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/172000593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8101d00 0x7f70b8108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.576+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/172000593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70b8101340 0x7f70b81017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.576+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/172000593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8100570 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.576+0000 7f70bf2b5640 1 -- 172.21.15.17:0/172000593 >> 172.21.15.17:0/172000593 conn(0x7f70b80fb6c0 msgr2=0x7f70b80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 -- 172.21.15.17:0/172000593 shutdown_connections 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 -- 172.21.15.17:0/172000593 wait complete. 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 Processor -- start 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 -- start start 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70b8101340 0x7f70b8196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8101d00 0x7f70b819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b819dcf0 con 0x7f70b8100150 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b819de60 con 0x7f70b8101340 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bf2b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b819e020 con 0x7f70b8101d00 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bd051640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bd051640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46962/0 (socket says 172.21.15.17:46962) 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bd051640 1 -- 172.21.15.17:0/1960711137 learned_addr learned my addr 172.21.15.17:0/1960711137 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.577+0000 7f70bd852640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8101d00 0x7f70b819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bd051640 1 -- 172.21.15.17:0/1960711137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8101d00 msgr2=0x7f70b819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bd051640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8101d00 0x7f70b819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bc850640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70b8101340 0x7f70b8196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bd051640 1 -- 172.21.15.17:0/1960711137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70b8101340 msgr2=0x7f70b8196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bd051640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70b8101340 0x7f70b8196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bd051640 1 -- 172.21.15.17:0/1960711137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0009560 con 0x7f70b8100150 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bd051640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8196200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70b0000c00 tx=0x7f70b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70ae7fc640 1 -- 172.21.15.17:0/1960711137 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00044e0 con 0x7f70b8100150 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b819e2a0 con 0x7f70b8100150 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70ae7fc640 1 -- 172.21.15.17:0/1960711137 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70b00054e0 con 0x7f70b8100150 2024-02-14T15:02:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70ae7fc640 1 -- 172.21.15.17:0/1960711137 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b003e450 con 0x7f70b8100150 2024-02-14T15:02:35.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.578+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b819e800 con 0x7f70b8100150 2024-02-14T15:02:35.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.579+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b81074f0 con 0x7f70b8100150 2024-02-14T15:02:35.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.580+0000 7f70ae7fc640 1 -- 172.21.15.17:0/1960711137 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f70b00035f0 con 0x7f70b8100150 2024-02-14T15:02:35.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.581+0000 7f70ae7fc640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f70a40a2150 0x7f70a40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.581+0000 7f70bc850640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f70a40a2150 0x7f70a40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.581+0000 7f70ae7fc640 1 -- 172.21.15.17:0/1960711137 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f70b00f0e50 con 0x7f70b8100150 2024-02-14T15:02:35.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.581+0000 7f70bc850640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f70a40a2150 0x7f70a40a4610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70b8197710 tx=0x7f70a8005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:35.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.585+0000 7f70ae7fc640 1 -- 172.21.15.17:0/1960711137 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70b00b9a70 con 0x7f70b8100150 2024-02-14T15:02:35.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:35 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2354788989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:35.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:35 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2201794861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:02:35.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:35 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1005488943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:02:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.770+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70b819e430 con 0x7f70b8100150 2024-02-14T15:02:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.770+0000 7f70ae7fc640 1 -- 172.21.15.17:0/1960711137 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70b819e430 con 0x7f70b8100150 2024-02-14T15:02:35.772 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:02:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f70a40a2150 msgr2=0x7f70a40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f70a40a2150 0x7f70a40a4610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70b8197710 tx=0x7f70a8005c60 comp rx=0 tx=0).stop 2024-02-14T15:02:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 msgr2=0x7f70b8196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8196200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70b0000c00 tx=0x7f70b00042f0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 shutdown_connections 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f70a40a2150 0x7f70a40a4610 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8101d00 0x7f70b819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70b8101340 0x7f70b8196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 --2- 172.21.15.17:0/1960711137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b8100150 0x7f70b8196200 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 >> 172.21.15.17:0/1960711137 conn(0x7f70b80fb6c0 msgr2=0x7f70b81046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 shutdown_connections 2024-02-14T15:02:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.772+0000 7f70bf2b5640 1 -- 172.21.15.17:0/1960711137 wait complete. 2024-02-14T15:02:35.785 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 -- 172.21.15.17:0/1742723971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 msgr2=0x7f1b4c108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 --2- 172.21.15.17:0/1742723971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c108590 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009ca0 tx=0x7f1b4402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 -- 172.21.15.17:0/1742723971 shutdown_connections 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 --2- 172.21.15.17:0/1742723971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c108590 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 --2- 172.21.15.17:0/1742723971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b4c101340 0x7f1b4c1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 --2- 172.21.15.17:0/1742723971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b4c100150 0x7f1b4c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 -- 172.21.15.17:0/1742723971 >> 172.21.15.17:0/1742723971 conn(0x7f1b4c0fb6c0 msgr2=0x7f1b4c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 -- 172.21.15.17:0/1742723971 shutdown_connections 2024-02-14T15:02:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 -- 172.21.15.17:0/1742723971 wait complete. 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 Processor -- start 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.889+0000 7f1b52610640 1 -- start start 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b52610640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b4c100150 0x7f1b4c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b52610640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b4c101340 0x7f1b4c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b52610640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b52610640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c19ed40 con 0x7f1b4c101d00 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b52610640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c19eeb0 con 0x7f1b4c101340 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b52610640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c19f070 con 0x7f1b4c100150 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46980/0 (socket says 172.21.15.17:46980) 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 -- 172.21.15.17:0/2299591665 learned_addr learned my addr 172.21.15.17:0/2299591665 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b4bfff640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b4c100150 0x7f1b4c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b4b7fe640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b4c101340 0x7f1b4c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 -- 172.21.15.17:0/2299591665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b4c100150 msgr2=0x7f1b4c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b4c100150 0x7f1b4c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 -- 172.21.15.17:0/2299591665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b4c101340 msgr2=0x7f1b4c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b4c101340 0x7f1b4c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 -- 172.21.15.17:0/2299591665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009900 con 0x7f1b4c101d00 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.890+0000 7f1b50bad640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c19e800 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b44037040 tx=0x7f1b44034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.891+0000 7f1b497fa640 1 -- 172.21.15.17:0/2299591665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440329b0 con 0x7f1b4c101d00 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.891+0000 7f1b497fa640 1 -- 172.21.15.17:0/2299591665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b44003e50 con 0x7f1b4c101d00 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.891+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c19f2f0 con 0x7f1b4c101d00 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.891+0000 7f1b497fa640 1 -- 172.21.15.17:0/2299591665 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44040450 con 0x7f1b4c101d00 2024-02-14T15:02:35.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.891+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c19f7f0 con 0x7f1b4c101d00 2024-02-14T15:02:35.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:35 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2354788989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:35.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:35 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2201794861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:02:35.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:35 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1005488943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:02:35.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.892+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c107470 con 0x7f1b4c101d00 2024-02-14T15:02:35.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.893+0000 7f1b497fa640 1 -- 172.21.15.17:0/2299591665 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1b4400a040 con 0x7f1b4c101d00 2024-02-14T15:02:35.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.893+0000 7f1b497fa640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1b340a21d0 0x7f1b340a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:35.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.893+0000 7f1b4bfff640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1b340a21d0 0x7f1b340a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:35.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.893+0000 7f1b497fa640 1 -- 172.21.15.17:0/2299591665 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1b440f0a20 con 0x7f1b4c101d00 2024-02-14T15:02:35.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.894+0000 7f1b4bfff640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1b340a21d0 0x7f1b340a4690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c1011a0 tx=0x7f1b3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:35.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:35.898+0000 7f1b497fa640 1 -- 172.21.15.17:0/2299591665 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b440b96a0 con 0x7f1b4c101d00 2024-02-14T15:02:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:35 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2354788989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:35 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2201794861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:02:35.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:35 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1005488943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:02:36.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.077+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1b4c02cdb0 con 0x7f1b4c101d00 2024-02-14T15:02:36.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.077+0000 7f1b497fa640 1 -- 172.21.15.17:0/2299591665 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1b440bcd60 con 0x7f1b4c101d00 2024-02-14T15:02:36.079 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:02:36.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1b340a21d0 msgr2=0x7f1b340a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1b340a21d0 0x7f1b340a4690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c1011a0 tx=0x7f1b3c005f90 comp rx=0 tx=0).stop 2024-02-14T15:02:36.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 msgr2=0x7f1b4c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c19e800 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b44037040 tx=0x7f1b44034a90 comp rx=0 tx=0).stop 2024-02-14T15:02:36.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 shutdown_connections 2024-02-14T15:02:36.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f1b340a21d0 0x7f1b340a4690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1b4c101d00 0x7f1b4c19e800 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b4c101340 0x7f1b4c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 --2- 172.21.15.17:0/2299591665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b4c100150 0x7f1b4c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 >> 172.21.15.17:0/2299591665 conn(0x7f1b4c0fb6c0 msgr2=0x7f1b4c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:36.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 shutdown_connections 2024-02-14T15:02:36.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.079+0000 7f1b52610640 1 -- 172.21.15.17:0/2299591665 wait complete. 2024-02-14T15:02:36.093 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:02:36.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.193+0000 7f0ef8084640 1 -- 172.21.15.17:0/593277471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0101280 msgr2=0x7f0ef01016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.193+0000 7f0ef8084640 1 --2- 172.21.15.17:0/593277471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0101280 0x7f0ef01016a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009900 tx=0x7f0ee802bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:36.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.193+0000 7f0ef8084640 1 -- 172.21.15.17:0/593277471 shutdown_connections 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.193+0000 7f0ef8084640 1 --2- 172.21.15.17:0/593277471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef0102e30 0x7f0ef01096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.193+0000 7f0ef8084640 1 --2- 172.21.15.17:0/593277471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef0102470 0x7f0ef01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.193+0000 7f0ef8084640 1 --2- 172.21.15.17:0/593277471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0101280 0x7f0ef01016a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.193+0000 7f0ef8084640 1 -- 172.21.15.17:0/593277471 >> 172.21.15.17:0/593277471 conn(0x7f0ef00fc7f0 msgr2=0x7f0ef00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 -- 172.21.15.17:0/593277471 shutdown_connections 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 -- 172.21.15.17:0/593277471 wait complete. 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 Processor -- start 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 -- start start 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef0101280 0x7f0ef01972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0102470 0x7f0ef0197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef0102e30 0x7f0ef019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef019edb0 con 0x7f0ef0102470 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef019ef20 con 0x7f0ef0101280 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef8084640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef019f0e0 con 0x7f0ef0102e30 2024-02-14T15:02:36.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef561f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0102470 0x7f0ef0197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef561f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0102470 0x7f0ef0197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47000/0 (socket says 172.21.15.17:47000) 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef561f640 1 -- 172.21.15.17:0/3872214307 learned_addr learned my addr 172.21.15.17:0/3872214307 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.194+0000 7f0ef6621640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef0102e30 0x7f0ef019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef561f640 1 -- 172.21.15.17:0/3872214307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef0102e30 msgr2=0x7f0ef019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef561f640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef0102e30 0x7f0ef019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef561f640 1 -- 172.21.15.17:0/3872214307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef0101280 msgr2=0x7f0ef01972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef561f640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef0101280 0x7f0ef01972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef561f640 1 -- 172.21.15.17:0/3872214307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee8009560 con 0x7f0ef0102470 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef561f640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0102470 0x7f0ef0197800 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000d950 tx=0x7f0ee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ee6ffd640 1 -- 172.21.15.17:0/3872214307 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0009c30 con 0x7f0ef0102470 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ee6ffd640 1 -- 172.21.15.17:0/3872214307 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ee0009d90 con 0x7f0ef0102470 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef019f3c0 con 0x7f0ef0102470 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ee6ffd640 1 -- 172.21.15.17:0/3872214307 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000f7e0 con 0x7f0ef0102470 2024-02-14T15:02:36.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.195+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef019f8d0 con 0x7f0ef0102470 2024-02-14T15:02:36.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.196+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef0108540 con 0x7f0ef0102470 2024-02-14T15:02:36.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.197+0000 7f0ee6ffd640 1 -- 172.21.15.17:0/3872214307 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0ee0010ad0 con 0x7f0ef0102470 2024-02-14T15:02:36.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.198+0000 7f0ee6ffd640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0edc0a2150 0x7f0edc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.198+0000 7f0ef5e20640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0edc0a2150 0x7f0edc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.198+0000 7f0ee6ffd640 1 -- 172.21.15.17:0/3872214307 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0ee00cde30 con 0x7f0ef0102470 2024-02-14T15:02:36.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.198+0000 7f0ef5e20640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0edc0a2150 0x7f0edc0a4610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ee802c6a0 tx=0x7f0ee8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:36.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.202+0000 7f0ee6ffd640 1 -- 172.21.15.17:0/3872214307 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ee0096940 con 0x7f0ef0102470 2024-02-14T15:02:36.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.424+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ef002ce50 con 0x7f0ef0102470 2024-02-14T15:02:36.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.425+0000 7f0ee6ffd640 1 -- 172.21.15.17:0/3872214307 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f0ee009a000 con 0x7f0ef0102470 2024-02-14T15:02:36.427 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:36.427 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:01:15.551062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:02:36.427 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0edc0a2150 msgr2=0x7f0edc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0edc0a2150 0x7f0edc0a4610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0ee802c6a0 tx=0x7f0ee8000bc0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0102470 msgr2=0x7f0ef0197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0102470 0x7f0ef0197800 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000d950 tx=0x7f0ee000de10 comp rx=0 tx=0).stop 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 shutdown_connections 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0edc0a2150 0x7f0edc0a4610 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef0102470 0x7f0ef0197800 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef0102e30 0x7f0ef019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 --2- 172.21.15.17:0/3872214307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef0101280 0x7f0ef01972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 >> 172.21.15.17:0/3872214307 conn(0x7f0ef00fc7f0 msgr2=0x7f0ef0105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 shutdown_connections 2024-02-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.427+0000 7f0ef8084640 1 -- 172.21.15.17:0/3872214307 wait complete. 2024-02-14T15:02:36.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:02:36.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 -- 172.21.15.17:0/3909893184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c101d10 msgr2=0x7f371c1085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 --2- 172.21.15.17:0/3909893184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c101d10 0x7f371c1085a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f370c009900 tx=0x7f370c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:36.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 -- 172.21.15.17:0/3909893184 shutdown_connections 2024-02-14T15:02:36.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 --2- 172.21.15.17:0/3909893184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c101d10 0x7f371c1085a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 --2- 172.21.15.17:0/3909893184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f371c101350 0x7f371c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 --2- 172.21.15.17:0/3909893184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f371c100160 0x7f371c100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 -- 172.21.15.17:0/3909893184 >> 172.21.15.17:0/3909893184 conn(0x7f371c0fb6d0 msgr2=0x7f371c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 -- 172.21.15.17:0/3909893184 shutdown_connections 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.541+0000 7f3722b02640 1 -- 172.21.15.17:0/3909893184 wait complete. 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 Processor -- start 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 -- start start 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c100160 0x7f371c10aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f371c101350 0x7f371c109060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f371c101d10 0x7f371c1095a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c109d00 con 0x7f371c100160 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c109e70 con 0x7f371c101350 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f3722b02640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1a12c0 con 0x7f371c101d10 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c100160 0x7f371c10aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c100160 0x7f371c10aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47010/0 (socket says 172.21.15.17:47010) 2024-02-14T15:02:36.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 -- 172.21.15.17:0/2911795892 learned_addr learned my addr 172.21.15.17:0/2911795892 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 -- 172.21.15.17:0/2911795892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f371c101d10 msgr2=0x7f371c1095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f371c101d10 0x7f371c1095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 -- 172.21.15.17:0/2911795892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f371c101350 msgr2=0x7f371c109060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f371c101350 0x7f371c109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.542+0000 7f372089e640 1 -- 172.21.15.17:0/2911795892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c009560 con 0x7f371c100160 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.543+0000 7f372089e640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c100160 0x7f371c10aa20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f371400eba0 tx=0x7f371400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.543+0000 7f3719ffb640 1 -- 172.21.15.17:0/2911795892 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714003b70 con 0x7f371c100160 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.543+0000 7f3719ffb640 1 -- 172.21.15.17:0/2911795892 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f371400ced0 con 0x7f371c100160 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.543+0000 7f3719ffb640 1 -- 172.21.15.17:0/2911795892 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37140106b0 con 0x7f371c100160 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.543+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c1a1570 con 0x7f371c100160 2024-02-14T15:02:36.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.543+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c1a1a30 con 0x7f371c100160 2024-02-14T15:02:36.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.544+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c107420 con 0x7f371c100160 2024-02-14T15:02:36.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.545+0000 7f3719ffb640 1 -- 172.21.15.17:0/2911795892 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f3714004020 con 0x7f371c100160 2024-02-14T15:02:36.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.545+0000 7f3719ffb640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37040a2150 0x7f37040a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.546+0000 7f371bfff640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37040a2150 0x7f37040a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.546+0000 7f3719ffb640 1 -- 172.21.15.17:0/2911795892 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3714014070 con 0x7f371c100160 2024-02-14T15:02:36.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.546+0000 7f371bfff640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37040a2150 0x7f37040a4610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3710005fd0 tx=0x7f3710005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:36.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.550+0000 7f3719ffb640 1 -- 172.21.15.17:0/2911795892 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3714097470 con 0x7f371c100160 2024-02-14T15:02:36.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.731+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f371c02cea0 con 0x7f371c100160 2024-02-14T15:02:36.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.733+0000 7f3719ffb640 1 -- 172.21.15.17:0/2911795892 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24451 (secure 0 0 0) 0x7f371409ab30 con 0x7f371c100160 2024-02-14T15:02:36.735 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:36.735 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":91,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:01:44.589457+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37040a2150 msgr2=0x7f37040a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37040a2150 0x7f37040a4610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3710005fd0 tx=0x7f3710005850 comp rx=0 tx=0).stop 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c100160 msgr2=0x7f371c10aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c100160 0x7f371c10aa20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f371400eba0 tx=0x7f371400c6a0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 shutdown_connections 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f37040a2150 0x7f37040a4610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f371c101d10 0x7f371c1095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f371c101350 0x7f371c109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 --2- 172.21.15.17:0/2911795892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c100160 0x7f371c10aa20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 >> 172.21.15.17:0/2911795892 conn(0x7f371c0fb6d0 msgr2=0x7f371c1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:36.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 shutdown_connections 2024-02-14T15:02:36.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.735+0000 7f3722b02640 1 -- 172.21.15.17:0/2911795892 wait complete. 2024-02-14T15:02:36.747 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:36 smithi017 ceph-mon[32549]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:36.747 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:36 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1960711137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:02:36.747 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:36 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2299591665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:02:36.747 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:36 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3872214307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:36.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:02:36.749 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.852+0000 7fef9dcc2640 1 -- 172.21.15.17:0/3171249791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98101240 msgr2=0x7fef98101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.852+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/3171249791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98101240 0x7fef98101660 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fef88009900 tx=0x7fef8802bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.852+0000 7fef9dcc2640 1 -- 172.21.15.17:0/3171249791 shutdown_connections 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.852+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/3171249791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98102df0 0x7fef98109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.852+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/3171249791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98102430 0x7fef981028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.852+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/3171249791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98101240 0x7fef98101660 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.852+0000 7fef9dcc2640 1 -- 172.21.15.17:0/3171249791 >> 172.21.15.17:0/3171249791 conn(0x7fef980fc7b0 msgr2=0x7fef980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 -- 172.21.15.17:0/3171249791 shutdown_connections 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 -- 172.21.15.17:0/3171249791 wait complete. 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 Processor -- start 2024-02-14T15:02:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 -- start start 2024-02-14T15:02:36.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98101240 0x7fef98074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98102430 0x7fef98073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98102df0 0x7fef98073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980742c0 con 0x7fef98102df0 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef98074430 con 0x7fef98101240 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.853+0000 7fef9dcc2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981a13a0 con 0x7fef98102430 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98102df0 0x7fef98073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98102df0 0x7fef98073b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47038/0 (socket says 172.21.15.17:47038) 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 -- 172.21.15.17:0/1622224086 learned_addr learned my addr 172.21.15.17:0/1622224086 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef96ffd640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98102430 0x7fef98073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef977fe640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98101240 0x7fef98074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 -- 172.21.15.17:0/1622224086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98102430 msgr2=0x7fef98073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98102430 0x7fef98073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 -- 172.21.15.17:0/1622224086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98101240 msgr2=0x7fef98074fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98101240 0x7fef98074fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 -- 172.21.15.17:0/1622224086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009560 con 0x7fef98102df0 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef977fe640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98101240 0x7fef98074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef97fff640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98102df0 0x7fef98073b60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fef9000db20 tx=0x7fef9000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef94ff9640 1 -- 172.21.15.17:0/1622224086 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9000bea0 con 0x7fef98102df0 2024-02-14T15:02:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef981a1650 con 0x7fef98102df0 2024-02-14T15:02:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef94ff9640 1 -- 172.21.15.17:0/1622224086 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef90009940 con 0x7fef98102df0 2024-02-14T15:02:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef94ff9640 1 -- 172.21.15.17:0/1622224086 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900106c0 con 0x7fef98102df0 2024-02-14T15:02:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.854+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef981a1b60 con 0x7fef98102df0 2024-02-14T15:02:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.855+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef98108680 con 0x7fef98102df0 2024-02-14T15:02:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.857+0000 7fef94ff9640 1 -- 172.21.15.17:0/1622224086 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fef90004240 con 0x7fef98102df0 2024-02-14T15:02:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.857+0000 7fef94ff9640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef800a2150 0x7fef800a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.857+0000 7fef977fe640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef800a2150 0x7fef800a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:36.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.857+0000 7fef94ff9640 1 -- 172.21.15.17:0/1622224086 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fef900cde80 con 0x7fef98102df0 2024-02-14T15:02:36.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.858+0000 7fef977fe640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef800a2150 0x7fef800a4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef88036040 tx=0x7fef88000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:36.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:36.862+0000 7fef94ff9640 1 -- 172.21.15.17:0/1622224086 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef900969f0 con 0x7fef98102df0 2024-02-14T15:02:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:36 smithi050 ceph-mon[36512]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:36 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1960711137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:02:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:36 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2299591665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:02:36.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:36 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3872214307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:36 smithi076 ceph-mon[37057]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:36 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1960711137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:02:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:36 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2299591665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:02:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:36 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3872214307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:37.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.043+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef9802cdb0 con 0x7fef98102df0 2024-02-14T15:02:37.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.043+0000 7fef94ff9640 1 -- 172.21.15.17:0/1622224086 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef90017020 con 0x7fef98102df0 2024-02-14T15:02:37.045 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:02:37.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef800a2150 msgr2=0x7fef800a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef800a2150 0x7fef800a4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef88036040 tx=0x7fef88000bc0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98102df0 msgr2=0x7fef98073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98102df0 0x7fef98073b60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fef9000db20 tx=0x7fef9000b260 comp rx=0 tx=0).stop 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 shutdown_connections 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fef800a2150 0x7fef800a4610 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef98102df0 0x7fef98073b60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98102430 0x7fef98073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 --2- 172.21.15.17:0/1622224086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef98101240 0x7fef98074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.045+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 >> 172.21.15.17:0/1622224086 conn(0x7fef980fc7b0 msgr2=0x7fef98105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.046+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 shutdown_connections 2024-02-14T15:02:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.046+0000 7fef9dcc2640 1 -- 172.21.15.17:0/1622224086 wait complete. 2024-02-14T15:02:37.058 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:02:37.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 -- 172.21.15.17:0/2233822849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c101cc0 msgr2=0x7fe74c1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 --2- 172.21.15.17:0/2233822849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c101cc0 0x7fe74c1040b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe748009900 tx=0x7fe74802bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 -- 172.21.15.17:0/2233822849 shutdown_connections 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 --2- 172.21.15.17:0/2233822849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe74c106f20 0x7fe74c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 --2- 172.21.15.17:0/2233822849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe74c1045f0 0x7fe74c1069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 --2- 172.21.15.17:0/2233822849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c101cc0 0x7fe74c1040b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 -- 172.21.15.17:0/2233822849 >> 172.21.15.17:0/2233822849 conn(0x7fe74c0fb6f0 msgr2=0x7fe74c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 -- 172.21.15.17:0/2233822849 shutdown_connections 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.162+0000 7fe753c9a640 1 -- 172.21.15.17:0/2233822849 wait complete. 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 Processor -- start 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 -- start start 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe74c101cc0 0x7fe74c101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe74c1045f0 0x7fe74c0ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c106f20 0x7fe74c1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c100910 con 0x7fe74c106f20 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c100a80 con 0x7fe74c1045f0 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe753c9a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c1a1380 con 0x7fe74c101cc0 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c106f20 0x7fe74c1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c106f20 0x7fe74c1001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47054/0 (socket says 172.21.15.17:47054) 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 -- 172.21.15.17:0/4223665580 learned_addr learned my addr 172.21.15.17:0/4223665580 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe751235640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe74c1045f0 0x7fe74c0ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe751a36640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe74c101cc0 0x7fe74c101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 -- 172.21.15.17:0/4223665580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe74c101cc0 msgr2=0x7fe74c101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe74c101cc0 0x7fe74c101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 -- 172.21.15.17:0/4223665580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe74c1045f0 msgr2=0x7fe74c0ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe74c1045f0 0x7fe74c0ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.163+0000 7fe752237640 1 -- 172.21.15.17:0/4223665580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe748009560 con 0x7fe74c106f20 2024-02-14T15:02:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.164+0000 7fe752237640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c106f20 0x7fe74c1001b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe74400ead0 tx=0x7fe74400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:37.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.164+0000 7fe742ffd640 1 -- 172.21.15.17:0/4223665580 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400cde0 con 0x7fe74c106f20 2024-02-14T15:02:37.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.164+0000 7fe742ffd640 1 -- 172.21.15.17:0/4223665580 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe744005020 con 0x7fe74c106f20 2024-02-14T15:02:37.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.164+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c1a1630 con 0x7fe74c106f20 2024-02-14T15:02:37.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.164+0000 7fe742ffd640 1 -- 172.21.15.17:0/4223665580 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440116b0 con 0x7fe74c106f20 2024-02-14T15:02:37.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.164+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c1a1b40 con 0x7fe74c106f20 2024-02-14T15:02:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.165+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c0fd1e0 con 0x7fe74c106f20 2024-02-14T15:02:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.166+0000 7fe742ffd640 1 -- 172.21.15.17:0/4223665580 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe744005520 con 0x7fe74c106f20 2024-02-14T15:02:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.167+0000 7fe742ffd640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe7380a21d0 0x7fe7380a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.167+0000 7fe751a36640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe7380a21d0 0x7fe7380a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.167+0000 7fe742ffd640 1 -- 172.21.15.17:0/4223665580 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe7440ce2e0 con 0x7fe74c106f20 2024-02-14T15:02:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.167+0000 7fe751a36640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe7380a21d0 0x7fe7380a4690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe74802c840 tx=0x7fe748000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:37.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.171+0000 7fe742ffd640 1 -- 172.21.15.17:0/4223665580 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe744096e50 con 0x7fe74c106f20 2024-02-14T15:02:37.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.352+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe74c02cdb0 con 0x7fe74c106f20 2024-02-14T15:02:37.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.352+0000 7fe742ffd640 1 -- 172.21.15.17:0/4223665580 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe74401f020 con 0x7fe74c106f20 2024-02-14T15:02:37.354 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:02:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe7380a21d0 msgr2=0x7fe7380a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe7380a21d0 0x7fe7380a4690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe74802c840 tx=0x7fe748000bc0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c106f20 msgr2=0x7fe74c1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c106f20 0x7fe74c1001b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe74400ead0 tx=0x7fe74400ef90 comp rx=0 tx=0).stop 2024-02-14T15:02:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 shutdown_connections 2024-02-14T15:02:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fe7380a21d0 0x7fe7380a4690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe74c106f20 0x7fe74c1001b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe74c1045f0 0x7fe74c0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 --2- 172.21.15.17:0/4223665580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe74c101cc0 0x7fe74c101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 >> 172.21.15.17:0/4223665580 conn(0x7fe74c0fb6f0 msgr2=0x7fe74c1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 shutdown_connections 2024-02-14T15:02:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.354+0000 7fe753c9a640 1 -- 172.21.15.17:0/4223665580 wait complete. 2024-02-14T15:02:37.367 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:02:37.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 -- 172.21.15.17:0/3344754393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44106f90 msgr2=0x7efe44109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 --2- 172.21.15.17:0/3344754393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44106f90 0x7efe44109380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009ca0 tx=0x7efe3c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 -- 172.21.15.17:0/3344754393 shutdown_connections 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 --2- 172.21.15.17:0/3344754393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44106f90 0x7efe44109380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 --2- 172.21.15.17:0/3344754393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44104660 0x7efe44106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 --2- 172.21.15.17:0/3344754393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe44101d30 0x7efe44104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 -- 172.21.15.17:0/3344754393 >> 172.21.15.17:0/3344754393 conn(0x7efe440fb6c0 msgr2=0x7efe440fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 -- 172.21.15.17:0/3344754393 shutdown_connections 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 -- 172.21.15.17:0/3344754393 wait complete. 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 Processor -- start 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.471+0000 7efe48e23640 1 -- start start 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44101d30 0x7efe44197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44104660 0x7efe441977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe44106f90 0x7efe4419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4419ed60 con 0x7efe44101d30 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4419eed0 con 0x7efe44106f90 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4419f090 con 0x7efe44104660 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44101d30 0x7efe44197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44101d30 0x7efe44197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47078/0 (socket says 172.21.15.17:47078) 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 -- 172.21.15.17:0/590143857 learned_addr learned my addr 172.21.15.17:0/590143857 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe42d9d640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe44106f90 0x7efe4419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe41d9b640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44104660 0x7efe441977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 -- 172.21.15.17:0/590143857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44104660 msgr2=0x7efe441977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44104660 0x7efe441977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 -- 172.21.15.17:0/590143857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe44106f90 msgr2=0x7efe4419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe44106f90 0x7efe4419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 -- 172.21.15.17:0/590143857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c009900 con 0x7efe44101d30 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe41d9b640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44104660 0x7efe441977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe4259c640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44101d30 0x7efe44197270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b4a0 tx=0x7efe3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe337fe640 1 -- 172.21.15.17:0/590143857 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34010610 con 0x7efe44101d30 2024-02-14T15:02:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4419f370 con 0x7efe44101d30 2024-02-14T15:02:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe337fe640 1 -- 172.21.15.17:0/590143857 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe34010c50 con 0x7efe44101d30 2024-02-14T15:02:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe337fe640 1 -- 172.21.15.17:0/590143857 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34017e20 con 0x7efe44101d30 2024-02-14T15:02:37.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.472+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4419f880 con 0x7efe44101d30 2024-02-14T15:02:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.473+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe440fd1b0 con 0x7efe44101d30 2024-02-14T15:02:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.474+0000 7efe337fe640 1 -- 172.21.15.17:0/590143857 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7efe34017480 con 0x7efe44101d30 2024-02-14T15:02:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.475+0000 7efe337fe640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe2c0a2150 0x7efe2c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.475+0000 7efe337fe640 1 -- 172.21.15.17:0/590143857 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efe340cd690 con 0x7efe44101d30 2024-02-14T15:02:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.475+0000 7efe41d9b640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe2c0a2150 0x7efe2c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.475+0000 7efe41d9b640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe2c0a2150 0x7efe2c0a4610 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe44198780 tx=0x7efe38005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.479+0000 7efe337fe640 1 -- 172.21.15.17:0/590143857 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe34096310 con 0x7efe44101d30 2024-02-14T15:02:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.665+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe4410cf50 con 0x7efe44101d30 2024-02-14T15:02:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.665+0000 7efe337fe640 1 -- 172.21.15.17:0/590143857 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe340999d0 con 0x7efe44101d30 2024-02-14T15:02:37.667 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe2c0a2150 msgr2=0x7efe2c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe2c0a2150 0x7efe2c0a4610 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe44198780 tx=0x7efe38005d90 comp rx=0 tx=0).stop 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44101d30 msgr2=0x7efe44197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44101d30 0x7efe44197270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b4a0 tx=0x7efe3400b960 comp rx=0 tx=0).stop 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 shutdown_connections 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe2c0a2150 0x7efe2c0a4610 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe44106f90 0x7efe4419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44104660 0x7efe441977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 --2- 172.21.15.17:0/590143857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe44101d30 0x7efe44197270 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 >> 172.21.15.17:0/590143857 conn(0x7efe440fb6c0 msgr2=0x7efe44102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 shutdown_connections 2024-02-14T15:02:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.667+0000 7efe48e23640 1 -- 172.21.15.17:0/590143857 wait complete. 2024-02-14T15:02:37.679 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:37 smithi017 ceph-mon[32549]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:37.679 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:37 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2911795892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:37.679 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:37 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1622224086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:02:37.679 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:37 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4223665580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:02:37.680 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:02:37.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 -- 172.21.15.17:0/373575357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102df0 msgr2=0x7f7118109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 --2- 172.21.15.17:0/373575357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102df0 0x7f7118109680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7110009ca0 tx=0x7f711002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 -- 172.21.15.17:0/373575357 shutdown_connections 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 --2- 172.21.15.17:0/373575357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102df0 0x7f7118109680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 --2- 172.21.15.17:0/373575357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7118102430 0x7f71181028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 --2- 172.21.15.17:0/373575357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7118101240 0x7f7118101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 -- 172.21.15.17:0/373575357 >> 172.21.15.17:0/373575357 conn(0x7f71180fc7b0 msgr2=0x7f71180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 -- 172.21.15.17:0/373575357 shutdown_connections 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 -- 172.21.15.17:0/373575357 wait complete. 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 Processor -- start 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.784+0000 7f711d1bc640 1 -- start start 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7118101240 0x7f7118074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102430 0x7f7118073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7118102df0 0x7f7118073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180742c0 con 0x7f7118102430 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7118074430 con 0x7f7118101240 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181a12c0 con 0x7f7118102df0 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102430 0x7f7118073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102430 0x7f7118073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47098/0 (socket says 172.21.15.17:47098) 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 -- 172.21.15.17:0/3079612540 learned_addr learned my addr 172.21.15.17:0/3079612540 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f7116d9d640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7118101240 0x7f7118074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711759e640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7118102df0 0x7f7118073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 -- 172.21.15.17:0/3079612540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7118102df0 msgr2=0x7f7118073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7118102df0 0x7f7118073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 -- 172.21.15.17:0/3079612540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7118101240 msgr2=0x7f7118074fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7118101240 0x7f7118074fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 -- 172.21.15.17:0/3079612540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110009900 con 0x7f7118102430 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f710ffff640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102430 0x7f7118073620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f710000d980 tx=0x7f710000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f7114d99640 1 -- 172.21.15.17:0/3079612540 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100009c30 con 0x7f7118102430 2024-02-14T15:02:37.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f7114d99640 1 -- 172.21.15.17:0/3079612540 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f710000b660 con 0x7f7118102430 2024-02-14T15:02:37.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71181a1570 con 0x7f7118102430 2024-02-14T15:02:37.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f7114d99640 1 -- 172.21.15.17:0/3079612540 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000f6b0 con 0x7f7118102430 2024-02-14T15:02:37.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.785+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71181a1a80 con 0x7f7118102430 2024-02-14T15:02:37.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.786+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118108500 con 0x7f7118102430 2024-02-14T15:02:37.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.788+0000 7f7114d99640 1 -- 172.21.15.17:0/3079612540 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7100010460 con 0x7f7118102430 2024-02-14T15:02:37.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.788+0000 7f7114d99640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f71040a2150 0x7f71040a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:37.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.788+0000 7f7116d9d640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f71040a2150 0x7f71040a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:37.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.788+0000 7f7114d99640 1 -- 172.21.15.17:0/3079612540 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f71000cddf0 con 0x7f7118102430 2024-02-14T15:02:37.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.789+0000 7f7116d9d640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f71040a2150 0x7f71040a4610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7118102290 tx=0x7f710800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:37.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.792+0000 7f7114d99640 1 -- 172.21.15.17:0/3079612540 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7100096960 con 0x7f7118102430 2024-02-14T15:02:37.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:37 smithi050 ceph-mon[36512]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:37.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:37 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2911795892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:37.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:37 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1622224086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:02:37.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:37 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4223665580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:02:37.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.974+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f711810cde0 con 0x7f7118102430 2024-02-14T15:02:37.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.974+0000 7f7114d99640 1 -- 172.21.15.17:0/3079612540 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7100020020 con 0x7f7118102430 2024-02-14T15:02:37.976 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f71040a2150 msgr2=0x7f71040a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f71040a2150 0x7f71040a4610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7118102290 tx=0x7f710800b390 comp rx=0 tx=0).stop 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102430 msgr2=0x7f7118073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102430 0x7f7118073620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f710000d980 tx=0x7f710000de40 comp rx=0 tx=0).stop 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 shutdown_connections 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f71040a2150 0x7f71040a4610 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7118102430 0x7f7118073620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7118102df0 0x7f7118073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 --2- 172.21.15.17:0/3079612540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7118101240 0x7f7118074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:37.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 >> 172.21.15.17:0/3079612540 conn(0x7f71180fc7b0 msgr2=0x7f71181056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:37.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 shutdown_connections 2024-02-14T15:02:37.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:37.976+0000 7f711d1bc640 1 -- 172.21.15.17:0/3079612540 wait complete. 2024-02-14T15:02:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:37 smithi076 ceph-mon[37057]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:37 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2911795892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:37 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1622224086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:02:37.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:37 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4223665580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:02:37.989 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:02:38.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.090+0000 7f227b640640 1 -- 172.21.15.17:0/94491276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274104740 msgr2=0x7f2274106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.090+0000 7f227b640640 1 --2- 172.21.15.17:0/94491276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274104740 0x7f2274106b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2264009900 tx=0x7f226402bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.090+0000 7f227b640640 1 -- 172.21.15.17:0/94491276 shutdown_connections 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.090+0000 7f227b640640 1 --2- 172.21.15.17:0/94491276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274104740 0x7f2274106b30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.090+0000 7f227b640640 1 --2- 172.21.15.17:0/94491276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2274107070 0x7f2274109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.090+0000 7f227b640640 1 --2- 172.21.15.17:0/94491276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2274101e10 0x7f2274104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.090+0000 7f227b640640 1 -- 172.21.15.17:0/94491276 >> 172.21.15.17:0/94491276 conn(0x7f22740fb760 msgr2=0x7f22740fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 -- 172.21.15.17:0/94491276 shutdown_connections 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 -- 172.21.15.17:0/94491276 wait complete. 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 Processor -- start 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 -- start start 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2274101e10 0x7f2274197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2274104740 0x7f2274197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274107070 0x7f227419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227419edf0 con 0x7f2274107070 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227419ef60 con 0x7f2274104740 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f227b640640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227419f120 con 0x7f2274101e10 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f2279bdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274107070 0x7f227419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f2279bdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274107070 0x7f227419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47124/0 (socket says 172.21.15.17:47124) 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f2279bdd640 1 -- 172.21.15.17:0/2724240213 learned_addr learned my addr 172.21.15.17:0/2724240213 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f2278bdb640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2274104740 0x7f2274197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.091+0000 7f22793dc640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2274101e10 0x7f2274197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f2279bdd640 1 -- 172.21.15.17:0/2724240213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2274101e10 msgr2=0x7f2274197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f2279bdd640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2274101e10 0x7f2274197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f2279bdd640 1 -- 172.21.15.17:0/2724240213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2274104740 msgr2=0x7f2274197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f2279bdd640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2274104740 0x7f2274197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f2279bdd640 1 -- 172.21.15.17:0/2724240213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009560 con 0x7f2274107070 2024-02-14T15:02:38.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f2279bdd640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274107070 0x7f227419e8b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f226c00eaa0 tx=0x7f226c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:38.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f226a7fc640 1 -- 172.21.15.17:0/2724240213 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00cdb0 con 0x7f2274107070 2024-02-14T15:02:38.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227419f400 con 0x7f2274107070 2024-02-14T15:02:38.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f226a7fc640 1 -- 172.21.15.17:0/2724240213 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f226c005020 con 0x7f2274107070 2024-02-14T15:02:38.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f226a7fc640 1 -- 172.21.15.17:0/2724240213 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c0116b0 con 0x7f2274107070 2024-02-14T15:02:38.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.092+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227419f8e0 con 0x7f2274107070 2024-02-14T15:02:38.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.093+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22740fd250 con 0x7f2274107070 2024-02-14T15:02:38.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.094+0000 7f226a7fc640 1 -- 172.21.15.17:0/2724240213 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f226c005520 con 0x7f2274107070 2024-02-14T15:02:38.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.095+0000 7f226a7fc640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f22600a2150 0x7f22600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.095+0000 7f22793dc640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f22600a2150 0x7f22600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.095+0000 7f226a7fc640 1 -- 172.21.15.17:0/2724240213 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f226c0ce770 con 0x7f2274107070 2024-02-14T15:02:38.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.095+0000 7f22793dc640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f22600a2150 0x7f22600a4610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2270005fd0 tx=0x7f2270005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:38.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.099+0000 7f226a7fc640 1 -- 172.21.15.17:0/2724240213 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f226c0972e0 con 0x7f2274107070 2024-02-14T15:02:38.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.319+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f227410cc80 con 0x7f2274107070 2024-02-14T15:02:38.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.320+0000 7f226a7fc640 1 -- 172.21.15.17:0/2724240213 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f226c01f070 con 0x7f2274107070 2024-02-14T15:02:38.322 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:38.322 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:01:15.551062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:02:38.322 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-02-14T15:02:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f22600a2150 msgr2=0x7f22600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f22600a2150 0x7f22600a4610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2270005fd0 tx=0x7f2270005850 comp rx=0 tx=0).stop 2024-02-14T15:02:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274107070 msgr2=0x7f227419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274107070 0x7f227419e8b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f226c00eaa0 tx=0x7f226c00ef60 comp rx=0 tx=0).stop 2024-02-14T15:02:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 shutdown_connections 2024-02-14T15:02:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f22600a2150 0x7f22600a4610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2274107070 0x7f227419e8b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2274104740 0x7f2274197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 --2- 172.21.15.17:0/2724240213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2274101e10 0x7f2274197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 >> 172.21.15.17:0/2724240213 conn(0x7f22740fb760 msgr2=0x7f22741027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 shutdown_connections 2024-02-14T15:02:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.322+0000 7f227b640640 1 -- 172.21.15.17:0/2724240213 wait complete. 2024-02-14T15:02:38.335 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:02:38.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 -- 172.21.15.17:0/1619962592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c106fb0 msgr2=0x7fdd1c109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 --2- 172.21.15.17:0/1619962592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c106fb0 0x7fdd1c109400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009ca0 tx=0x7fdd0c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 -- 172.21.15.17:0/1619962592 shutdown_connections 2024-02-14T15:02:38.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 --2- 172.21.15.17:0/1619962592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c106fb0 0x7fdd1c109400 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 --2- 172.21.15.17:0/1619962592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c104680 0x7fdd1c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 --2- 172.21.15.17:0/1619962592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd1c101d50 0x7fdd1c104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 -- 172.21.15.17:0/1619962592 >> 172.21.15.17:0/1619962592 conn(0x7fdd1c0fb760 msgr2=0x7fdd1c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 -- 172.21.15.17:0/1619962592 shutdown_connections 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.436+0000 7fdd241b5640 1 -- 172.21.15.17:0/1619962592 wait complete. 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 Processor -- start 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 -- start start 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd1c101d50 0x7fdd1c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c104680 0x7fdd1c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c106fb0 0x7fdd1c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19edb0 con 0x7fdd1c104680 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19ef20 con 0x7fdd1c106fb0 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd241b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19f0e0 con 0x7fdd1c101d50 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c104680 0x7fdd1c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c104680 0x7fdd1c197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47132/0 (socket says 172.21.15.17:47132) 2024-02-14T15:02:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 -- 172.21.15.17:0/11263964 learned_addr learned my addr 172.21.15.17:0/11263964 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd22752640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c106fb0 0x7fdd1c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21f51640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd1c101d50 0x7fdd1c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 -- 172.21.15.17:0/11263964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd1c101d50 msgr2=0x7fdd1c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd1c101d50 0x7fdd1c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 -- 172.21.15.17:0/11263964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c106fb0 msgr2=0x7fdd1c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c106fb0 0x7fdd1c19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 -- 172.21.15.17:0/11263964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c009900 con 0x7fdd1c104680 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd22752640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c106fb0 0x7fdd1c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21f51640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd1c101d50 0x7fdd1c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.437+0000 7fdd21750640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c104680 0x7fdd1c197800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c970 tx=0x7fdd1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.438+0000 7fdd1affd640 1 -- 172.21.15.17:0/11263964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10007bb0 con 0x7fdd1c104680 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.438+0000 7fdd1affd640 1 -- 172.21.15.17:0/11263964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd10007d10 con 0x7fdd1c104680 2024-02-14T15:02:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.438+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c19f3c0 con 0x7fdd1c104680 2024-02-14T15:02:38.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.438+0000 7fdd1affd640 1 -- 172.21.15.17:0/11263964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000f450 con 0x7fdd1c104680 2024-02-14T15:02:38.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.438+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c19f8d0 con 0x7fdd1c104680 2024-02-14T15:02:38.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.439+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c0fd250 con 0x7fdd1c104680 2024-02-14T15:02:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.440+0000 7fdd1affd640 1 -- 172.21.15.17:0/11263964 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fdd10004f20 con 0x7fdd1c104680 2024-02-14T15:02:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.441+0000 7fdd1affd640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd040a2150 0x7fdd040a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.441+0000 7fdd21f51640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd040a2150 0x7fdd040a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.441+0000 7fdd1affd640 1 -- 172.21.15.17:0/11263964 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdd100ce290 con 0x7fdd1c104680 2024-02-14T15:02:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.441+0000 7fdd21f51640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd040a2150 0x7fdd040a4610 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd140058c0 tx=0x7fdd14005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:38.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.445+0000 7fdd1affd640 1 -- 172.21.15.17:0/11263964 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd10096e00 con 0x7fdd1c104680 2024-02-14T15:02:38.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.628+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd1c10cc70 con 0x7fdd1c104680 2024-02-14T15:02:38.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.630+0000 7fdd1affd640 1 -- 172.21.15.17:0/11263964 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24451 (secure 0 0 0) 0x7fdd1001d020 con 0x7fdd1c104680 2024-02-14T15:02:38.632 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:38.632 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":91,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:01:44.589457+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd040a2150 msgr2=0x7fdd040a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd040a2150 0x7fdd040a4610 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd140058c0 tx=0x7fdd14005d30 comp rx=0 tx=0).stop 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c104680 msgr2=0x7fdd1c197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c104680 0x7fdd1c197800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c970 tx=0x7fdd1000ce30 comp rx=0 tx=0).stop 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 shutdown_connections 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdd040a2150 0x7fdd040a4610 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c104680 0x7fdd1c197800 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd1c106fb0 0x7fdd1c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 --2- 172.21.15.17:0/11263964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd1c101d50 0x7fdd1c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 >> 172.21.15.17:0/11263964 conn(0x7fdd1c0fb760 msgr2=0x7fdd1c1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:38.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 shutdown_connections 2024-02-14T15:02:38.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.632+0000 7fdd241b5640 1 -- 172.21.15.17:0/11263964 wait complete. 2024-02-14T15:02:38.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:02:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 -- 172.21.15.17:0/3464832203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 msgr2=0x7fb5e0102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 --2- 172.21.15.17:0/3464832203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e0102930 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009900 tx=0x7fb5d402bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 -- 172.21.15.17:0/3464832203 shutdown_connections 2024-02-14T15:02:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 --2- 172.21.15.17:0/3464832203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e0102930 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 --2- 172.21.15.17:0/3464832203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5e0102e70 0x7fb5e0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 --2- 172.21.15.17:0/3464832203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5e01012c0 0x7fb5e01016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 -- 172.21.15.17:0/3464832203 >> 172.21.15.17:0/3464832203 conn(0x7fb5e00fc850 msgr2=0x7fb5e00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 -- 172.21.15.17:0/3464832203 shutdown_connections 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.746+0000 7fb5e609a640 1 -- 172.21.15.17:0/3464832203 wait complete. 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 Processor -- start 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 -- start start 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5e01012c0 0x7fb5e0197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e01977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5e0102e70 0x7fb5e019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e019ed80 con 0x7fb5e01024b0 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e019eef0 con 0x7fb5e01012c0 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5e609a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e019f0b0 con 0x7fb5e0102e70 2024-02-14T15:02:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e01977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e01977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47144/0 (socket says 172.21.15.17:47144) 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 -- 172.21.15.17:0/4074896918 learned_addr learned my addr 172.21.15.17:0/4074896918 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5df7fe640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5e01012c0 0x7fb5e0197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5dffff640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5e0102e70 0x7fb5e019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 -- 172.21.15.17:0/4074896918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5e0102e70 msgr2=0x7fb5e019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5e0102e70 0x7fb5e019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 -- 172.21.15.17:0/4074896918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5e01012c0 msgr2=0x7fb5e0197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5e01012c0 0x7fb5e0197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5deffd640 1 -- 172.21.15.17:0/4074896918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d4009560 con 0x7fb5e01024b0 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.747+0000 7fb5df7fe640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5e01012c0 0x7fb5e0197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.748+0000 7fb5deffd640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e01977d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4006010 tx=0x7fb5d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.748+0000 7fb5dcff9640 1 -- 172.21.15.17:0/4074896918 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40044e0 con 0x7fb5e01024b0 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.748+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e019f330 con 0x7fb5e01024b0 2024-02-14T15:02:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.748+0000 7fb5dcff9640 1 -- 172.21.15.17:0/4074896918 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5d40054e0 con 0x7fb5e01024b0 2024-02-14T15:02:38.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.748+0000 7fb5dcff9640 1 -- 172.21.15.17:0/4074896918 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d403e450 con 0x7fb5e01024b0 2024-02-14T15:02:38.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.748+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e019f890 con 0x7fb5e01024b0 2024-02-14T15:02:38.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.749+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e01085e0 con 0x7fb5e01024b0 2024-02-14T15:02:38.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.750+0000 7fb5dcff9640 1 -- 172.21.15.17:0/4074896918 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb5d40035f0 con 0x7fb5e01024b0 2024-02-14T15:02:38.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.750+0000 7fb5dcff9640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5c80a2150 0x7fb5c80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:38.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.751+0000 7fb5df7fe640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5c80a2150 0x7fb5c80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:38.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.751+0000 7fb5dcff9640 1 -- 172.21.15.17:0/4074896918 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb5d40f04e0 con 0x7fb5e01024b0 2024-02-14T15:02:38.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.751+0000 7fb5df7fe640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5c80a2150 0x7fb5c80a4610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0005d00 tx=0x7fb5d0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:38.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.755+0000 7fb5dcff9640 1 -- 172.21.15.17:0/4074896918 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5d40b9160 con 0x7fb5e01024b0 2024-02-14T15:02:38.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:38 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/590143857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:02:38.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:38 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3079612540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:02:38.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:38 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2724240213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.975+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5e019f4c0 con 0x7fb5e01024b0 2024-02-14T15:02:38.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.975+0000 7fb5dcff9640 1 -- 172.21.15.17:0/4074896918 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11317 (secure 0 0 0) 0x7fb5e019f4c0 con 0x7fb5e01024b0 2024-02-14T15:02:38.977 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:38.977 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:01:15.551062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:02:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-02-14T15:02:38.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5c80a2150 msgr2=0x7fb5c80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5c80a2150 0x7fb5c80a4610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0005d00 tx=0x7fb5d0005c90 comp rx=0 tx=0).stop 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 msgr2=0x7fb5e01977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e01977d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4006010 tx=0x7fb5d40042f0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 shutdown_connections 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5c80a2150 0x7fb5c80a4610 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5e01024b0 0x7fb5e01977d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5e0102e70 0x7fb5e019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 --2- 172.21.15.17:0/4074896918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5e01012c0 0x7fb5e0197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.977+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 >> 172.21.15.17:0/4074896918 conn(0x7fb5e00fc850 msgr2=0x7fb5e01057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.978+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 shutdown_connections 2024-02-14T15:02:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:38.978+0000 7fb5e609a640 1 -- 172.21.15.17:0/4074896918 wait complete. 2024-02-14T15:02:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:38 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/590143857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:02:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:38 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3079612540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:02:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:38 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2724240213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:38.989 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:38 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/590143857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:02:38.989 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:38 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3079612540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:02:38.989 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:38 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2724240213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:38.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-14T15:00:48.619843+0000', 'modified': '2024-02-14T15:01:15.551062+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24502, 'mds_2': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6835/3474801477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 3474801477}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 3474801477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.76:6835/1787917010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 1787917010}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 1787917010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6837/1706654420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 1706654420}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 1706654420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.76:6837/3638854935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3638854935}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3638854935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.76:6839/666966898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 666966898}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 666966898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6839/916171404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 916171404}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 916171404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-14T15:02:38.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-14T15:02:38.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 -- 172.21.15.17:0/1884045197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c102430 msgr2=0x7f026c1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 --2- 172.21.15.17:0/1884045197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c102430 0x7f026c1028b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f025c009900 tx=0x7f025c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 -- 172.21.15.17:0/1884045197 shutdown_connections 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 --2- 172.21.15.17:0/1884045197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c102430 0x7f026c1028b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 --2- 172.21.15.17:0/1884045197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f026c102df0 0x7f026c109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 --2- 172.21.15.17:0/1884045197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f026c101240 0x7f026c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 -- 172.21.15.17:0/1884045197 >> 172.21.15.17:0/1884045197 conn(0x7f026c0fc7b0 msgr2=0x7f026c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 -- 172.21.15.17:0/1884045197 shutdown_connections 2024-02-14T15:02:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.093+0000 7f027345c640 1 -- 172.21.15.17:0/1884045197 wait complete. 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 Processor -- start 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 -- start start 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c101240 0x7f026c074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f026c102430 0x7f026c073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f026c102df0 0x7f026c073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0742c0 con 0x7f026c101240 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c074430 con 0x7f026c102430 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f027345c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c1a1330 con 0x7f026c102df0 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c101240 0x7f026c074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c101240 0x7f026c074fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47158/0 (socket says 172.21.15.17:47158) 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 -- 172.21.15.17:0/3431634214 learned_addr learned my addr 172.21.15.17:0/3431634214 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02719f9640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f026c102df0 0x7f026c073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02709f7640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f026c102430 0x7f026c073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 -- 172.21.15.17:0/3431634214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f026c102df0 msgr2=0x7f026c073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f026c102df0 0x7f026c073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 -- 172.21.15.17:0/3431634214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f026c102430 msgr2=0x7f026c073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f026c102430 0x7f026c073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.094+0000 7f02711f8640 1 -- 172.21.15.17:0/3431634214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c009560 con 0x7f026c101240 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.095+0000 7f02711f8640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c101240 0x7f026c074fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f026800d980 tx=0x7f026800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.095+0000 7f02627fc640 1 -- 172.21.15.17:0/3431634214 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268009c30 con 0x7f026c101240 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.095+0000 7f02627fc640 1 -- 172.21.15.17:0/3431634214 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0268009d90 con 0x7f026c101240 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.095+0000 7f02627fc640 1 -- 172.21.15.17:0/3431634214 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800f790 con 0x7f026c101240 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.095+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c1a15e0 con 0x7f026c101240 2024-02-14T15:02:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.095+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c1a1b10 con 0x7f026c101240 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.097+0000 7f02627fc640 1 -- 172.21.15.17:0/3431634214 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0268010ad0 con 0x7f026c101240 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 Processor -- start 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 -- start start 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece3e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3ee50 0x5562ece43950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02627fc640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02580a21d0 0x7f02580a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece46450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ecdfafd0 con 0x5562ece43fc0 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ecdadcb0 con 0x5562ece3e460 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f027602e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ecdab1e0 con 0x5562ece3ee50 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02709f7640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02580a21d0 0x7f02580a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece46450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece46450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47160/0 (socket says 172.21.15.17:47160) 2024-02-14T15:02:39.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 -- 172.21.15.17:0/3356942976 learned_addr learned my addr 172.21.15.17:0/3356942976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02627fc640 1 -- 172.21.15.17:0/3431634214 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f02680cdce0 con 0x7f026c101240 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece3e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02711f8640 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3ee50 0x5562ece43950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 -- 172.21.15.17:0/3356942976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3ee50 msgr2=0x5562ece43950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3ee50 0x5562ece43950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 -- 172.21.15.17:0/3356942976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 msgr2=0x5562ece3e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece3e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.098+0000 7f02719f9640 1 -- 172.21.15.17:0/3356942976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562ecd69100 con 0x5562ece43fc0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.099+0000 7f02709f7640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02580a21d0 0x7f02580a4690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f025c009230 tx=0x7f025c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.099+0000 7f02719f9640 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece46450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0264005c20 tx=0x7f0264009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67240b2ec225b8fb server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.099+0000 7f02577fe640 1 -- 172.21.15.17:0/3356942976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264003b70 con 0x5562ece43fc0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.099+0000 7f02577fe640 1 -- 172.21.15.17:0/3356942976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0264010d50 con 0x5562ece43fc0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.099+0000 7f02577fe640 1 -- 172.21.15.17:0/3356942976 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264004630 con 0x5562ece43fc0 2024-02-14T15:02:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.099+0000 7f027602e740 1 -- 172.21.15.17:0/3356942976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 msgr2=0x5562ece46450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.099+0000 7f027602e740 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece46450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0264005c20 tx=0x7f0264009aa0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- 172.21.15.17:0/3356942976 shutdown_connections 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece46450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3ee50 0x5562ece43950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 --2- 172.21.15.17:0/3356942976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece3e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- 172.21.15.17:0/3356942976 >> 172.21.15.17:0/3356942976 conn(0x5562ece399d0 msgr2=0x5562ece39dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- 172.21.15.17:0/3356942976 shutdown_connections 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- 172.21.15.17:0/3356942976 wait complete. 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- start start 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 Processor -- start 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562ece48be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece4fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ecda69c0 con 0x5562ece43fc0 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ecd4bcd0 con 0x5562ece3ee50 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562eca7a310 con 0x5562ece3e460 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f027602e740 10 client.0 _ll_register_callbacks cb 0x5562ecab7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f02719f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece4fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f02719f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece4fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47162/0 (socket says 172.21.15.17:47162) 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f02719f9640 1 -- 172.21.15.17:0/1415567865 learned_addr learned my addr 172.21.15.17:0/1415567865 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.100+0000 7f02711f8640 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f02719f9640 1 -- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 msgr2=0x5562ece486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f02719f9640 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f02709f7640 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562ece48be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f02719f9640 1 -- 172.21.15.17:0/1415567865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 msgr2=0x5562ece48be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f02719f9640 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562ece48be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f02719f9640 1 -- 172.21.15.17:0/1415567865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02640123f0 con 0x5562ece43fc0 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f02719f9640 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece4fc20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0264005bf0 tx=0x7f0264010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90199a60c19cb11 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f0254ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f0254ff9640 1 -- 172.21.15.17:0/1415567865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264023070 con 0x5562ece43fc0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f0254ff9640 1 -- 172.21.15.17:0/1415567865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f02640045c0 con 0x5562ece43fc0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562ecaa66c0 con 0x5562ece43fc0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f0254ff9640 1 -- 172.21.15.17:0/1415567865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0264004780 con 0x5562ece43fc0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f027602e740 10 client.15255 fetch_fsmap learned FSMap version 23 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5562ecda5040 con 0x5562ece43fc0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.101+0000 7f0254ff9640 1 -- 172.21.15.17:0/1415567865 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5562ecaa66c0 con 0x5562ece43fc0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f0254ff9640 1 -- 172.21.15.17:0/1415567865 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0264005020 con 0x5562ece43fc0 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f027602e740 10 client.15255 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f027602e740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f027602e740 20 client.15255 populate_metadata read hostname 'smithi017' 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f027602e740 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece53d40 0x5562ece58e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f027602e740 4 client.15255 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:02:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562ecaf7270 con 0x5562ece53d40 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f02711f8640 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece53d40 0x5562ece58e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f0254ff9640 10 client.15255 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.102+0000 7f02711f8640 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece53d40 0x5562ece58e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7cfe4a2a41a9beb server_cookie=be143a09cb8eb50e in_seq=0 out_seq=0 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f0254ff9640 1 -- 172.21.15.17:0/1415567865 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5562ecaf7270 con 0x5562ece53d40 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f0254ff9640 10 client.15255 handle_command_reply: tid=0 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 client.15255 shutdown 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 msgr2=0x5562ece4fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece4fc20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0264005bf0 tx=0x7f0264010aa0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 shutdown_connections 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece53d40 0x5562ece58e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece43fc0 0x5562ece4fc20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562ece48be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f0254ff9640 0 client.15255 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:02:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 --2- 172.21.15.17:0/1415567865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 >> 172.21.15.17:0/1415567865 conn(0x5562ece399d0 msgr2=0x5562ece38c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 shutdown_connections 2024-02-14T15:02:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 1 -- 172.21.15.17:0/1415567865 wait complete. 2024-02-14T15:02:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.103+0000 7f027602e740 20 client.15255 trim_cache size 0 max 16384 2024-02-14T15:02:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 Processor -- start 2024-02-14T15:02:39.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 -- start start 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece399d0 0x5562ece5d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece3ee50 0x5562ece383a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ecd2c490 con 0x5562ece3ee50 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ece2e540 con 0x5562ece3e460 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.119+0000 7f027602e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ecaf7270 con 0x5562ece399d0 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece3ee50 0x5562ece383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece3ee50 0x5562ece383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47166/0 (socket says 172.21.15.17:47166) 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 -- 172.21.15.17:0/356926983 learned_addr learned my addr 172.21.15.17:0/356926983 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02709f7640 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02711f8640 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece399d0 0x5562ece5d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 -- 172.21.15.17:0/356926983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece399d0 msgr2=0x5562ece5d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece399d0 0x5562ece5d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 -- 172.21.15.17:0/356926983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 msgr2=0x5562ece4fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece4fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 -- 172.21.15.17:0/356926983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562ecd41a90 con 0x5562ece3ee50 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02711f8640 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece399d0 0x5562ece5d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f02719f9640 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece3ee50 0x5562ece383a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0264009940 tx=0x7f0264009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a1a42d83bf72b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f0256ffd640 1 -- 172.21.15.17:0/356926983 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264023040 con 0x5562ece3ee50 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f0256ffd640 1 -- 172.21.15.17:0/356926983 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0264009d20 con 0x5562ece3ee50 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.120+0000 7f0256ffd640 1 -- 172.21.15.17:0/356926983 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264028890 con 0x5562ece3ee50 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- 172.21.15.17:0/356926983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece3ee50 msgr2=0x5562ece383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece3ee50 0x5562ece383a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0264009940 tx=0x7f0264009c10 comp rx=0 tx=0).stop 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- 172.21.15.17:0/356926983 shutdown_connections 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece3ee50 0x5562ece383a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3e460 0x5562ece4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 --2- 172.21.15.17:0/356926983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece399d0 0x5562ece5d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- 172.21.15.17:0/356926983 >> 172.21.15.17:0/356926983 conn(0x5562ece3a120 msgr2=0x5562ece5ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- 172.21.15.17:0/356926983 shutdown_connections 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- 172.21.15.17:0/356926983 wait complete. 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- start start 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 Processor -- start 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece399d0 0x5562ece4c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece4cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562eceaab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ece4fd40 con 0x5562ece399d0 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.121+0000 7f027602e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ece4feb0 con 0x5562ece3ee50 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f027602e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562eceab070 con 0x5562ece3e460 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f027602e740 10 client.0 _ll_register_callbacks cb 0x7f02640099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece399d0 0x5562ece4c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02709f7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece4cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece399d0 0x5562ece4c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47174/0 (socket says 172.21.15.17:47174) 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 -- 172.21.15.17:0/3700114728 learned_addr learned my addr 172.21.15.17:0/3700114728 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02719f9640 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562eceaab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 -- 172.21.15.17:0/3700114728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 msgr2=0x5562ece4cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece4cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 -- 172.21.15.17:0/3700114728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 msgr2=0x5562eceaab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562eceaab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 -- 172.21.15.17:0/3700114728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562ecda5040 con 0x5562ece399d0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02719f9640 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562eceaab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f02711f8640 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece399d0 0x5562ece4c830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0268096a10 tx=0x7f02680ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cce73243899b7b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f0254ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f0254ff9640 1 -- 172.21.15.17:0/3700114728 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026809e700 con 0x5562ece399d0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f0254ff9640 1 -- 172.21.15.17:0/3700114728 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f02680b0070 con 0x5562ece399d0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562ece553c0 con 0x5562ece399d0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f0254ff9640 1 -- 172.21.15.17:0/3700114728 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02680a2d50 con 0x5562ece399d0 2024-02-14T15:02:39.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f027602e740 10 client.15261 fetch_fsmap learned FSMap version 23 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.122+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5562ecd67960 con 0x5562ece399d0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f0254ff9640 1 -- 172.21.15.17:0/3700114728 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5562ece553c0 con 0x5562ece399d0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f0254ff9640 1 -- 172.21.15.17:0/3700114728 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f02680b8d20 con 0x5562ece399d0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f027602e740 10 client.15261 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f027602e740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f027602e740 20 client.15261 populate_metadata read hostname 'smithi017' 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f027602e740 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece43fc0 0x5562eceac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f027602e740 4 client.15261 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f0264010d50 con 0x5562ece43fc0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.123+0000 7f02709f7640 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece43fc0 0x5562eceac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f0254ff9640 10 client.15261 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f02709f7640 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece43fc0 0x5562eceac670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c6b2876ccaf58ac server_cookie=5c88b01cb1a3a84 in_seq=0 out_seq=0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f0254ff9640 1 -- 172.21.15.17:0/3700114728 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f0264010d50 con 0x5562ece43fc0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f0254ff9640 10 client.15261 handle_command_reply: tid=0 2024-02-14T15:02:39.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 client.15261 shutdown 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece399d0 msgr2=0x5562ece4c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece399d0 0x5562ece4c830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0268096a10 tx=0x7f02680ba040 comp rx=0 tx=0).stop 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 shutdown_connections 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5562ece43fc0 0x5562eceac670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ece3ee50 0x5562eceaab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f0254ff9640 0 client.15261 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562ece3e460 0x5562ece4cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 --2- 172.21.15.17:0/3700114728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5562ece399d0 0x5562ece4c830 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 >> 172.21.15.17:0/3700114728 conn(0x5562ece3a120 msgr2=0x5562ece5d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 shutdown_connections 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 1 -- 172.21.15.17:0/3700114728 wait complete. 2024-02-14T15:02:39.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.124+0000 7f027602e740 20 client.15261 trim_cache size 0 max 16384 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 2, 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24481, 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 84.545615533000003, 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 23, 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-02-14T15:02:39.133 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 130.32967326799999 2024-02-14T15:02:39.134 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:02:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02580a21d0 msgr2=0x7f02580a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02580a21d0 0x7f02580a4690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f025c009230 tx=0x7f025c005fd0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c101240 msgr2=0x7f026c074fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c101240 0x7f026c074fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f026800d980 tx=0x7f026800de40 comp rx=0 tx=0).stop 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 shutdown_connections 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02580a21d0 0x7f02580a4690 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f026c102df0 0x7f026c073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f026c102430 0x7f026c073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 --2- 172.21.15.17:0/3431634214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026c101240 0x7f026c074fe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 >> 172.21.15.17:0/3431634214 conn(0x7f026c0fc7b0 msgr2=0x7f026c105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 shutdown_connections 2024-02-14T15:02:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.125+0000 7f027345c640 1 -- 172.21.15.17:0/3431634214 wait complete. 2024-02-14T15:02:39.137 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- 172.21.15.17:0/2677761894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734101d00 msgr2=0x7f9734108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2677761894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734101d00 0x7f9734108590 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9724009900 tx=0x7f972402bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- 172.21.15.17:0/2677761894 shutdown_connections 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2677761894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734101d00 0x7f9734108590 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2677761894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9734101340 0x7f97341017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2677761894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9734100150 0x7f9734100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- 172.21.15.17:0/2677761894 >> 172.21.15.17:0/2677761894 conn(0x7f97340fb6e0 msgr2=0x7f97340fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- 172.21.15.17:0/2677761894 shutdown_connections 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- 172.21.15.17:0/2677761894 wait complete. 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 Processor -- start 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- start start 2024-02-14T15:02:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734100150 0x7f97341993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9734101340 0x7f9734199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9734101d00 0x7f97341a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97341a0eb0 con 0x7f9734100150 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97341a1020 con 0x7f9734101d00 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.239+0000 7f973b1e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97341a11e0 con 0x7f9734101340 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734100150 0x7f97341993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734100150 0x7f97341993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47192/0 (socket says 172.21.15.17:47192) 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 -- 172.21.15.17:0/2977025530 learned_addr learned my addr 172.21.15.17:0/2977025530 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9733fff640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9734101340 0x7f9734199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 -- 172.21.15.17:0/2977025530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9734101340 msgr2=0x7f9734199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9734101340 0x7f9734199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 -- 172.21.15.17:0/2977025530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9734101d00 msgr2=0x7f97341a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f973977f640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9734101d00 0x7f97341a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9734101d00 0x7f97341a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 -- 172.21.15.17:0/2977025530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9724009560 con 0x7f9734100150 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9738f7e640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734100150 0x7f97341993c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f972c00ccb0 tx=0x7f972c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9731ffb640 1 -- 172.21.15.17:0/2977025530 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c003b70 con 0x7f9734100150 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9731ffb640 1 -- 172.21.15.17:0/2977025530 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f972c00ae50 con 0x7f9734100150 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97341a14c0 con 0x7f9734100150 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f9731ffb640 1 -- 172.21.15.17:0/2977025530 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c00f7d0 con 0x7f9734100150 2024-02-14T15:02:39.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.240+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97341a19d0 con 0x7f9734100150 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.242+0000 7f9731ffb640 1 -- 172.21.15.17:0/2977025530 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f972c004000 con 0x7f9734100150 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 Processor -- start 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 -- start start 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f9731ffb640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f971c0a2150 0x7f971c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6a460 0x5583c3f6a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f6f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ffc0 0x5583c3f72450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3f26fd0 con 0x5583c3f6ae50 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3ed9cb0 con 0x5583c3f6a460 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973ddb9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3ed71e0 con 0x5583c3f6ffc0 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f9731ffb640 1 -- 172.21.15.17:0/2977025530 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f972c0ce600 con 0x7f9734100150 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f973977f640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f971c0a2150 0x7f971c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f9733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f6f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f9733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f6f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47196/0 (socket says 172.21.15.17:47196) 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.243+0000 7f9733fff640 1 -- 172.21.15.17:0/1459544844 learned_addr learned my addr 172.21.15.17:0/1459544844 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6a460 0x5583c3f6a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f9733fff640 1 -- 172.21.15.17:0/1459544844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ffc0 msgr2=0x5583c3f72450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f9733fff640 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ffc0 0x5583c3f72450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f9733fff640 1 -- 172.21.15.17:0/1459544844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6a460 msgr2=0x5583c3f6a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f9733fff640 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6a460 0x5583c3f6a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f9733fff640 1 -- 172.21.15.17:0/1459544844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583c3e95100 con 0x5583c3f6ae50 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f973977f640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f971c0a2150 0x7f971c0a4610 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f972402c6a0 tx=0x7f972402c8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f9733fff640 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f6f950 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f972800bc30 tx=0x7f97280107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b76bfcc547acd17 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f971affd640 1 -- 172.21.15.17:0/1459544844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728004030 con 0x5583c3f6ae50 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f971affd640 1 -- 172.21.15.17:0/1459544844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9728015070 con 0x5583c3f6ae50 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.244+0000 7f971affd640 1 -- 172.21.15.17:0/1459544844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280116a0 con 0x5583c3f6ae50 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- 172.21.15.17:0/1459544844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 msgr2=0x5583c3f6f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f6f950 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f972800bc30 tx=0x7f97280107a0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- 172.21.15.17:0/1459544844 shutdown_connections 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f6f950 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ffc0 0x5583c3f72450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1459544844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6a460 0x5583c3f6a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- 172.21.15.17:0/1459544844 >> 172.21.15.17:0/1459544844 conn(0x5583c3f659d0 msgr2=0x5583c3f65dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- 172.21.15.17:0/1459544844 shutdown_connections 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- 172.21.15.17:0/1459544844 wait complete. 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- start start 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 Processor -- start 2024-02-14T15:02:39.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f7bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3ed29c0 con 0x5583c3f6a460 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3e77cd0 con 0x5583c3f6ffc0 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.245+0000 7f973ddb9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3ba6310 con 0x5583c3f6ae50 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f973ddb9740 10 client.0 _ll_register_callbacks cb 0x5583c3be3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47212/0 (socket says 172.21.15.17:47212) 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 -- 172.21.15.17:0/1186841821 learned_addr learned my addr 172.21.15.17:0/1186841821 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f973977f640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f7bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9733fff640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 -- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 msgr2=0x5583c3f74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 -- 172.21.15.17:0/1186841821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 msgr2=0x5583c3f7bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f7bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 -- 172.21.15.17:0/1186841821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972800b8e0 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f973977f640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f7bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f745e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f972c005fd0 tx=0x7f972c049d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a720bc360fecaa server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9713fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9713fff640 1 -- 172.21.15.17:0/1186841821 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c09b5b0 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9713fff640 1 -- 172.21.15.17:0/1186841821 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f972c09aeb0 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583c3bd26c0 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f9713fff640 1 -- 172.21.15.17:0/1186841821 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f972c09f000 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f973ddb9740 10 client.15273 fetch_fsmap learned FSMap version 23 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.246+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5583c3ed1040 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f9713fff640 1 -- 172.21.15.17:0/1186841821 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583c3bd26c0 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f9713fff640 1 -- 172.21.15.17:0/1186841821 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f972c020070 con 0x5583c3f6a460 2024-02-14T15:02:39.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f973ddb9740 10 client.15273 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f973ddb9740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f973ddb9740 20 client.15273 populate_metadata read hostname 'smithi017' 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3f80ba0 0x5583c3f7fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f973ddb9740 4 client.15273 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583c3c23270 con 0x5583c3f80ba0 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f9733fff640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3f80ba0 0x5583c3f7fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f9713fff640 10 client.15273 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.247+0000 7f9733fff640 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3f80ba0 0x5583c3f7fe50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eeb3ef2f936c3937 server_cookie=bf1d760751ecc5e5 in_seq=0 out_seq=0 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f9713fff640 1 -- 172.21.15.17:0/1186841821 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583c3c23270 con 0x5583c3f80ba0 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f9713fff640 10 client.15273 handle_command_reply: tid=0 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 client.15273 shutdown 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 msgr2=0x5583c3f745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f745e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f972c005fd0 tx=0x7f972c049d70 comp rx=0 tx=0).stop 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 shutdown_connections 2024-02-14T15:02:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3f80ba0 0x5583c3f7fe50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f7bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f9713fff640 0 client.15273 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1186841821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f745e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.248+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 >> 172.21.15.17:0/1186841821 conn(0x5583c3f659d0 msgr2=0x5583c3f64c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.249+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 shutdown_connections 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.249+0000 7f973ddb9740 1 -- 172.21.15.17:0/1186841821 wait complete. 2024-02-14T15:02:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.249+0000 7f973ddb9740 20 client.15273 trim_cache size 0 max 16384 2024-02-14T15:02:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 Processor -- start 2024-02-14T15:02:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 -- start start 2024-02-14T15:02:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f7bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f65db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3e58490 con 0x5583c3f6a460 2024-02-14T15:02:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3ecf620 con 0x5583c3f6ffc0 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.264+0000 7f973ddb9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3e788e0 con 0x5583c3f6ae50 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f73190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47216/0 (socket says 172.21.15.17:47216) 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 -- 172.21.15.17:0/2710140902 learned_addr learned my addr 172.21.15.17:0/2710140902 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 -- 172.21.15.17:0/2710140902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 msgr2=0x5583c3f7bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9733fff640 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f7bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f973977f640 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f65db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f7bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 -- 172.21.15.17:0/2710140902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 msgr2=0x5583c3f65db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f65db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 -- 172.21.15.17:0/2710140902 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583c3e6da90 con 0x5583c3f6a460 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9733fff640 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f7bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f973977f640 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f65db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f9738f7e640 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f73190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f972c016000 tx=0x7f972c09ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b701b05ac3be5437 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f971a7fc640 1 -- 172.21.15.17:0/2710140902 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c003cd0 con 0x5583c3f6a460 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f971a7fc640 1 -- 172.21.15.17:0/2710140902 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f972c09b5b0 con 0x5583c3f6a460 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.265+0000 7f971a7fc640 1 -- 172.21.15.17:0/2710140902 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c097520 con 0x5583c3f6a460 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- 172.21.15.17:0/2710140902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 msgr2=0x5583c3f73190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f73190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f972c016000 tx=0x7f972c09ad90 comp rx=0 tx=0).stop 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- 172.21.15.17:0/2710140902 shutdown_connections 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3f65db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6ae50 0x5583c3f7bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 --2- 172.21.15.17:0/2710140902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6a460 0x5583c3f73190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- 172.21.15.17:0/2710140902 >> 172.21.15.17:0/2710140902 conn(0x5583c3f659d0 msgr2=0x5583c3f6b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- 172.21.15.17:0/2710140902 shutdown_connections 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- 172.21.15.17:0/2710140902 wait complete. 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- start start 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 Processor -- start 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6a460 0x5583c3f786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3fd67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3f7bc10 con 0x5583c3f6ae50 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.266+0000 7f973ddb9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3f7bd80 con 0x5583c3f6ffc0 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f973ddb9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583c3fd6ce0 con 0x5583c3f6a460 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f973ddb9740 10 client.0 _ll_register_callbacks cb 0x7f972800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f78bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47232/0 (socket says 172.21.15.17:47232) 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 -- 172.21.15.17:0/1918980120 learned_addr learned my addr 172.21.15.17:0/1918980120 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6a460 0x5583c3f786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 -- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6a460 msgr2=0x5583c3f786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6a460 0x5583c3f786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 -- 172.21.15.17:0/1918980120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 msgr2=0x5583c3fd67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3fd67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 -- 172.21.15.17:0/1918980120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583c3ed1040 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6a460 0x5583c3f786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9733fff640 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f78bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5583c3f6b8c0 tx=0x7f972800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46538d4315a9548 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9713fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9713fff640 1 -- 172.21.15.17:0/1918980120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728005450 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9713fff640 1 -- 172.21.15.17:0/1918980120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9728017070 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9728015070 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f9713fff640 1 -- 172.21.15.17:0/1918980120 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97280055b0 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f973ddb9740 10 client.15279 fetch_fsmap learned FSMap version 23 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.267+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5583c3e93960 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f9713fff640 1 -- 172.21.15.17:0/1918980120 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9728015070 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f9713fff640 1 -- 172.21.15.17:0/1918980120 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9728011440 con 0x5583c3f6ae50 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f973ddb9740 10 client.15279 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:02:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f973ddb9740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f973ddb9740 20 client.15279 populate_metadata read hostname 'smithi017' 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3fdbee0 0x5583c3fd8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f973ddb9740 4 client.15279 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5583c3c0f610 con 0x5583c3fdbee0 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.268+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3fdbee0 0x5583c3fd8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f9738f7e640 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3fdbee0 0x5583c3fd8300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=811f3455cacf3134 server_cookie=4de9a07912a7a47e in_seq=0 out_seq=0 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f9713fff640 10 client.15279 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f9713fff640 1 -- 172.21.15.17:0/1918980120 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5583c3c0f610 con 0x5583c3fdbee0 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f9713fff640 10 client.15279 handle_command_reply: tid=0 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 client.15279 shutdown 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 msgr2=0x5583c3f78bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f78bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5583c3f6b8c0 tx=0x7f972800b860 comp rx=0 tx=0).stop 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 shutdown_connections 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5583c3fdbee0 0x5583c3fd8300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583c3f6ae50 0x5583c3f78bf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583c3f6ffc0 0x5583c3fd67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f9713fff640 0 client.15279 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 --2- 172.21.15.17:0/1918980120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5583c3f6a460 0x5583c3f786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 >> 172.21.15.17:0/1918980120 conn(0x5583c3f659d0 msgr2=0x5583c3f6f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 shutdown_connections 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 1 -- 172.21.15.17:0/1918980120 wait complete. 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.269+0000 7f973ddb9740 20 client.15279 trim_cache size 0 max 16384 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2024-02-14T15:02:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24502, 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 85.697015871999994, 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 23, 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 124.807571821 2024-02-14T15:02:39.279 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:02:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f971c0a2150 msgr2=0x7f971c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f971c0a2150 0x7f971c0a4610 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f972402c6a0 tx=0x7f972402c8e0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734100150 msgr2=0x7f97341993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734100150 0x7f97341993c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f972c00ccb0 tx=0x7f972c00a620 comp rx=0 tx=0).stop 2024-02-14T15:02:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 shutdown_connections 2024-02-14T15:02:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f971c0a2150 0x7f971c0a4610 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9734101d00 0x7f97341a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9734101340 0x7f9734199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 --2- 172.21.15.17:0/2977025530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9734100150 0x7f97341993c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 >> 172.21.15.17:0/2977025530 conn(0x7f97340fb6e0 msgr2=0x7f97340fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 shutdown_connections 2024-02-14T15:02:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.270+0000 7f973b1e2640 1 -- 172.21.15.17:0/2977025530 wait complete. 2024-02-14T15:02:39.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.383+0000 7f9d38aae640 1 -- 172.21.15.17:0/1997368039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101340 msgr2=0x7f9d341017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.383+0000 7f9d38aae640 1 --2- 172.21.15.17:0/1997368039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101340 0x7f9d341017c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009900 tx=0x7f9d1c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- 172.21.15.17:0/1997368039 shutdown_connections 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 --2- 172.21.15.17:0/1997368039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101340 0x7f9d341017c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 --2- 172.21.15.17:0/1997368039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d34101d00 0x7f9d34108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 --2- 172.21.15.17:0/1997368039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d34100150 0x7f9d34100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- 172.21.15.17:0/1997368039 >> 172.21.15.17:0/1997368039 conn(0x7f9d340fb6c0 msgr2=0x7f9d340fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- 172.21.15.17:0/1997368039 shutdown_connections 2024-02-14T15:02:39.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- 172.21.15.17:0/1997368039 wait complete. 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 Processor -- start 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- start start 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d34100150 0x7f9d34197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d34101340 0x7f9d34197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101d00 0x7f9d3419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3419ee20 con 0x7f9d34101d00 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3419ef90 con 0x7f9d34100150 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.384+0000 7f9d38aae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3419f150 con 0x7f9d34101340 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101d00 0x7f9d3419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101d00 0x7f9d3419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47242/0 (socket says 172.21.15.17:47242) 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 -- 172.21.15.17:0/679843635 learned_addr learned my addr 172.21.15.17:0/679843635 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d34101340 0x7f9d34197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 -- 172.21.15.17:0/679843635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d34101340 msgr2=0x7f9d34197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d3259c640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d34100150 0x7f9d34197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d34101340 0x7f9d34197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 -- 172.21.15.17:0/679843635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d34100150 msgr2=0x7f9d34197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d34100150 0x7f9d34197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 -- 172.21.15.17:0/679843635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009560 con 0x7f9d34101d00 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d34101340 0x7f9d34197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d3259c640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d34100150 0x7f9d34197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101d00 0x7f9d3419e8e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400cad0 tx=0x7f9d2400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d2b7fe640 1 -- 172.21.15.17:0/679843635 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400ae20 con 0x7f9d34101d00 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d2b7fe640 1 -- 172.21.15.17:0/679843635 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d24005020 con 0x7f9d34101d00 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3419f430 con 0x7f9d34101d00 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d2b7fe640 1 -- 172.21.15.17:0/679843635 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400f6b0 con 0x7f9d34101d00 2024-02-14T15:02:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.385+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3419f8f0 con 0x7f9d34101d00 2024-02-14T15:02:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d2b7fe640 1 -- 172.21.15.17:0/679843635 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f9d24005520 con 0x7f9d34101d00 2024-02-14T15:02:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 Processor -- start 2024-02-14T15:02:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 -- start start 2024-02-14T15:02:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8a460 0x555611f8a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d2b7fe640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9d200a2150 0x7f9d200a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f92450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611f46fd0 con 0x555611f8ffc0 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611ef9cb0 con 0x555611f8ae50 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611ef71e0 con 0x555611f8a460 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d3259c640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9d200a2150 0x7f9d200a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d32d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f92450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d31d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8a460 0x555611f8a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d31d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8a460 0x555611f8a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:44758/0 (socket says 172.21.15.17:44758) 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.388+0000 7f9d31d9b640 1 -- 172.21.15.17:0/1982976506 learned_addr learned my addr 172.21.15.17:0/1982976506 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d2b7fe640 1 -- 172.21.15.17:0/679843635 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d240ce270 con 0x7f9d34101d00 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d32d9d640 1 -- 172.21.15.17:0/1982976506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8a460 msgr2=0x555611f8a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8a460 0x555611f8a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d32d9d640 1 -- 172.21.15.17:0/1982976506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 msgr2=0x555611f8f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f8f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d32d9d640 1 -- 172.21.15.17:0/1982976506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555611eb5100 con 0x555611f8ffc0 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d3259c640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9d200a2150 0x7f9d200a4610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9d341011a0 tx=0x7f9d2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f92450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9d240cb8c0 tx=0x7f9d2409a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91e66819140b102 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d28ff9640 1 -- 172.21.15.17:0/1982976506 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240b3810 con 0x555611f8ffc0 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d28ff9640 1 -- 172.21.15.17:0/1982976506 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d2409b1e0 con 0x555611f8ffc0 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.389+0000 7f9d28ff9640 1 -- 172.21.15.17:0/1982976506 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240b3bd0 con 0x555611f8ffc0 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- 172.21.15.17:0/1982976506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 msgr2=0x555611f92450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f92450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9d240cb8c0 tx=0x7f9d2409a620 comp rx=0 tx=0).stop 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- 172.21.15.17:0/1982976506 shutdown_connections 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f92450 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/1982976506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8a460 0x555611f8a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- 172.21.15.17:0/1982976506 >> 172.21.15.17:0/1982976506 conn(0x555611f859d0 msgr2=0x555611f85dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- 172.21.15.17:0/1982976506 shutdown_connections 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- 172.21.15.17:0/1982976506 wait complete. 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- start start 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 Processor -- start 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8a460 0x555611f94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f94bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f9bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611ef29c0 con 0x555611f8ffc0 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611e97cd0 con 0x555611f8a460 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.390+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611bc6310 con 0x555611f8ae50 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d3b67f740 10 client.0 _ll_register_callbacks cb 0x555611c03ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f9bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f9bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47256/0 (socket says 172.21.15.17:47256) 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 -- 172.21.15.17:0/831573115 learned_addr learned my addr 172.21.15.17:0/831573115 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d3259c640 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8a460 0x555611f94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f94bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 -- 172.21.15.17:0/831573115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 msgr2=0x555611f94bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f94bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 -- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8a460 msgr2=0x555611f94680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8a460 0x555611f94680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 -- 172.21.15.17:0/831573115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3419f8f0 con 0x555611f8ffc0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f9bc00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9d240cb8f0 tx=0x7f9d240cbd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cc649df2955656 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d19ffb640 1 -- 172.21.15.17:0/831573115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240053a0 con 0x555611f8ffc0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d19ffb640 1 -- 172.21.15.17:0/831573115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d2409ec30 con 0x555611f8ffc0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555611bf26c0 con 0x555611f8ffc0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d19ffb640 1 -- 172.21.15.17:0/831573115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d240ce030 con 0x555611f8ffc0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d3b67f740 10 client.15291 fetch_fsmap learned FSMap version 23 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.391+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555611ef1040 con 0x555611f8ffc0 2024-02-14T15:02:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d19ffb640 1 -- 172.21.15.17:0/831573115 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555611bf26c0 con 0x555611f8ffc0 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d19ffb640 1 -- 172.21.15.17:0/831573115 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d24016030 con 0x555611f8ffc0 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3b67f740 10 client.15291 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3b67f740 10 client.15291 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3b67f740 20 client.15291 populate_metadata read hostname 'smithi017' 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611fa0150 0x555611fa4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3b67f740 4 client.15291 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555611c43270 con 0x555611fa0150 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3259c640 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611fa0150 0x555611fa4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d3259c640 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611fa0150 0x555611fa4bd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1564191c6b7c79 server_cookie=a5b68410cc595570 in_seq=0 out_seq=0 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.392+0000 7f9d19ffb640 10 client.15291 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d19ffb640 1 -- 172.21.15.17:0/831573115 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555611c43270 con 0x555611fa0150 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d19ffb640 10 client.15291 handle_command_reply: tid=0 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 client.15291 shutdown 2024-02-14T15:02:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 msgr2=0x555611f9bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f9bc00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9d240cb8f0 tx=0x7f9d240cbd40 comp rx=0 tx=0).stop 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 shutdown_connections 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611fa0150 0x555611fa4bd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8ffc0 0x555611f9bc00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f94bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d19ffb640 0 client.15291 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/831573115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8a460 0x555611f94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 >> 172.21.15.17:0/831573115 conn(0x555611f859d0 msgr2=0x555611f84c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 shutdown_connections 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 1 -- 172.21.15.17:0/831573115 wait complete. 2024-02-14T15:02:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.393+0000 7f9d3b67f740 20 client.15291 trim_cache size 0 max 16384 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 Processor -- start 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 -- start start 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f8c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f8c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ffc0 0x555611f9bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611e78490 con 0x555611f8a460 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611f9cbf0 con 0x555611f8ffc0 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611f9c4c0 con 0x555611f8ae50 2024-02-14T15:02:39.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f8c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f8c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47270/0 (socket says 172.21.15.17:47270) 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d3259c640 1 -- 172.21.15.17:0/4245772256 learned_addr learned my addr 172.21.15.17:0/4245772256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.409+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f8c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3259c640 1 -- 172.21.15.17:0/4245772256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 msgr2=0x555611f8c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3259c640 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f8c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3259c640 1 -- 172.21.15.17:0/4245772256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ffc0 msgr2=0x555611f9bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3259c640 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ffc0 0x555611f9bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3259c640 1 -- 172.21.15.17:0/4245772256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555611e8da90 con 0x555611f8a460 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3259c640 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f8c3e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x555611fa2140 tx=0x7f9d2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f067ce44a07fb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d1bfff640 1 -- 172.21.15.17:0/4245772256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c028de0 con 0x555611f8a460 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d1bfff640 1 -- 172.21.15.17:0/4245772256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d2c012e30 con 0x555611f8a460 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d1bfff640 1 -- 172.21.15.17:0/4245772256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c020bf0 con 0x555611f8a460 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3b67f740 1 -- 172.21.15.17:0/4245772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 msgr2=0x555611f8c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f8c3e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x555611fa2140 tx=0x7f9d2c016000 comp rx=0 tx=0).stop 2024-02-14T15:02:39.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3b67f740 1 -- 172.21.15.17:0/4245772256 shutdown_connections 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ffc0 0x555611f9bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ae50 0x555611f8c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/4245772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f8c3e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.410+0000 7f9d3b67f740 1 -- 172.21.15.17:0/4245772256 >> 172.21.15.17:0/4245772256 conn(0x555611f859d0 msgr2=0x555611f8b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 -- 172.21.15.17:0/4245772256 shutdown_connections 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 -- 172.21.15.17:0/4245772256 wait complete. 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 -- start start 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 Processor -- start 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f98650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f98b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ffc0 0x555611ff6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611f9bbb0 con 0x555611f8a460 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611f9bd20 con 0x555611f8ae50 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555611f9bee0 con 0x555611f8ffc0 2024-02-14T15:02:39.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3b67f740 10 client.0 _ll_register_callbacks cb 0x7f9d24096660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f98650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f98650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47276/0 (socket says 172.21.15.17:47276) 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 -- 172.21.15.17:0/418323658 learned_addr learned my addr 172.21.15.17:0/418323658 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d32d9d640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ffc0 0x555611ff6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f98b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 -- 172.21.15.17:0/418323658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ffc0 msgr2=0x555611ff6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ffc0 0x555611ff6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 -- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 msgr2=0x555611f98b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f98b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d3259c640 1 -- 172.21.15.17:0/418323658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c010e90 con 0x555611f8a460 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.411+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f98b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3259c640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f98650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c013fd0 tx=0x7f9d2c028c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327e39b730060afd server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d19ffb640 1 -- 172.21.15.17:0/418323658 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c020d80 con 0x555611f8a460 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d19ffb640 1 -- 172.21.15.17:0/418323658 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9d2c025070 con 0x555611f8a460 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555611fa1780 con 0x555611f8a460 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d19ffb640 1 -- 172.21.15.17:0/418323658 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d2c005050 con 0x555611f8a460 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 10 client.15297 fetch_fsmap learned FSMap version 23 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555611eb3960 con 0x555611f8a460 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d19ffb640 1 -- 172.21.15.17:0/418323658 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555611fa1780 con 0x555611f8a460 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d19ffb640 1 -- 172.21.15.17:0/418323658 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d2c036030 con 0x555611f8a460 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 10 client.15297 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 10 client.15297 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 20 client.15297 populate_metadata read hostname 'smithi017' 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611ffbd10 0x555611ff8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 4 client.15297 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9d2409b1e0 con 0x555611ffbd10 2024-02-14T15:02:39.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.412+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611ffbd10 0x555611ff8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d31d9b640 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611ffbd10 0x555611ff8230 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faabd0dc618f0880 server_cookie=b051e92261fccd86 in_seq=0 out_seq=0 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d19ffb640 10 client.15297 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d19ffb640 1 -- 172.21.15.17:0/418323658 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f9d2409b1e0 con 0x555611ffbd10 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d19ffb640 10 client.15297 handle_command_reply: tid=0 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 client.15297 shutdown 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 msgr2=0x555611f98650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f98650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c013fd0 tx=0x7f9d2c028c70 comp rx=0 tx=0).stop 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 shutdown_connections 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555611ffbd10 0x555611ff8230 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555611f8ffc0 0x555611ff6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555611f8ae50 0x555611f98b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 --2- 172.21.15.17:0/418323658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555611f8a460 0x555611f98650 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d19ffb640 0 client.15297 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:02:39.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 >> 172.21.15.17:0/418323658 conn(0x555611f859d0 msgr2=0x555611f8c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 shutdown_connections 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 1 -- 172.21.15.17:0/418323658 wait complete. 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.413+0000 7f9d3b67f740 20 client.15297 trim_cache size 0 max 16384 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24509, 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-02-14T15:02:39.423 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 110.150124291, 2024-02-14T15:02:39.424 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 23, 2024-02-14T15:02:39.424 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 91, 2024-02-14T15:02:39.424 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-02-14T15:02:39.424 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 116.258858436 2024-02-14T15:02:39.424 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9d200a2150 msgr2=0x7f9d200a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9d200a2150 0x7f9d200a4610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9d341011a0 tx=0x7f9d2c005f90 comp rx=0 tx=0).stop 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101d00 msgr2=0x7f9d3419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101d00 0x7f9d3419e8e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400cad0 tx=0x7f9d2400cf90 comp rx=0 tx=0).stop 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 shutdown_connections 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9d200a2150 0x7f9d200a4610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34101d00 0x7f9d3419e8e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d34101340 0x7f9d34197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 --2- 172.21.15.17:0/679843635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d34100150 0x7f9d34197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 >> 172.21.15.17:0/679843635 conn(0x7f9d340fb6c0 msgr2=0x7f9d34104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 shutdown_connections 2024-02-14T15:02:39.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:39.414+0000 7f9d38aae640 1 -- 172.21.15.17:0/679843635 wait complete. 2024-02-14T15:02:39.426 INFO:tasks.cephadm:Setup complete, yielding 2024-02-14T15:02:39.427 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-14T15:02:39.438 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:02:39.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- bash -c 'ceph orch status' 2024-02-14T15:02:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:39 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/11263964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:39 smithi050 ceph-mon[36512]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:39.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:39 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4074896918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:39.927 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:39 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/11263964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:39.927 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:39 smithi017 ceph-mon[32549]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:39.927 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:39 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4074896918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:39 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/11263964' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:02:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:39 smithi076 ceph-mon[37057]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:02:39.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:39 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4074896918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:02:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/1470228698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc381005d0 msgr2=0x7fcc381009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/1470228698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc381005d0 0x7fcc381009f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009990 tx=0x7fcc28009ca0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/1470228698 shutdown_connections 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/1470228698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc381018f0 0x7fcc38101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/1470228698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc381005d0 0x7fcc381009f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/1470228698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc38100f30 0x7fcc381013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/1470228698 >> 172.21.15.17:0/1470228698 conn(0x7fcc380fc1f0 msgr2=0x7fcc380fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/1470228698 shutdown_connections 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/1470228698 wait complete. 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.491+0000 7fcc3dbb3700 1 Processor -- start 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc3dbb3700 1 -- start start 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc3dbb3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc381005d0 0x7fcc38199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc3dbb3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc38100f30 0x7fcc3819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc3dbb3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc381018f0 0x7fcc381a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc3dbb3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc381a1b10 con 0x7fcc38100f30 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc3dbb3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc381a1c50 con 0x7fcc381005d0 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc3dbb3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc381a1de0 con 0x7fcc381018f0 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc38100f30 0x7fcc3819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc38100f30 0x7fcc3819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46910/0 (socket says 172.21.15.17:46910) 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 -- 172.21.15.17:0/637950568 learned_addr learned my addr 172.21.15.17:0/637950568 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc37fff700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc381018f0 0x7fcc381a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc377fe700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc381005d0 0x7fcc38199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 -- 172.21.15.17:0/637950568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc381018f0 msgr2=0x7fcc381a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc381018f0 0x7fcc381a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 -- 172.21.15.17:0/637950568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc381005d0 msgr2=0x7fcc38199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc381005d0 0x7fcc38199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 -- 172.21.15.17:0/637950568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280095f0 con 0x7fcc38100f30 2024-02-14T15:02:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.492+0000 7fcc36ffd700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc38100f30 0x7fcc3819a4c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000d980 tx=0x7fcc2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.493+0000 7fcc34ff9700 1 -- 172.21.15.17:0/637950568 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200098d0 con 0x7fcc38100f30 2024-02-14T15:02:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.493+0000 7fcc34ff9700 1 -- 172.21.15.17:0/637950568 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc20010460 con 0x7fcc38100f30 2024-02-14T15:02:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.493+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc381a20c0 con 0x7fcc38100f30 2024-02-14T15:02:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.493+0000 7fcc34ff9700 1 -- 172.21.15.17:0/637950568 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000f620 con 0x7fcc38100f30 2024-02-14T15:02:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.493+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc381a2610 con 0x7fcc38100f30 2024-02-14T15:02:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.494+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3810b3b0 con 0x7fcc38100f30 2024-02-14T15:02:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.496+0000 7fcc34ff9700 1 -- 172.21.15.17:0/637950568 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fcc200108f0 con 0x7fcc38100f30 2024-02-14T15:02:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.496+0000 7fcc34ff9700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc240a2260 0x7fcc240a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.496+0000 7fcc377fe700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc240a2260 0x7fcc240a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.496+0000 7fcc34ff9700 1 -- 172.21.15.17:0/637950568 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcc200cdf70 con 0x7fcc38100f30 2024-02-14T15:02:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.497+0000 7fcc377fe700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc240a2260 0x7fcc240a4720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800b5c0 tx=0x7fcc28003860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.500+0000 7fcc34ff9700 1 -- 172.21.15.17:0/637950568 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc20096ae0 con 0x7fcc38100f30 2024-02-14T15:02:41.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.721+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc3802d880 con 0x7fcc240a2260 2024-02-14T15:02:41.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.723+0000 7fcc34ff9700 1 -- 172.21.15.17:0/637950568 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fcc3802d880 con 0x7fcc240a2260 2024-02-14T15:02:41.774 INFO:teuthology.orchestra.run.smithi017.stdout:Backend: cephadm 2024-02-14T15:02:41.774 INFO:teuthology.orchestra.run.smithi017.stdout:Available: Yes 2024-02-14T15:02:41.774 INFO:teuthology.orchestra.run.smithi017.stdout:Paused: No 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc240a2260 msgr2=0x7fcc240a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc240a2260 0x7fcc240a4720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800b5c0 tx=0x7fcc28003860 comp rx=0 tx=0).stop 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc38100f30 msgr2=0x7fcc3819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc38100f30 0x7fcc3819a4c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000d980 tx=0x7fcc2000dc90 comp rx=0 tx=0).stop 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 shutdown_connections 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc240a2260 0x7fcc240a4720 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc381018f0 0x7fcc381a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc381005d0 0x7fcc38199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 --2- 172.21.15.17:0/637950568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc38100f30 0x7fcc3819a4c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 >> 172.21.15.17:0/637950568 conn(0x7fcc380fc1f0 msgr2=0x7fcc380fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 shutdown_connections 2024-02-14T15:02:41.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:41.725+0000 7fcc3dbb3700 1 -- 172.21.15.17:0/637950568 wait complete. 2024-02-14T15:02:42.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-14T15:02:42.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:42 smithi017 ceph-mon[32549]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:42.805 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:42.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:42 smithi050 ceph-mon[36512]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:42 smithi076 ceph-mon[37057]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:02:43.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:43 smithi050 ceph-mon[36512]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:43.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:43 smithi076 ceph-mon[37057]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:44.014 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:43 smithi017 ceph-mon[32549]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:44.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3279530550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d01027f0 msgr2=0x7f40d01090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3279530550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d01027f0 0x7f40d01090a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f40c000b0d0 tx=0x7f40c000b3e0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3279530550 shutdown_connections 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3279530550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d01027f0 0x7f40d01090a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3279530550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40d0100c50 0x7f40d0101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3279530550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d0101e50 0x7f40d01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3279530550 >> 172.21.15.17:0/3279530550 conn(0x7f40d00fc210 msgr2=0x7f40d00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3279530550 shutdown_connections 2024-02-14T15:02:44.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.573+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3279530550 wait complete. 2024-02-14T15:02:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 Processor -- start 2024-02-14T15:02:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 -- start start 2024-02-14T15:02:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d0100c50 0x7f40d019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d0101e50 0x7f40d019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40d01027f0 0x7f40d01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d01a1ba0 con 0x7f40d0100c50 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d01a1ce0 con 0x7f40d0101e50 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.574+0000 7f40d4bb6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d01a1e70 con 0x7f40d01027f0 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d0100c50 0x7f40d019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d0100c50 0x7f40d019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46940/0 (socket says 172.21.15.17:46940) 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 -- 172.21.15.17:0/3895407908 learned_addr learned my addr 172.21.15.17:0/3895407908 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ced9d700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40d01027f0 0x7f40d01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40cdd9b700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d0101e50 0x7f40d019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 -- 172.21.15.17:0/3895407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40d01027f0 msgr2=0x7f40d01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40d01027f0 0x7f40d01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 -- 172.21.15.17:0/3895407908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d0101e50 msgr2=0x7f40d019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d0101e50 0x7f40d019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 -- 172.21.15.17:0/3895407908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0009d00 con 0x7f40d0100c50 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40ce59c700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d0100c50 0x7f40d019a010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f40c800d9b0 tx=0x7f40c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8004020 con 0x7f40d0100c50 2024-02-14T15:02:44.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d01a2150 con 0x7f40d0100c50 2024-02-14T15:02:44.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40c8009730 con 0x7f40d0100c50 2024-02-14T15:02:44.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d01a26a0 con 0x7f40d0100c50 2024-02-14T15:02:44.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.575+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8003be0 con 0x7f40d0100c50 2024-02-14T15:02:44.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.576+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d00689e0 con 0x7f40d0100c50 2024-02-14T15:02:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.577+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f40c8009ba0 con 0x7f40d0100c50 2024-02-14T15:02:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.578+0000 7f40c77fe700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f40bc0a2260 0x7f40bc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.578+0000 7f40cdd9b700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f40bc0a2260 0x7f40bc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.578+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f40c80ce0a0 con 0x7f40d0100c50 2024-02-14T15:02:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.578+0000 7f40cdd9b700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f40bc0a2260 0x7f40bc0a4720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40b8005fd0 tx=0x7f40b800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.582+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40c8096c90 con 0x7f40d0100c50 2024-02-14T15:02:44.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.617+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f40c8003d40 con 0x7f40d0100c50 2024-02-14T15:02:44.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.802+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f40d00fe320 con 0x7f40bc0a2260 2024-02-14T15:02:44.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.816+0000 7f40c77fe700 1 -- 172.21.15.17:0/3895407908 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4092 (secure 0 0 0) 0x7f40d00fe320 con 0x7f40bc0a2260 2024-02-14T15:02:44.818 INFO:teuthology.orchestra.run.smithi017.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-14T15:02:44.818 INFO:teuthology.orchestra.run.smithi017.stdout:mds.a smithi017 running (2m) 20s ago 2m 13.8M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 5695e59a76e7 2024-02-14T15:02:44.818 INFO:teuthology.orchestra.run.smithi017.stdout:mds.b smithi050 running (2m) 55s ago 2m 25.5M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf c345db9c2b62 2024-02-14T15:02:44.818 INFO:teuthology.orchestra.run.smithi017.stdout:mds.c smithi076 running (2m) 55s ago 2m 16.0M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 2323ce9ebc75 2024-02-14T15:02:44.818 INFO:teuthology.orchestra.run.smithi017.stdout:mds.d smithi017 running (2m) 20s ago 2m 12.9M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 0686cf100b38 2024-02-14T15:02:44.819 INFO:teuthology.orchestra.run.smithi017.stdout:mds.e smithi050 running (2m) 55s ago 2m 17.4M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 6b8e546e9b1e 2024-02-14T15:02:44.819 INFO:teuthology.orchestra.run.smithi017.stdout:mds.f smithi076 running (2m) 55s ago 2m 15.9M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf fbbd60dda08a 2024-02-14T15:02:44.819 INFO:teuthology.orchestra.run.smithi017.stdout:mds.g smithi017 running (2m) 20s ago 2m 15.1M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf f8d1ceeaf43d 2024-02-14T15:02:44.819 INFO:teuthology.orchestra.run.smithi017.stdout:mds.h smithi050 running (2m) 55s ago 2m 18.3M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 62c016a31def 2024-02-14T15:02:44.819 INFO:teuthology.orchestra.run.smithi017.stdout:mds.i smithi076 running (2m) 55s ago 2m 16.0M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 4a5faa0da1a2 2024-02-14T15:02:44.819 INFO:teuthology.orchestra.run.smithi017.stdout:mds.j smithi017 running (2m) 20s ago 2m 16.6M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 8342a096b421 2024-02-14T15:02:44.820 INFO:teuthology.orchestra.run.smithi017.stdout:mds.k smithi050 running (2m) 55s ago 2m 13.8M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf bfcf9c1b0bed 2024-02-14T15:02:44.820 INFO:teuthology.orchestra.run.smithi017.stdout:mds.l smithi076 running (2m) 55s ago 2m 14.3M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 177ff9ea3110 2024-02-14T15:02:44.820 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.x smithi017 *:9283,8765 running (9m) 20s ago 9m 514M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 75150f7c5cee 2024-02-14T15:02:44.820 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.y smithi050 *:8443,8765 running (7m) 55s ago 7m 451M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf a8c20e957e06 2024-02-14T15:02:44.820 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.z smithi076 *:8443,8765 running (7m) 55s ago 7m 451M - 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf e8a8f5f084dc 2024-02-14T15:02:44.820 INFO:teuthology.orchestra.run.smithi017.stdout:mon.a smithi017 running (9m) 20s ago 9m 49.8M 2048M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 5ea5e790fba5 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:mon.b smithi050 running (8m) 55s ago 8m 42.8M 2048M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf e78f83618c28 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:mon.c smithi076 running (8m) 55s ago 8m 39.9M 2048M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 8a2c4279921b 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 smithi017 running (7m) 20s ago 7m 45.0M 4273M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 843518aa90b9 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 smithi050 running (7m) 55s ago 7m 47.4M 4271M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 9e82fe03358f 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 smithi076 running (6m) 55s ago 6m 56.5M 4270M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf b9c9def6367e 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 smithi017 running (6m) 20s ago 6m 43.7M 4273M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 980f7349ee64 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 smithi050 running (5m) 55s ago 5m 49.6M 4271M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 3aaa2bd32fe8 2024-02-14T15:02:44.821 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 smithi076 running (5m) 55s ago 5m 53.1M 4270M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 48f57b09fd4a 2024-02-14T15:02:44.822 INFO:teuthology.orchestra.run.smithi017.stdout:osd.6 smithi017 running (5m) 20s ago 5m 51.0M 4273M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 30b22004a457 2024-02-14T15:02:44.822 INFO:teuthology.orchestra.run.smithi017.stdout:osd.7 smithi050 running (4m) 55s ago 4m 43.9M 4271M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 7f109cce6ef8 2024-02-14T15:02:44.822 INFO:teuthology.orchestra.run.smithi017.stdout:osd.8 smithi076 running (4m) 55s ago 4m 43.6M 4270M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 90d5d4249ed0 2024-02-14T15:02:44.822 INFO:teuthology.orchestra.run.smithi017.stdout:osd.9 smithi017 running (3m) 20s ago 3m 50.1M 4273M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf d25c302cc3b9 2024-02-14T15:02:44.822 INFO:teuthology.orchestra.run.smithi017.stdout:osd.10 smithi050 running (3m) 55s ago 3m 51.2M 4271M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 23ec8f18b8a1 2024-02-14T15:02:44.822 INFO:teuthology.orchestra.run.smithi017.stdout:osd.11 smithi076 running (2m) 55s ago 2m 42.1M 4270M 19.0.0-1280-ga6fe3fa3 87bb2d90c3bf 02282b60983b 2024-02-14T15:02:44.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f40bc0a2260 msgr2=0x7f40bc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:44.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f40bc0a2260 0x7f40bc0a4720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40b8005fd0 tx=0x7f40b800a560 comp rx=0 tx=0).stop 2024-02-14T15:02:44.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d0100c50 msgr2=0x7f40d019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:44.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d0100c50 0x7f40d019a010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f40c800d9b0 tx=0x7f40c800dcc0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 shutdown_connections 2024-02-14T15:02:44.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f40bc0a2260 0x7f40bc0a4720 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40d01027f0 0x7f40d01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d0100c50 0x7f40d019a010 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 --2- 172.21.15.17:0/3895407908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40d0101e50 0x7f40d019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:44.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 >> 172.21.15.17:0/3895407908 conn(0x7f40d00fc210 msgr2=0x7f40d00fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:44.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 shutdown_connections 2024-02-14T15:02:44.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:44.818+0000 7f40d4bb6700 1 -- 172.21.15.17:0/3895407908 wait complete. 2024-02-14T15:02:44.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:44 smithi050 ceph-mon[36512]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:44.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-02-14T15:02:44.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-02-14T15:02:44.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:44 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]: dispatch 2024-02-14T15:02:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:44 smithi076 ceph-mon[37057]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-02-14T15:02:44.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-02-14T15:02:44.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:44 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]: dispatch 2024-02-14T15:02:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:44 smithi017 ceph-mon[32549]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:02:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-02-14T15:02:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-02-14T15:02:45.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:44 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]: dispatch 2024-02-14T15:02:45.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-14T15:02:45.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-02-14T15:02:45.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-02-14T15:02:45.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:45 smithi017 ceph-mon[32549]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]': finished 2024-02-14T15:02:45.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:45 smithi017 ceph-mon[32549]: osdmap e92: 12 total, 12 up, 12 in 2024-02-14T15:02:45.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:45 smithi017 ceph-mon[32549]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-02-14T15:02:45.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:45 smithi017 ceph-mon[32549]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-02-14T15:02:45.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:45 smithi017 ceph-mon[32549]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:45.768 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:45.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-02-14T15:02:45.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-02-14T15:02:45.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:45 smithi050 ceph-mon[36512]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]': finished 2024-02-14T15:02:45.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:45 smithi050 ceph-mon[36512]: osdmap e92: 12 total, 12 up, 12 in 2024-02-14T15:02:45.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:45 smithi050 ceph-mon[36512]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-02-14T15:02:45.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:45 smithi050 ceph-mon[36512]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-02-14T15:02:45.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:45 smithi050 ceph-mon[36512]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-02-14T15:02:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-02-14T15:02:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:45 smithi076 ceph-mon[37057]: from='mgr.14150 172.21.15.17:0/3933889772' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 10]}]': finished 2024-02-14T15:02:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:45 smithi076 ceph-mon[37057]: osdmap e92: 12 total, 12 up, 12 in 2024-02-14T15:02:45.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:45 smithi076 ceph-mon[37057]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-02-14T15:02:45.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:45 smithi076 ceph-mon[37057]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-02-14T15:02:45.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:45 smithi076 ceph-mon[37057]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:46.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:46 smithi050 ceph-mon[36512]: osdmap e93: 12 total, 12 up, 12 in 2024-02-14T15:02:46.954 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:46 smithi017 ceph-mon[32549]: osdmap e93: 12 total, 12 up, 12 in 2024-02-14T15:02:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:46 smithi076 ceph-mon[37057]: osdmap e93: 12 total, 12 up, 12 in 2024-02-14T15:02:47.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.486+0000 7ffb574ad700 1 -- 172.21.15.17:0/1589083110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50103910 msgr2=0x7ffb5010a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.486+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1589083110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50103910 0x7ffb5010a1c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800b0d0 tx=0x7ffb4800b3e0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.486+0000 7ffb574ad700 1 -- 172.21.15.17:0/1589083110 shutdown_connections 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.486+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1589083110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50103910 0x7ffb5010a1c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1589083110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb50101d70 0x7ffb50102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1589083110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb50102f70 0x7ffb501033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 -- 172.21.15.17:0/1589083110 >> 172.21.15.17:0/1589083110 conn(0x7ffb500fd330 msgr2=0x7ffb500ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 -- 172.21.15.17:0/1589083110 shutdown_connections 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 -- 172.21.15.17:0/1589083110 wait complete. 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 Processor -- start 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 -- start start 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50101d70 0x7ffb50075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb50102f70 0x7ffb50074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb50103910 0x7ffb50074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50074dd0 con 0x7ffb50101d70 2024-02-14T15:02:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50074f40 con 0x7ffb50102f70 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb574ad700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501a1e00 con 0x7ffb50103910 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.487+0000 7ffb55249700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50101d70 0x7ffb50075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50101d70 0x7ffb50075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46964/0 (socket says 172.21.15.17:46964) 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 -- 172.21.15.17:0/1725862943 learned_addr learned my addr 172.21.15.17:0/1725862943 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55a4a700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb50103910 0x7ffb50074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb54a48700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb50102f70 0x7ffb50074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 -- 172.21.15.17:0/1725862943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb50103910 msgr2=0x7ffb50074650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb50103910 0x7ffb50074650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 -- 172.21.15.17:0/1725862943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb50102f70 msgr2=0x7ffb50074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb50102f70 0x7ffb50074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 -- 172.21.15.17:0/1725862943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb48009d00 con 0x7ffb50101d70 2024-02-14T15:02:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb55249700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50101d70 0x7ffb50075ac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000d980 tx=0x7ffb4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:47.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb467fc700 1 -- 172.21.15.17:0/1725862943 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40004020 con 0x7ffb50101d70 2024-02-14T15:02:47.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb467fc700 1 -- 172.21.15.17:0/1725862943 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb4000f690 con 0x7ffb50101d70 2024-02-14T15:02:47.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb501a20e0 con 0x7ffb50101d70 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb467fc700 1 -- 172.21.15.17:0/1725862943 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40003b40 con 0x7ffb50101d70 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.488+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb501a25e0 con 0x7ffb50101d70 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.489+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb501090d0 con 0x7ffb50101d70 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.490+0000 7ffb467fc700 1 -- 172.21.15.17:0/1725862943 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffb4000fb20 con 0x7ffb50101d70 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.491+0000 7ffb467fc700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb3c0a2150 0x7ffb3c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.491+0000 7ffb54a48700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb3c0a2150 0x7ffb3c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.491+0000 7ffb467fc700 1 -- 172.21.15.17:0/1725862943 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7ffb400cdb90 con 0x7ffb50101d70 2024-02-14T15:02:47.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.491+0000 7ffb54a48700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb3c0a2150 0x7ffb3c0a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c007be0 tx=0x7ffb4c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:47.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.495+0000 7ffb467fc700 1 -- 172.21.15.17:0/1725862943 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb40096810 con 0x7ffb50101d70 2024-02-14T15:02:47.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.707+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb5002d880 con 0x7ffb3c0a2150 2024-02-14T15:02:47.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.713+0000 7ffb467fc700 1 -- 172.21.15.17:0/1725862943 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7ffb5002d880 con 0x7ffb3c0a2150 2024-02-14T15:02:47.715 INFO:teuthology.orchestra.run.smithi017.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-14T15:02:47.715 INFO:teuthology.orchestra.run.smithi017.stdout:mds.all 12/12 58s ago 2m smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi076=c;smithi076=f;smithi076=i;smithi076=l;count:12 2024-02-14T15:02:47.715 INFO:teuthology.orchestra.run.smithi017.stdout:mgr 3/3 58s ago 7m smithi017=x;smithi050=y;smithi076=z;count:3 2024-02-14T15:02:47.715 INFO:teuthology.orchestra.run.smithi017.stdout:mon 3/3 58s ago 8m smithi017:172.21.15.17=a;smithi050:172.21.15.50=b;smithi076:172.21.15.76=c;count:3 2024-02-14T15:02:47.715 INFO:teuthology.orchestra.run.smithi017.stdout:osd 12 58s ago - 2024-02-14T15:02:47.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb3c0a2150 msgr2=0x7ffb3c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:47.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb3c0a2150 0x7ffb3c0a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c007be0 tx=0x7ffb4c0073d0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50101d70 msgr2=0x7ffb50075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:47.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50101d70 0x7ffb50075ac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000d980 tx=0x7ffb4000dc90 comp rx=0 tx=0).stop 2024-02-14T15:02:47.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 shutdown_connections 2024-02-14T15:02:47.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ffb3c0a2150 0x7ffb3c0a4610 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb50103910 0x7ffb50074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb50101d70 0x7ffb50075ac0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 --2- 172.21.15.17:0/1725862943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb50102f70 0x7ffb50074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:47.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.715+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 >> 172.21.15.17:0/1725862943 conn(0x7ffb500fd330 msgr2=0x7ffb50106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:47.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.716+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 shutdown_connections 2024-02-14T15:02:47.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:47.716+0000 7ffb574ad700 1 -- 172.21.15.17:0/1725862943 wait complete. 2024-02-14T15:02:47.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:47 smithi050 ceph-mon[36512]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-02-14T15:02:47.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:47 smithi076 ceph-mon[37057]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-02-14T15:02:48.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:47 smithi017 ceph-mon[32549]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-02-14T15:02:48.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-14T15:02:48.797 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:48.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:48 smithi050 ceph-mon[36512]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-14T15:02:48.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:48 smithi050 ceph-mon[36512]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:48.919 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:48 smithi017 ceph-mon[32549]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-14T15:02:48.919 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:48 smithi017 ceph-mon[32549]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:48.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:48 smithi076 ceph-mon[37057]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-14T15:02:48.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:48 smithi076 ceph-mon[37057]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:49.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:49 smithi050 ceph-mon[36512]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 6 op/s 2024-02-14T15:02:49.941 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:49 smithi017 ceph-mon[32549]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 6 op/s 2024-02-14T15:02:49.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:49 smithi076 ceph-mon[37057]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 6 op/s 2024-02-14T15:02:50.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 -- 172.21.15.17:0/2830291171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 msgr2=0x7f39c4109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:50.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/2830291171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c4109f20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00b0a0 tx=0x7f39bc00b3b0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 -- 172.21.15.17:0/2830291171 shutdown_connections 2024-02-14T15:02:50.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/2830291171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c4109f20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/2830291171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4102850 0x7f39c4104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/2830291171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c4105180 0x7f39c4107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 -- 172.21.15.17:0/2830291171 >> 172.21.15.17:0/2830291171 conn(0x7f39c40fc280 msgr2=0x7f39c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 -- 172.21.15.17:0/2830291171 shutdown_connections 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.485+0000 7f39c9c0d700 1 -- 172.21.15.17:0/2830291171 wait complete. 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 Processor -- start 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 -- start start 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c4102850 0x7f39c4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4105180 0x7f39c4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c419f8c0 con 0x7f39c4107ab0 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c419fa00 con 0x7f39c4102850 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c9c0d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c419fb90 con 0x7f39c4105180 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39baffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4105180 0x7f39c4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39baffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4105180 0x7f39c4198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38726/0 (socket says 172.21.15.17:38726) 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33690/0 (socket says 172.21.15.17:33690) 2024-02-14T15:02:50.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.486+0000 7f39baffd700 1 -- 172.21.15.17:0/1937882623 learned_addr learned my addr 172.21.15.17:0/1937882623 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c37fe700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c4102850 0x7f39c4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39baffd700 1 -- 172.21.15.17:0/1937882623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c4102850 msgr2=0x7f39c4197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39baffd700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c4102850 0x7f39c4197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39baffd700 1 -- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 msgr2=0x7f39c419f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39baffd700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c419f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39baffd700 1 -- 172.21.15.17:0/1937882623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc009d00 con 0x7f39c4105180 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c3fff700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c37fe700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c4102850 0x7f39c4197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39baffd700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4105180 0x7f39c4198270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00c9e0 tx=0x7f39ac00ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c17fa700 1 -- 172.21.15.17:0/1937882623 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac007ac0 con 0x7f39c4105180 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c419fe70 con 0x7f39c4105180 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c17fa700 1 -- 172.21.15.17:0/1937882623 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39ac007c20 con 0x7f39c4105180 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c17fa700 1 -- 172.21.15.17:0/1937882623 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac018660 con 0x7f39c4105180 2024-02-14T15:02:50.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.487+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c41a0330 con 0x7f39c4105180 2024-02-14T15:02:50.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.488+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c40fde00 con 0x7f39c4105180 2024-02-14T15:02:50.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.490+0000 7f39c17fa700 1 -- 172.21.15.17:0/1937882623 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f39ac00f450 con 0x7f39c4105180 2024-02-14T15:02:50.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.490+0000 7f39c17fa700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f39b00a2150 0x7f39b00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:50.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.490+0000 7f39c37fe700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f39b00a2150 0x7f39b00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:50.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.490+0000 7f39c17fa700 1 -- 172.21.15.17:0/1937882623 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f39ac0ce280 con 0x7f39c4105180 2024-02-14T15:02:50.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.491+0000 7f39c37fe700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f39b00a2150 0x7f39b00a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39b4005950 tx=0x7f39b40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:50.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.495+0000 7f39c17fa700 1 -- 172.21.15.17:0/1937882623 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39ac096f30 con 0x7f39c4105180 2024-02-14T15:02:50.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.703+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f39c402d8e0 con 0x7f39b00a2150 2024-02-14T15:02:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.704+0000 7f39c17fa700 1 -- 172.21.15.17:0/1937882623 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7f39c402d8e0 con 0x7f39b00a2150 2024-02-14T15:02:50.706 INFO:teuthology.orchestra.run.smithi017.stdout:HOST ADDR LABELS STATUS 2024-02-14T15:02:50.706 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017 172.21.15.17 2024-02-14T15:02:50.706 INFO:teuthology.orchestra.run.smithi017.stdout:smithi050 172.21.15.50 2024-02-14T15:02:50.706 INFO:teuthology.orchestra.run.smithi017.stdout:smithi076 172.21.15.76 2024-02-14T15:02:50.706 INFO:teuthology.orchestra.run.smithi017.stdout:3 hosts in cluster 2024-02-14T15:02:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f39b00a2150 msgr2=0x7f39b00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f39b00a2150 0x7f39b00a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39b4005950 tx=0x7f39b40058e0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4105180 msgr2=0x7f39c4198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4105180 0x7f39c4198270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00c9e0 tx=0x7f39ac00ccf0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 shutdown_connections 2024-02-14T15:02:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f39b00a2150 0x7f39b00a4610 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c4107ab0 0x7f39c419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39c4102850 0x7f39c4197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 --2- 172.21.15.17:0/1937882623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c4105180 0x7f39c4198270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.706+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 >> 172.21.15.17:0/1937882623 conn(0x7f39c40fc280 msgr2=0x7f39c4103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.707+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 shutdown_connections 2024-02-14T15:02:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:50.707+0000 7f39c9c0d700 1 -- 172.21.15.17:0/1937882623 wait complete. 2024-02-14T15:02:51.544 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-14T15:02:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:51 smithi017 ceph-mon[32549]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 8 op/s 2024-02-14T15:02:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:51 smithi017 ceph-mon[32549]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:51 smithi017 ceph-mon[32549]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-14T15:02:51.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:51 smithi017 ceph-mon[32549]: Cluster is now healthy 2024-02-14T15:02:51.793 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:51.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:51 smithi076 ceph-mon[37057]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 8 op/s 2024-02-14T15:02:51.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:51 smithi076 ceph-mon[37057]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:51.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:51 smithi076 ceph-mon[37057]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-14T15:02:51.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:51 smithi076 ceph-mon[37057]: Cluster is now healthy 2024-02-14T15:02:52.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:51 smithi050 ceph-mon[36512]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 8 op/s 2024-02-14T15:02:52.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:51 smithi050 ceph-mon[36512]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:52.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:51 smithi050 ceph-mon[36512]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-14T15:02:52.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:51 smithi050 ceph-mon[36512]: Cluster is now healthy 2024-02-14T15:02:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 -- 172.21.15.17:0/1621845269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c103910 msgr2=0x7fee0c10a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 --2- 172.21.15.17:0/1621845269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c103910 0x7fee0c10a1c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fee040099c0 tx=0x7fee04009cd0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 -- 172.21.15.17:0/1621845269 shutdown_connections 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 --2- 172.21.15.17:0/1621845269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c103910 0x7fee0c10a1c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 --2- 172.21.15.17:0/1621845269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c101d70 0x7fee0c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 --2- 172.21.15.17:0/1621845269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee0c102f70 0x7fee0c1033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 -- 172.21.15.17:0/1621845269 >> 172.21.15.17:0/1621845269 conn(0x7fee0c0fd2d0 msgr2=0x7fee0c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 -- 172.21.15.17:0/1621845269 shutdown_connections 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.487+0000 7fee11c3f700 1 -- 172.21.15.17:0/1621845269 wait complete. 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 Processor -- start 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 -- start start 2024-02-14T15:02:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c101d70 0x7fee0c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee0c102f70 0x7fee0c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c103910 0x7fee0c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c19f870 con 0x7fee0c101d70 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c19f9b0 con 0x7fee0c102f70 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee11c3f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c19fb70 con 0x7fee0c103910 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c101d70 0x7fee0c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c101d70 0x7fee0c197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33702/0 (socket says 172.21.15.17:33702) 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 -- 172.21.15.17:0/3793356710 learned_addr learned my addr 172.21.15.17:0/3793356710 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0bfff700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c103910 0x7fee0c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0affd700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee0c102f70 0x7fee0c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 -- 172.21.15.17:0/3793356710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c103910 msgr2=0x7fee0c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c103910 0x7fee0c19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 -- 172.21.15.17:0/3793356710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee0c102f70 msgr2=0x7fee0c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:53.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee0c102f70 0x7fee0c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.488+0000 7fee0b7fe700 1 -- 172.21.15.17:0/3793356710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040095f0 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.489+0000 7fee0b7fe700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c101d70 0x7fee0c197d20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00ada0 tx=0x7fedfc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.489+0000 7fee08ff9700 1 -- 172.21.15.17:0/3793356710 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00f070 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.489+0000 7fee08ff9700 1 -- 172.21.15.17:0/3793356710 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedfc007be0 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.489+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c19fe50 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.489+0000 7fee08ff9700 1 -- 172.21.15.17:0/3793356710 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc004420 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.489+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c1a0260 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.490+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c109060 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.491+0000 7fee08ff9700 1 -- 172.21.15.17:0/3793356710 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fedfc00caa0 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.491+0000 7fee08ff9700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedf80a2150 0x7fedf80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.492+0000 7fee0affd700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedf80a2150 0x7fedf80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.492+0000 7fee08ff9700 1 -- 172.21.15.17:0/3793356710 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fedfc0cd940 con 0x7fee0c101d70 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.492+0000 7fee0affd700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedf80a2150 0x7fedf80a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005fd0 tx=0x7fedf4005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:53.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.495+0000 7fee08ff9700 1 -- 172.21.15.17:0/3793356710 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fedfc0965f0 con 0x7fee0c101d70 2024-02-14T15:02:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.711+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fee0c02da50 con 0x7fedf80a2150 2024-02-14T15:02:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.713+0000 7fee08ff9700 1 -- 172.21.15.17:0/3793356710 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fee0c02da50 con 0x7fedf80a2150 2024-02-14T15:02:53.714 INFO:teuthology.orchestra.run.smithi017.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-14T15:02:53.714 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7473003Q480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-14T15:02:53.714 INFO:teuthology.orchestra.run.smithi017.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-14T15:02:53.714 INFO:teuthology.orchestra.run.smithi017.stdout:smithi076 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400U4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-14T15:02:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedf80a2150 msgr2=0x7fedf80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedf80a2150 0x7fedf80a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005fd0 tx=0x7fedf4005ee0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c101d70 msgr2=0x7fee0c197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c101d70 0x7fee0c197d20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00ada0 tx=0x7fedfc00c330 comp rx=0 tx=0).stop 2024-02-14T15:02:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 shutdown_connections 2024-02-14T15:02:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fedf80a2150 0x7fedf80a4610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c103910 0x7fee0c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0c101d70 0x7fee0c197d20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 --2- 172.21.15.17:0/3793356710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee0c102f70 0x7fee0c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 >> 172.21.15.17:0/3793356710 conn(0x7fee0c0fd2d0 msgr2=0x7fee0c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 shutdown_connections 2024-02-14T15:02:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:53.715+0000 7fee11c3f700 1 -- 172.21.15.17:0/3793356710 wait complete. 2024-02-14T15:02:53.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:53 smithi017 ceph-mon[32549]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 253 B/s wr, 6 op/s 2024-02-14T15:02:54.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:53 smithi050 ceph-mon[36512]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 253 B/s wr, 6 op/s 2024-02-14T15:02:54.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:53 smithi076 ceph-mon[37057]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 253 B/s wr, 6 op/s 2024-02-14T15:02:54.424 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-14T15:02:54.434 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:02:54.434 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-14T15:02:54.688 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:55.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:54 smithi017 ceph-mon[32549]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:54 smithi050 ceph-mon[36512]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:54 smithi076 ceph-mon[37057]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:02:55.767 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:55 smithi017 ceph-mon[32549]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-14T15:02:56.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:55 smithi050 ceph-mon[36512]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-14T15:02:56.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:55 smithi076 ceph-mon[37057]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-02-14T15:02:56.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.397+0000 7f0612734700 1 -- 172.21.15.17:0/4199684622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c102f50 msgr2=0x7f060c1033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:56.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.397+0000 7f0612734700 1 --2- 172.21.15.17:0/4199684622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c102f50 0x7f060c1033b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f06000099c0 tx=0x7f0600009cd0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 -- 172.21.15.17:0/4199684622 shutdown_connections 2024-02-14T15:02:56.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 --2- 172.21.15.17:0/4199684622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f060c1038f0 0x7f060c10a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 --2- 172.21.15.17:0/4199684622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f060c101d50 0x7f060c102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 --2- 172.21.15.17:0/4199684622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c102f50 0x7f060c1033b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 -- 172.21.15.17:0/4199684622 >> 172.21.15.17:0/4199684622 conn(0x7f060c0fd330 msgr2=0x7f060c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 -- 172.21.15.17:0/4199684622 shutdown_connections 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 -- 172.21.15.17:0/4199684622 wait complete. 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 Processor -- start 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.398+0000 7f0612734700 1 -- start start 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0612734700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c101d50 0x7f060c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0612734700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f060c102f50 0x7f060c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0612734700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f060c1038f0 0x7f060c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0612734700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c19f870 con 0x7f060c101d50 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0612734700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c19f9b0 con 0x7f060c1038f0 2024-02-14T15:02:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0612734700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c19fb70 con 0x7f060c102f50 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c101d50 0x7f060c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c101d50 0x7f060c197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33722/0 (socket says 172.21.15.17:33722) 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 -- 172.21.15.17:0/1887624718 learned_addr learned my addr 172.21.15.17:0/1887624718 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060b7fe700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f060c102f50 0x7f060c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0610cd1700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f060c1038f0 0x7f060c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 -- 172.21.15.17:0/1887624718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f060c102f50 msgr2=0x7f060c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f060c102f50 0x7f060c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 -- 172.21.15.17:0/1887624718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f060c1038f0 msgr2=0x7f060c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f060c1038f0 0x7f060c19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 -- 172.21.15.17:0/1887624718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000095f0 con 0x7f060c101d50 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f060bfff700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c101d50 0x7f060c197d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00d980 tx=0x7f05fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f06097fa700 1 -- 172.21.15.17:0/1887624718 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc004020 con 0x7f060c101d50 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f06097fa700 1 -- 172.21.15.17:0/1887624718 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05fc00f690 con 0x7f060c101d50 2024-02-14T15:02:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f06097fa700 1 -- 172.21.15.17:0/1887624718 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc003b40 con 0x7f060c101d50 2024-02-14T15:02:56.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.399+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c19fe50 con 0x7f060c101d50 2024-02-14T15:02:56.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.400+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c1a0260 con 0x7f060c101d50 2024-02-14T15:02:56.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.400+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c109110 con 0x7f060c101d50 2024-02-14T15:02:56.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.401+0000 7f06097fa700 1 -- 172.21.15.17:0/1887624718 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f05fc00fb20 con 0x7f060c101d50 2024-02-14T15:02:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.402+0000 7f06097fa700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05f40a2150 0x7f05f40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.402+0000 7f060b7fe700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05f40a2150 0x7f05f40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.402+0000 7f06097fa700 1 -- 172.21.15.17:0/1887624718 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f05fc0cdb30 con 0x7f060c101d50 2024-02-14T15:02:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.403+0000 7f060b7fe700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05f40a2150 0x7f05f40a4610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0600009990 tx=0x7f0600005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:56.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.407+0000 7f06097fa700 1 -- 172.21.15.17:0/1887624718 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05fc0967b0 con 0x7f060c101d50 2024-02-14T15:02:56.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.653+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f060c10d690 con 0x7f060c101d50 2024-02-14T15:02:56.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.653+0000 7f06097fa700 1 -- 172.21.15.17:0/1887624718 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2925 (secure 0 0 0) 0x7f05fc099e70 con 0x7f060c101d50 2024-02-14T15:02:56.654 INFO:teuthology.orchestra.run.smithi017.stdout:e23 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:legacy client fscid: 1 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:Filesystem 'cephfs' (1) 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:fs_name cephfs 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 23 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-02-14T15:00:48.619843+0000 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2024-02-14T15:01:15.551062+0000 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:tableserver 0 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:root 0 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:session_timeout 300 2024-02-14T15:02:56.655 INFO:teuthology.orchestra.run.smithi017.stdout:session_autoclose 300 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:max_file_size 1099511627776 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:max_xattr_size 65536 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:required_client_features {} 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:last_failure 0 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:last_failure_osd_epoch 0 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:max_mds 3 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:in 0,1,2 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:up {0=24509,1=24502,2=24481} 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:failed 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:damaged 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:stopped 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:data_pools [3,4] 2024-02-14T15:02:56.656 INFO:teuthology.orchestra.run.smithi017.stdout:metadata_pool 2 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:inline_data disabled 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:balancer 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:bal_rank_mask -1 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:standby_count_wanted 0 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b{0:24509} state up:active seq 3 addr [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.h{0:24479} state up:standby-replay seq 1 addr [v2:172.21.15.50:6836/1706654420,v1:172.21.15.50:6837/1706654420] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.i{1:24502} state up:active seq 11 addr [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.f{1:24469} state up:standby-replay seq 1 addr [v2:172.21.15.76:6834/1787917010,v1:172.21.15.76:6835/1787917010] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c{2:24481} state up:active seq 13 addr [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.e{2:24446} state up:standby-replay seq 1 addr [v2:172.21.15.50:6834/3474801477,v1:172.21.15.50:6835/3474801477] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout:Standby daemons: 2024-02-14T15:02:56.657 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:02:56.658 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.j{-1:14568} state up:standby seq 1 addr [v2:172.21.15.17:6834/763584482,v1:172.21.15.17:6835/763584482] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.658 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.d{-1:14580} state up:standby seq 1 addr [v2:172.21.15.17:6836/2485072299,v1:172.21.15.17:6837/2485072299] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.658 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.g{-1:14649} state up:standby seq 1 addr [v2:172.21.15.17:6838/516562430,v1:172.21.15.17:6839/516562430] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.658 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a{-1:14661} state up:standby seq 1 addr [v2:172.21.15.17:6840/3297312936,v1:172.21.15.17:6841/3297312936] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.658 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.k{-1:24416} state up:standby seq 1 addr [v2:172.21.15.50:6832/277751222,v1:172.21.15.50:6833/277751222] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.658 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.l{-1:24424} state up:standby seq 1 addr [v2:172.21.15.76:6832/580745027,v1:172.21.15.76:6833/580745027] compat {c=[1],r=[1],i=[fff]}] 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05f40a2150 msgr2=0x7f05f40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05f40a2150 0x7f05f40a4610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0600009990 tx=0x7f0600005e80 comp rx=0 tx=0).stop 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c101d50 msgr2=0x7f060c197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c101d50 0x7f060c197d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00d980 tx=0x7f05fc00dc90 comp rx=0 tx=0).stop 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 shutdown_connections 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f05f40a2150 0x7f05f40a4610 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f060c1038f0 0x7f060c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f060c101d50 0x7f060c197d20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 --2- 172.21.15.17:0/1887624718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f060c102f50 0x7f060c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 >> 172.21.15.17:0/1887624718 conn(0x7f060c0fd330 msgr2=0x7f060c106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 shutdown_connections 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:56.657+0000 7f0612734700 1 -- 172.21.15.17:0/1887624718 wait complete. 2024-02-14T15:02:56.659 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-02-14T15:02:56.956 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:56 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1887624718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-14T15:02:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:56 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1887624718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-14T15:02:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:56 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1887624718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-14T15:02:57.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-14T15:02:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:02:57.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:57 smithi017 ceph-mon[32549]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-14T15:02:58.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:57 smithi050 ceph-mon[36512]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-14T15:02:58.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:57 smithi076 ceph-mon[37057]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-14T15:02:59.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 -- 172.21.15.17:0/2146960211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e00105100 msgr2=0x7f4e001074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:59.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 --2- 172.21.15.17:0/2146960211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e00105100 0x7f4e001074f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009990 tx=0x7f4de8009ca0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 -- 172.21.15.17:0/2146960211 shutdown_connections 2024-02-14T15:02:59.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 --2- 172.21.15.17:0/2146960211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e00107a30 0x7f4e00109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 --2- 172.21.15.17:0/2146960211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e001027d0 0x7f4e00104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 --2- 172.21.15.17:0/2146960211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e00105100 0x7f4e001074f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 -- 172.21.15.17:0/2146960211 >> 172.21.15.17:0/2146960211 conn(0x7f4e000fc1e0 msgr2=0x7f4e000fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 -- 172.21.15.17:0/2146960211 shutdown_connections 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.265+0000 7f4e04da9700 1 -- 172.21.15.17:0/2146960211 wait complete. 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 Processor -- start 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 -- start start 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e001027d0 0x7f4e00197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e00105100 0x7f4e00198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e00107a30 0x7f4e0019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0019f880 con 0x7f4e001027d0 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0019f9c0 con 0x7f4e00105100 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4e04da9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0019fb80 con 0x7f4e00107a30 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4dfe59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e001027d0 0x7f4e00197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4dfe59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e001027d0 0x7f4e00197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33734/0 (socket says 172.21.15.17:33734) 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4dfe59c700 1 -- 172.21.15.17:0/3663082161 learned_addr learned my addr 172.21.15.17:0/3663082161 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:02:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4dfdd9b700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e00105100 0x7f4e00198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.266+0000 7f4dfed9d700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e00107a30 0x7f4e0019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4dfe59c700 1 -- 172.21.15.17:0/3663082161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e00107a30 msgr2=0x7f4e0019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4dfe59c700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e00107a30 0x7f4e0019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4dfe59c700 1 -- 172.21.15.17:0/3663082161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e00105100 msgr2=0x7f4e00198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4dfe59c700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e00105100 0x7f4e00198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4dfe59c700 1 -- 172.21.15.17:0/3663082161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de80095f0 con 0x7f4e001027d0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4dfe59c700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e001027d0 0x7f4e00197d30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b530 tx=0x7f4df800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4df77fe700 1 -- 172.21.15.17:0/3663082161 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8011830 con 0x7f4e001027d0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4df77fe700 1 -- 172.21.15.17:0/3663082161 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4df8011e70 con 0x7f4e001027d0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0019fe60 con 0x7f4e001027d0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4df77fe700 1 -- 172.21.15.17:0/3663082161 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800f2e0 con 0x7f4e001027d0 2024-02-14T15:02:59.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.267+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e001a0270 con 0x7f4e001027d0 2024-02-14T15:02:59.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.268+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e000689e0 con 0x7f4e001027d0 2024-02-14T15:02:59.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.269+0000 7f4df77fe700 1 -- 172.21.15.17:0/3663082161 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4df8010610 con 0x7f4e001027d0 2024-02-14T15:02:59.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.270+0000 7f4df77fe700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4dec0a2150 0x7f4dec0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:02:59.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.270+0000 7f4dfdd9b700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4dec0a2150 0x7f4dec0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:02:59.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.270+0000 7f4df77fe700 1 -- 172.21.15.17:0/3663082161 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f4df80cd9a0 con 0x7f4e001027d0 2024-02-14T15:02:59.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.270+0000 7f4dfdd9b700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4dec0a2150 0x7f4dec0a4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4de800b5c0 tx=0x7f4de80038a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:02:59.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.274+0000 7f4df77fe700 1 -- 172.21.15.17:0/3663082161 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4df8096620 con 0x7f4e001027d0 2024-02-14T15:02:59.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.480+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f4e000fdd60 con 0x7f4e001027d0 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.481+0000 7f4df77fe700 1 -- 172.21.15.17:0/3663082161 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5624 (secure 0 0 0) 0x7f4df8099ce0 con 0x7f4e001027d0 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 93 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-02-14T14:52:46.029095+0000 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2024-02-14T15:02:45.606961+0000 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:crush_version 27 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:full_ratio 0.95 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:backfillfull_ratio 0.9 2024-02-14T15:02:59.482 INFO:teuthology.orchestra.run.smithi017.stdout:nearfull_ratio 0.85 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:require_min_compat_client luminous 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:min_compat_client luminous 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:require_osd_release squid 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:stretch_mode_enabled false 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:max_osd 12 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6802/1615583912,v1:172.21.15.17:6803/1615583912] [v2:172.21.15.17:6804/1615583912,v1:172.21.15.17:6805/1615583912] exists,up 423af2fd-89fb-4e39-913c-be53474fe012 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6800/2051103540,v1:172.21.15.50:6801/2051103540] [v2:172.21.15.50:6802/2051103540,v1:172.21.15.50:6803/2051103540] exists,up 3f59afa4-e5b3-462c-89a0-e9552f262730 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/2271935501,v1:172.21.15.76:6801/2271935501] [v2:172.21.15.76:6802/2271935501,v1:172.21.15.76:6803/2271935501] exists,up 0f1d2809-6bb1-46aa-9ba8-dd08192a591b 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6810/1293297743,v1:172.21.15.17:6811/1293297743] [v2:172.21.15.17:6812/1293297743,v1:172.21.15.17:6813/1293297743] exists,up 6c1e9407-028a-4d4b-af15-2f0d9b05669b 2024-02-14T15:02:59.483 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6808/4058610796,v1:172.21.15.50:6809/4058610796] [v2:172.21.15.50:6810/4058610796,v1:172.21.15.50:6811/4058610796] exists,up 445cd928-d6fe-4815-8094-e407331ec2db 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 up in weight 1 up_from 38 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6808/75527481,v1:172.21.15.76:6809/75527481] [v2:172.21.15.76:6810/75527481,v1:172.21.15.76:6811/75527481] exists,up f7c1a10c-7507-426e-b91a-9f788f8edd9a 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6818/2957627579,v1:172.21.15.17:6819/2957627579] [v2:172.21.15.17:6820/2957627579,v1:172.21.15.17:6821/2957627579] exists,up 837621c6-44f2-4ede-83d8-031db03b8ad2 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6816/3407611050,v1:172.21.15.50:6817/3407611050] [v2:172.21.15.50:6818/3407611050,v1:172.21.15.50:6819/3407611050] exists,up 8ea6e493-7d1b-4570-a7d0-2f50d2adf530 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.8 up in weight 1 up_from 59 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6816/416823437,v1:172.21.15.76:6817/416823437] [v2:172.21.15.76:6818/416823437,v1:172.21.15.76:6819/416823437] exists,up e4cf495a-56b2-45b3-9527-0ee4f829e144 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6826/2843045035,v1:172.21.15.17:6827/2843045035] [v2:172.21.15.17:6828/2843045035,v1:172.21.15.17:6829/2843045035] exists,up e2b22fd6-b721-41e5-b910-db93fa49a826 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6824/156762867,v1:172.21.15.50:6825/156762867] [v2:172.21.15.50:6826/156762867,v1:172.21.15.50:6827/156762867] exists,up d4f225cb-2077-48f1-ab76-3e94e0566653 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6824/3113998070,v1:172.21.15.76:6825/3113998070] [v2:172.21.15.76:6826/3113998070,v1:172.21.15.76:6827/3113998070] exists,up 982cfd49-b3f7-4e98-a92a-7927c0aee762 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.6 [8,11] 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.b [8,11] 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.c [2,1] 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.14 [2,4] 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.3 [4,9] 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.11 [4,8] 2024-02-14T15:02:59.484 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.21 [5,9,4,6] 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.29 [3,9] 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.6 [9,11] 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.13 [9,8] 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.14 [9,8] 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.1d [5,10] 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/3179868836 expires 2024-02-15T14:53:42.569888+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/3756063081 expires 2024-02-15T14:53:42.569888+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6801/1044675107 expires 2024-02-15T14:53:42.569888+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/1117796267 expires 2024-02-15T14:53:15.465600+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/3241397077 expires 2024-02-15T14:53:42.569888+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6800/2869822133 expires 2024-02-15T14:53:15.465600+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6801/2869822133 expires 2024-02-15T14:53:15.465600+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/3371941300 expires 2024-02-15T14:53:15.465600+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6800/1044675107 expires 2024-02-15T14:53:42.569888+0000 2024-02-14T15:02:59.485 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/2319986884 expires 2024-02-15T14:53:15.465600+0000 2024-02-14T15:02:59.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4dec0a2150 msgr2=0x7f4dec0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:59.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4dec0a2150 0x7f4dec0a4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4de800b5c0 tx=0x7f4de80038a0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e001027d0 msgr2=0x7f4e00197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:02:59.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e001027d0 0x7f4e00197d30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b530 tx=0x7f4df800b840 comp rx=0 tx=0).stop 2024-02-14T15:02:59.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 shutdown_connections 2024-02-14T15:02:59.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f4dec0a2150 0x7f4dec0a4610 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e00107a30 0x7f4e0019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e001027d0 0x7f4e00197d30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 --2- 172.21.15.17:0/3663082161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e00105100 0x7f4e00198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:02:59.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 >> 172.21.15.17:0/3663082161 conn(0x7f4e000fc1e0 msgr2=0x7f4e00103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:02:59.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 shutdown_connections 2024-02-14T15:02:59.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:02:59.483+0000 7f4e04da9700 1 -- 172.21.15.17:0/3663082161 wait complete. 2024-02-14T15:03:00.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:59 smithi017 ceph-mon[32549]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-14T15:03:00.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:02:59 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3663082161' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T15:03:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:59 smithi050 ceph-mon[36512]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-14T15:03:00.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:02:59 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3663082161' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T15:03:00.209 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-14T15:03:00.224 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:03:00.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:59 smithi076 ceph-mon[37057]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-14T15:03:00.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:02:59 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3663082161' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T15:03:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 -- 172.21.15.17:0/4190432913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 msgr2=0x7f5c4c1040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 --2- 172.21.15.17:0/4190432913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c1040d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009900 tx=0x7f5c3c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 -- 172.21.15.17:0/4190432913 shutdown_connections 2024-02-14T15:03:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 --2- 172.21.15.17:0/4190432913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c4c106f40 0x7f5c4c109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 --2- 172.21.15.17:0/4190432913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c4c104610 0x7f5c4c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 --2- 172.21.15.17:0/4190432913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c1040d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 -- 172.21.15.17:0/4190432913 >> 172.21.15.17:0/4190432913 conn(0x7f5c4c0fb6f0 msgr2=0x7f5c4c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 -- 172.21.15.17:0/4190432913 shutdown_connections 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 -- 172.21.15.17:0/4190432913 wait complete. 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 Processor -- start 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.334+0000 7f5c528ae640 1 -- start start 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c4c104610 0x7f5c4c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c4c106f40 0x7f5c4c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c19ee70 con 0x7f5c4c101ce0 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c19efe0 con 0x7f5c4c104610 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c19f1a0 con 0x7f5c4c106f40 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33756/0 (socket says 172.21.15.17:33756) 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 -- 172.21.15.17:0/2259964762 learned_addr learned my addr 172.21.15.17:0/2259964762 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c50e4b640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c4c106f40 0x7f5c4c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 -- 172.21.15.17:0/2259964762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c4c106f40 msgr2=0x7f5c4c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c4c106f40 0x7f5c4c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 -- 172.21.15.17:0/2259964762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c4c104610 msgr2=0x7f5c4c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4b7fe640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c4c104610 0x7f5c4c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c4c104610 0x7f5c4c1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 -- 172.21.15.17:0/2259964762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c009560 con 0x7f5c4c101ce0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c50e4b640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c4c106f40 0x7f5c4c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c4bfff640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c197380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005ab0 tx=0x7f5c3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c497fa640 1 -- 172.21.15.17:0/2259964762 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0044e0 con 0x7f5c4c101ce0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c497fa640 1 -- 172.21.15.17:0/2259964762 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c3c0054e0 con 0x7f5c4c101ce0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c19f420 con 0x7f5c4c101ce0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c497fa640 1 -- 172.21.15.17:0/2259964762 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c03e450 con 0x7f5c4c101ce0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.335+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c19f9b0 con 0x7f5c4c101ce0 2024-02-14T15:03:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.336+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c0fd1e0 con 0x7f5c4c101ce0 2024-02-14T15:03:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.337+0000 7f5c497fa640 1 -- 172.21.15.17:0/2259964762 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5c3c0035f0 con 0x7f5c4c101ce0 2024-02-14T15:03:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.338+0000 7f5c497fa640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5c380a20c0 0x7f5c380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.338+0000 7f5c4b7fe640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5c380a20c0 0x7f5c380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.338+0000 7f5c497fa640 1 -- 172.21.15.17:0/2259964762 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f5c3c0f05e0 con 0x7f5c4c101ce0 2024-02-14T15:03:00.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.338+0000 7f5c4b7fe640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5c380a20c0 0x7f5c380a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c198890 tx=0x7f5c34005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:00.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.342+0000 7f5c497fa640 1 -- 172.21.15.17:0/2259964762 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c3c0b9260 con 0x7f5c4c101ce0 2024-02-14T15:03:00.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.525+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c4c02cde0 con 0x7f5c4c101ce0 2024-02-14T15:03:00.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.527+0000 7f5c497fa640 1 -- 172.21.15.17:0/2259964762 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f5c3c0bc920 con 0x7f5c4c101ce0 2024-02-14T15:03:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:00.528 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:02:45.606961+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5c380a20c0 msgr2=0x7f5c380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5c380a20c0 0x7f5c380a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c198890 tx=0x7f5c34005c60 comp rx=0 tx=0).stop 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 msgr2=0x7f5c4c197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c197380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005ab0 tx=0x7f5c3c0042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 shutdown_connections 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5c380a20c0 0x7f5c380a4580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c4c106f40 0x7f5c4c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c4c104610 0x7f5c4c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 --2- 172.21.15.17:0/2259964762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c4c101ce0 0x7f5c4c197380 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 >> 172.21.15.17:0/2259964762 conn(0x7f5c4c0fb6f0 msgr2=0x7f5c4c102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 shutdown_connections 2024-02-14T15:03:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.529+0000 7f5c528ae640 1 -- 172.21.15.17:0/2259964762 wait complete. 2024-02-14T15:03:00.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:03:00.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:03:00.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 -- 172.21.15.17:0/584379491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 msgr2=0x7fb278109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 --2- 172.21.15.17:0/584379491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb278109380 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb270009900 tx=0x7fb27002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:00.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 -- 172.21.15.17:0/584379491 shutdown_connections 2024-02-14T15:03:00.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 --2- 172.21.15.17:0/584379491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb278109380 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 --2- 172.21.15.17:0/584379491 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb278104600 0x7fb2781069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 --2- 172.21.15.17:0/584379491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb278101cd0 0x7fb2781040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 -- 172.21.15.17:0/584379491 >> 172.21.15.17:0/584379491 conn(0x7fb2780fb6e0 msgr2=0x7fb2780fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 -- 172.21.15.17:0/584379491 shutdown_connections 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.654+0000 7fb27f98c640 1 -- 172.21.15.17:0/584379491 wait complete. 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 Processor -- start 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 -- start start 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb278101cd0 0x7fb278197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb278104600 0x7fb278197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb27819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27819ee10 con 0x7fb278106f30 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27819ef80 con 0x7fb278101cd0 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27f98c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27819f140 con 0x7fb278104600 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27df29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb27819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27df29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb27819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57886/0 (socket says 172.21.15.17:57886) 2024-02-14T15:03:00.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27df29640 1 -- 172.21.15.17:0/54054952 learned_addr learned my addr 172.21.15.17:0/54054952 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27cf27640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb278104600 0x7fb278197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27d728640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb278101cd0 0x7fb278197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.655+0000 7fb27df29640 1 -- 172.21.15.17:0/54054952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb278104600 msgr2=0x7fb278197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27df29640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb278104600 0x7fb278197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27df29640 1 -- 172.21.15.17:0/54054952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb278101cd0 msgr2=0x7fb278197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27df29640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb278101cd0 0x7fb278197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27df29640 1 -- 172.21.15.17:0/54054952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270009560 con 0x7fb278106f30 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27d728640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb278101cd0 0x7fb278197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27df29640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb27819e8d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb270009230 tx=0x7fb270004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb26e7fc640 1 -- 172.21.15.17:0/54054952 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270004df0 con 0x7fb278106f30 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27819f3c0 con 0x7fb278106f30 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb26e7fc640 1 -- 172.21.15.17:0/54054952 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb270005430 con 0x7fb278106f30 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb26e7fc640 1 -- 172.21.15.17:0/54054952 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270003d20 con 0x7fb278106f30 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.656+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27819f9a0 con 0x7fb278106f30 2024-02-14T15:03:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.657+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2780fd1d0 con 0x7fb278106f30 2024-02-14T15:03:00.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.658+0000 7fb26e7fc640 1 -- 172.21.15.17:0/54054952 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb2700055a0 con 0x7fb278106f30 2024-02-14T15:03:00.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.659+0000 7fb26e7fc640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2640a2140 0x7fb2640a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.659+0000 7fb27d728640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2640a2140 0x7fb2640a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.659+0000 7fb26e7fc640 1 -- 172.21.15.17:0/54054952 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fb2700f0c30 con 0x7fb278106f30 2024-02-14T15:03:00.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.659+0000 7fb27d728640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2640a2140 0x7fb2640a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb274005fd0 tx=0x7fb2740093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:00.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.662+0000 7fb26e7fc640 1 -- 172.21.15.17:0/54054952 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2700b98b0 con 0x7fb278106f30 2024-02-14T15:03:00.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.844+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb27802ce50 con 0x7fb278106f30 2024-02-14T15:03:00.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.845+0000 7fb26e7fc640 1 -- 172.21.15.17:0/54054952 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2700bcf70 con 0x7fb278106f30 2024-02-14T15:03:00.846 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2640a2140 msgr2=0x7fb2640a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2640a2140 0x7fb2640a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb274005fd0 tx=0x7fb2740093e0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 msgr2=0x7fb27819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb27819e8d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb270009230 tx=0x7fb270004270 comp rx=0 tx=0).stop 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 shutdown_connections 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb2640a2140 0x7fb2640a4600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb278106f30 0x7fb27819e8d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb278104600 0x7fb278197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 --2- 172.21.15.17:0/54054952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb278101cd0 0x7fb278197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 >> 172.21.15.17:0/54054952 conn(0x7fb2780fb6e0 msgr2=0x7fb278102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:00.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 shutdown_connections 2024-02-14T15:03:00.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.847+0000 7fb27f98c640 1 -- 172.21.15.17:0/54054952 wait complete. 2024-02-14T15:03:00.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:00 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2259964762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:00.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 -- 172.21.15.17:0/4060905121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c101240 msgr2=0x7fcc5c101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 --2- 172.21.15.17:0/4060905121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c101240 0x7fcc5c101660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009900 tx=0x7fcc5802bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 -- 172.21.15.17:0/4060905121 shutdown_connections 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 --2- 172.21.15.17:0/4060905121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc5c102df0 0x7fcc5c109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 --2- 172.21.15.17:0/4060905121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5c102430 0x7fcc5c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 --2- 172.21.15.17:0/4060905121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c101240 0x7fcc5c101660 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 -- 172.21.15.17:0/4060905121 >> 172.21.15.17:0/4060905121 conn(0x7fcc5c0fc7b0 msgr2=0x7fcc5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 -- 172.21.15.17:0/4060905121 shutdown_connections 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.963+0000 7fcc646fb640 1 -- 172.21.15.17:0/4060905121 wait complete. 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 Processor -- start 2024-02-14T15:03:00.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 -- start start 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5c101240 0x7fcc5c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc5c102430 0x7fcc5c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c102df0 0x7fcc5c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c19edf0 con 0x7fcc5c102df0 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c19ef60 con 0x7fcc5c102430 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc646fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c19f120 con 0x7fcc5c101240 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c102df0 0x7fcc5c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c102df0 0x7fcc5c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57892/0 (socket says 172.21.15.17:57892) 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 -- 172.21.15.17:0/957697994 learned_addr learned my addr 172.21.15.17:0/957697994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62497640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5c101240 0x7fcc5c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc61c96640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc5c102430 0x7fcc5c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 -- 172.21.15.17:0/957697994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5c101240 msgr2=0x7fcc5c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5c101240 0x7fcc5c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 -- 172.21.15.17:0/957697994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc5c102430 msgr2=0x7fcc5c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc5c102430 0x7fcc5c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.964+0000 7fcc62c98640 1 -- 172.21.15.17:0/957697994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58009560 con 0x7fcc5c102df0 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.965+0000 7fcc62c98640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c102df0 0x7fcc5c19e8b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400eb10 tx=0x7fcc5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.965+0000 7fcc537fe640 1 -- 172.21.15.17:0/957697994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54004b20 con 0x7fcc5c102df0 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.965+0000 7fcc537fe640 1 -- 172.21.15.17:0/957697994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc54004c80 con 0x7fcc5c102df0 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.965+0000 7fcc537fe640 1 -- 172.21.15.17:0/957697994 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54010650 con 0x7fcc5c102df0 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.965+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c19f400 con 0x7fcc5c102df0 2024-02-14T15:03:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.965+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c19f8c0 con 0x7fcc5c102df0 2024-02-14T15:03:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.966+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c108600 con 0x7fcc5c102df0 2024-02-14T15:03:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.967+0000 7fcc537fe640 1 -- 172.21.15.17:0/957697994 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcc54005020 con 0x7fcc5c102df0 2024-02-14T15:03:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.967+0000 7fcc537fe640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc480a20c0 0x7fcc480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.968+0000 7fcc62497640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc480a20c0 0x7fcc480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.968+0000 7fcc537fe640 1 -- 172.21.15.17:0/957697994 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fcc54014070 con 0x7fcc5c102df0 2024-02-14T15:03:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.968+0000 7fcc62497640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc480a20c0 0x7fcc480a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005ab0 tx=0x7fcc58005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:00.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:00.972+0000 7fcc537fe640 1 -- 172.21.15.17:0/957697994 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc540972c0 con 0x7fcc5c102df0 2024-02-14T15:03:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:00 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2259964762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:01.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.151+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc5c02ce50 con 0x7fcc5c102df0 2024-02-14T15:03:01.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.151+0000 7fcc537fe640 1 -- 172.21.15.17:0/957697994 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc5409a980 con 0x7fcc5c102df0 2024-02-14T15:03:01.152 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc480a20c0 msgr2=0x7fcc480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc480a20c0 0x7fcc480a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005ab0 tx=0x7fcc58005fd0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c102df0 msgr2=0x7fcc5c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c102df0 0x7fcc5c19e8b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400eb10 tx=0x7fcc5400c2d0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 shutdown_connections 2024-02-14T15:03:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcc480a20c0 0x7fcc480a4580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc5c102df0 0x7fcc5c19e8b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc5c102430 0x7fcc5c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 --2- 172.21.15.17:0/957697994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5c101240 0x7fcc5c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 >> 172.21.15.17:0/957697994 conn(0x7fcc5c0fc7b0 msgr2=0x7fcc5c1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 shutdown_connections 2024-02-14T15:03:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.153+0000 7fcc646fb640 1 -- 172.21.15.17:0/957697994 wait complete. 2024-02-14T15:03:01.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:03:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:00 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2259964762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 -- 172.21.15.17:0/1406515716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58106f90 msgr2=0x7f0f58109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 --2- 172.21.15.17:0/1406515716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58106f90 0x7f0f58109380 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009ca0 tx=0x7f0f4802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 -- 172.21.15.17:0/1406515716 shutdown_connections 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 --2- 172.21.15.17:0/1406515716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58106f90 0x7f0f58109380 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 --2- 172.21.15.17:0/1406515716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58104660 0x7f0f58106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 --2- 172.21.15.17:0/1406515716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f58101d30 0x7f0f58104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 -- 172.21.15.17:0/1406515716 >> 172.21.15.17:0/1406515716 conn(0x7f0f580fb6e0 msgr2=0x7f0f580fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 -- 172.21.15.17:0/1406515716 shutdown_connections 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 -- 172.21.15.17:0/1406515716 wait complete. 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 Processor -- start 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.271+0000 7f0f5d639640 1 -- start start 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58101d30 0x7f0f58197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58104660 0x7f0f58197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f58106f90 0x7f0f5819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5819ecf0 con 0x7f0f58104660 2024-02-14T15:03:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5819ee60 con 0x7f0f58101d30 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5819f020 con 0x7f0f58106f90 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58104660 0x7f0f58197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58104660 0x7f0f58197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57916/0 (socket says 172.21.15.17:57916) 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 -- 172.21.15.17:0/4161154731 learned_addr learned my addr 172.21.15.17:0/4161154731 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f577fe640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f58106f90 0x7f0f5819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f56ffd640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58101d30 0x7f0f58197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 -- 172.21.15.17:0/4161154731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f58106f90 msgr2=0x7f0f5819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f58106f90 0x7f0f5819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 -- 172.21.15.17:0/4161154731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58101d30 msgr2=0x7f0f58197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58101d30 0x7f0f58197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 -- 172.21.15.17:0/4161154731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48009900 con 0x7f0f58104660 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f56ffd640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58101d30 0x7f0f58197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f567fc640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58104660 0x7f0f58197740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c970 tx=0x7f0f4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f3ffff640 1 -- 172.21.15.17:0/4161154731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c004440 con 0x7f0f58104660 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f3ffff640 1 -- 172.21.15.17:0/4161154731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c007d40 con 0x7f0f58104660 2024-02-14T15:03:01.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5819f300 con 0x7f0f58104660 2024-02-14T15:03:01.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f3ffff640 1 -- 172.21.15.17:0/4161154731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00f450 con 0x7f0f58104660 2024-02-14T15:03:01.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.272+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5819f810 con 0x7f0f58104660 2024-02-14T15:03:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.273+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f580fd1d0 con 0x7f0f58104660 2024-02-14T15:03:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.274+0000 7f0f3ffff640 1 -- 172.21.15.17:0/4161154731 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0f4c007860 con 0x7f0f58104660 2024-02-14T15:03:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.275+0000 7f0f3ffff640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0f400a20c0 0x7f0f400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.275+0000 7f0f56ffd640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0f400a20c0 0x7f0f400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.275+0000 7f0f3ffff640 1 -- 172.21.15.17:0/4161154731 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f0f4c0ce110 con 0x7f0f58104660 2024-02-14T15:03:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.276+0000 7f0f56ffd640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0f400a20c0 0x7f0f400a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005fd0 tx=0x7f0f50005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:01.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.280+0000 7f0f3ffff640 1 -- 172.21.15.17:0/4161154731 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f4c096d90 con 0x7f0f58104660 2024-02-14T15:03:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.461+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f5802ce50 con 0x7f0f58104660 2024-02-14T15:03:01.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.462+0000 7f0f3ffff640 1 -- 172.21.15.17:0/4161154731 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f4c09a450 con 0x7f0f58104660 2024-02-14T15:03:01.463 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:03:01.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.463+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0f400a20c0 msgr2=0x7f0f400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.463+0000 7f0f5d639640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0f400a20c0 0x7f0f400a4580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005fd0 tx=0x7f0f50005ea0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.463+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58104660 msgr2=0x7f0f58197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.463+0000 7f0f5d639640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58104660 0x7f0f58197740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c970 tx=0x7f0f4c00ce30 comp rx=0 tx=0).stop 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 shutdown_connections 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f0f400a20c0 0x7f0f400a4580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58104660 0x7f0f58197740 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f58106f90 0x7f0f5819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 --2- 172.21.15.17:0/4161154731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58101d30 0x7f0f58197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 >> 172.21.15.17:0/4161154731 conn(0x7f0f580fb6e0 msgr2=0x7f0f58102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 shutdown_connections 2024-02-14T15:03:01.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.464+0000 7f0f5d639640 1 -- 172.21.15.17:0/4161154731 wait complete. 2024-02-14T15:03:01.475 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 -- 172.21.15.17:0/1783283953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00101340 msgr2=0x7fee001017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 --2- 172.21.15.17:0/1783283953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00101340 0x7fee001017c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009900 tx=0x7fedf002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 -- 172.21.15.17:0/1783283953 shutdown_connections 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 --2- 172.21.15.17:0/1783283953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00101340 0x7fee001017c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 --2- 172.21.15.17:0/1783283953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee00101d00 0x7fee00108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 --2- 172.21.15.17:0/1783283953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee00100150 0x7fee00100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 -- 172.21.15.17:0/1783283953 >> 172.21.15.17:0/1783283953 conn(0x7fee000fb6c0 msgr2=0x7fee000fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:01.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 -- 172.21.15.17:0/1783283953 shutdown_connections 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.580+0000 7fee06040640 1 -- 172.21.15.17:0/1783283953 wait complete. 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 Processor -- start 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 -- start start 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00100150 0x7fee001040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee00101340 0x7fee00102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee00101d00 0x7fee00102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee001033c0 con 0x7fee00100150 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee00103530 con 0x7fee00101d00 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fee06040640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee001a13b0 con 0x7fee00101340 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fedff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00100150 0x7fee001040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fedff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00100150 0x7fee001040e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57934/0 (socket says 172.21.15.17:57934) 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.581+0000 7fedff7fe640 1 -- 172.21.15.17:0/2353058275 learned_addr learned my addr 172.21.15.17:0/2353058275 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:01.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedfeffd640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee00101340 0x7fee00102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedfffff640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee00101d00 0x7fee00102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedff7fe640 1 -- 172.21.15.17:0/2353058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee00101340 msgr2=0x7fee00102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedff7fe640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee00101340 0x7fee00102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedff7fe640 1 -- 172.21.15.17:0/2353058275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee00101d00 msgr2=0x7fee00102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedff7fe640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee00101d00 0x7fee00102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedff7fe640 1 -- 172.21.15.17:0/2353058275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0009560 con 0x7fee00100150 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedff7fe640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00100150 0x7fee001040e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e9f0 tx=0x7fedf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedfcff9640 1 -- 172.21.15.17:0/2353058275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf40096e0 con 0x7fee00100150 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedfcff9640 1 -- 172.21.15.17:0/2353058275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf400cd80 con 0x7fee00100150 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee001a1660 con 0x7fee00100150 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fedfcff9640 1 -- 172.21.15.17:0/2353058275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4010430 con 0x7fee00100150 2024-02-14T15:03:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.582+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee001a1b20 con 0x7fee00100150 2024-02-14T15:03:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.583+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0010bb60 con 0x7fee00100150 2024-02-14T15:03:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.585+0000 7fedfcff9640 1 -- 172.21.15.17:0/2353058275 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fedf4003680 con 0x7fee00100150 2024-02-14T15:03:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.586+0000 7fedfcff9640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fede80a20c0 0x7fede80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.586+0000 7fedfeffd640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fede80a20c0 0x7fede80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.586+0000 7fedfcff9640 1 -- 172.21.15.17:0/2353058275 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fedf4014070 con 0x7fee00100150 2024-02-14T15:03:01.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.586+0000 7fedfeffd640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fede80a20c0 0x7fede80a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee00103e50 tx=0x7fedf0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:01.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.591+0000 7fedfcff9640 1 -- 172.21.15.17:0/2353058275 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fedf4097100 con 0x7fee00100150 2024-02-14T15:03:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.771+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fee001a2040 con 0x7fee00100150 2024-02-14T15:03:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.772+0000 7fedfcff9640 1 -- 172.21.15.17:0/2353058275 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fedf409a7c0 con 0x7fee00100150 2024-02-14T15:03:01.773 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fede80a20c0 msgr2=0x7fede80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fede80a20c0 0x7fede80a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee00103e50 tx=0x7fedf0000bc0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00100150 msgr2=0x7fee001040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00100150 0x7fee001040e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e9f0 tx=0x7fedf400eeb0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 shutdown_connections 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fede80a20c0 0x7fede80a4580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee00101d00 0x7fee00102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee00101340 0x7fee00102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 --2- 172.21.15.17:0/2353058275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee00100150 0x7fee001040e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 >> 172.21.15.17:0/2353058275 conn(0x7fee000fb6c0 msgr2=0x7fee000fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 shutdown_connections 2024-02-14T15:03:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.774+0000 7fee06040640 1 -- 172.21.15.17:0/2353058275 wait complete. 2024-02-14T15:03:01.786 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:03:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.887+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2813023248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 msgr2=0x7f5a64104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.887+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2813023248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a64104150 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009900 tx=0x7f5a5402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.887+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2813023248 shutdown_connections 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.887+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2813023248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a64106fc0 0x7f5a641093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.887+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2813023248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a64104690 0x7f5a64106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.887+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2813023248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a64104150 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.887+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2813023248 >> 172.21.15.17:0/2813023248 conn(0x7f5a640fb6d0 msgr2=0x7f5a640fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2813023248 shutdown_connections 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2813023248 wait complete. 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 Processor -- start 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 -- start start 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a641972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a64104690 0x7f5a64197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a64106fc0 0x7f5a6419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6419edb0 con 0x7f5a64101d60 2024-02-14T15:03:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6419ef20 con 0x7f5a64104690 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a6a4d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6419f0e0 con 0x7f5a64106fc0 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.888+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a641972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a641972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57952/0 (socket says 172.21.15.17:57952) 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 -- 172.21.15.17:0/2294982175 learned_addr learned my addr 172.21.15.17:0/2294982175 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a68a71640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a64106fc0 0x7f5a6419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 -- 172.21.15.17:0/2294982175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a64106fc0 msgr2=0x7f5a6419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a64106fc0 0x7f5a6419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 -- 172.21.15.17:0/2294982175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a64104690 msgr2=0x7f5a64197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a637fe640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a64104690 0x7f5a64197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a64104690 0x7f5a64197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 -- 172.21.15.17:0/2294982175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54009560 con 0x7f5a64101d60 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a637fe640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a64104690 0x7f5a64197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a63fff640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a641972c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5a54006010 tx=0x7f5a540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a617fa640 1 -- 172.21.15.17:0/2294982175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540044e0 con 0x7f5a64101d60 2024-02-14T15:03:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a617fa640 1 -- 172.21.15.17:0/2294982175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a540054e0 con 0x7f5a64101d60 2024-02-14T15:03:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a617fa640 1 -- 172.21.15.17:0/2294982175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5403e450 con 0x7f5a64101d60 2024-02-14T15:03:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6419f360 con 0x7f5a64101d60 2024-02-14T15:03:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.889+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6419f870 con 0x7f5a64101d60 2024-02-14T15:03:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.890+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a640fd1c0 con 0x7f5a64101d60 2024-02-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.891+0000 7f5a617fa640 1 -- 172.21.15.17:0/2294982175 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5a540035f0 con 0x7f5a64101d60 2024-02-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.892+0000 7f5a617fa640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5a4c0a20c0 0x7f5a4c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.892+0000 7f5a637fe640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5a4c0a20c0 0x7f5a4c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.892+0000 7f5a617fa640 1 -- 172.21.15.17:0/2294982175 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f5a540f0450 con 0x7f5a64101d60 2024-02-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.892+0000 7f5a637fe640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5a4c0a20c0 0x7f5a4c0a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5a641987d0 tx=0x7f5a58005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:01.896+0000 7f5a617fa640 1 -- 172.21.15.17:0/2294982175 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a540b9070 con 0x7f5a64101d60 2024-02-14T15:03:02.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:01 smithi017 ceph-mon[32549]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:03:02.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:01 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/54054952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:01 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/957697994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:01 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4161154731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.115+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a640fcc30 con 0x7f5a64101d60 2024-02-14T15:03:02.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.116+0000 7f5a617fa640 1 -- 172.21.15.17:0/2294982175 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f5a540bc730 con 0x7f5a64101d60 2024-02-14T15:03:02.117 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:02.117 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:01:15.551062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:03:02.117 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5a4c0a20c0 msgr2=0x7f5a4c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5a4c0a20c0 0x7f5a4c0a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5a641987d0 tx=0x7f5a58005d30 comp rx=0 tx=0).stop 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 msgr2=0x7f5a641972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a641972c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5a54006010 tx=0x7f5a540042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 shutdown_connections 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f5a4c0a20c0 0x7f5a4c0a4580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a64106fc0 0x7f5a6419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a64104690 0x7f5a64197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 --2- 172.21.15.17:0/2294982175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a64101d60 0x7f5a641972c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 >> 172.21.15.17:0/2294982175 conn(0x7f5a640fb6d0 msgr2=0x7f5a64102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:02.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 shutdown_connections 2024-02-14T15:03:02.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.118+0000 7f5a6a4d4640 1 -- 172.21.15.17:0/2294982175 wait complete. 2024-02-14T15:03:02.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:03:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:01 smithi050 ceph-mon[36512]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:03:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:01 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/54054952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:01 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/957697994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:01 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4161154731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 -- 172.21.15.17:0/3366881318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434101d10 msgr2=0x7f74341085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 --2- 172.21.15.17:0/3366881318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434101d10 0x7f74341085a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f742c009ca0 tx=0x7f742c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 -- 172.21.15.17:0/3366881318 shutdown_connections 2024-02-14T15:03:02.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 --2- 172.21.15.17:0/3366881318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434101d10 0x7f74341085a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 --2- 172.21.15.17:0/3366881318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7434101350 0x7f74341017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 --2- 172.21.15.17:0/3366881318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7434100160 0x7f7434100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 -- 172.21.15.17:0/3366881318 >> 172.21.15.17:0/3366881318 conn(0x7f74340fb6f0 msgr2=0x7f74340fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:02.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 -- 172.21.15.17:0/3366881318 shutdown_connections 2024-02-14T15:03:02.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.233+0000 7f7438c15640 1 -- 172.21.15.17:0/3366881318 wait complete. 2024-02-14T15:03:02.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:01 smithi076 ceph-mon[37057]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:03:02.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:01 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/54054952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:01 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/957697994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:01 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4161154731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:02.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 Processor -- start 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 -- start start 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434100160 0x7f7434196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7434101350 0x7f74341966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7434101d10 0x7f743419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743419dc80 con 0x7f7434100160 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743419ddf0 con 0x7f7434101d10 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7438c15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743419dfb0 con 0x7f7434101350 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f743259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434100160 0x7f7434196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f743259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434100160 0x7f7434196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57958/0 (socket says 172.21.15.17:57958) 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f743259c640 1 -- 172.21.15.17:0/1649631623 learned_addr learned my addr 172.21.15.17:0/1649631623 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f7432d9d640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7434101d10 0x7f743419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f742bfff640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7434101350 0x7f74341966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.234+0000 7f743259c640 1 -- 172.21.15.17:0/1649631623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7434101350 msgr2=0x7f74341966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f743259c640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7434101350 0x7f74341966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f743259c640 1 -- 172.21.15.17:0/1649631623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7434101d10 msgr2=0x7f743419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f743259c640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7434101d10 0x7f743419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f743259c640 1 -- 172.21.15.17:0/1649631623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c009900 con 0x7f7434100160 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f742bfff640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7434101350 0x7f74341966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f743259c640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434100160 0x7f7434196190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f742400d950 tx=0x7f742400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f742b7fe640 1 -- 172.21.15.17:0/1649631623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424009c30 con 0x7f7434100160 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f742b7fe640 1 -- 172.21.15.17:0/1649631623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7424009d90 con 0x7f7434100160 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743419e290 con 0x7f7434100160 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f742b7fe640 1 -- 172.21.15.17:0/1649631623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400f720 con 0x7f7434100160 2024-02-14T15:03:02.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.235+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743419e750 con 0x7f7434100160 2024-02-14T15:03:02.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.236+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7434107420 con 0x7f7434100160 2024-02-14T15:03:02.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.238+0000 7f742b7fe640 1 -- 172.21.15.17:0/1649631623 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7424010a60 con 0x7f7434100160 2024-02-14T15:03:02.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.238+0000 7f742b7fe640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f74200a20c0 0x7f74200a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.238+0000 7f742b7fe640 1 -- 172.21.15.17:0/1649631623 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f74240cdce0 con 0x7f7434100160 2024-02-14T15:03:02.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.238+0000 7f742bfff640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f74200a20c0 0x7f74200a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.239+0000 7f742bfff640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f74200a20c0 0x7f74200a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f74341976a0 tx=0x7f741c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:02.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.242+0000 7f742b7fe640 1 -- 172.21.15.17:0/1649631623 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7424096960 con 0x7f7434100160 2024-02-14T15:03:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.424+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f743410bd10 con 0x7f7434100160 2024-02-14T15:03:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.426+0000 7f742b7fe640 1 -- 172.21.15.17:0/1649631623 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f743410bd10 con 0x7f7434100160 2024-02-14T15:03:02.427 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:02.427 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:02:45.606961+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f74200a20c0 msgr2=0x7f74200a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f74200a20c0 0x7f74200a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f74341976a0 tx=0x7f741c005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434100160 msgr2=0x7f7434196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434100160 0x7f7434196190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f742400d950 tx=0x7f742400de10 comp rx=0 tx=0).stop 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 shutdown_connections 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f74200a20c0 0x7f74200a4580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7434101d10 0x7f743419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7434101350 0x7f74341966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 --2- 172.21.15.17:0/1649631623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7434100160 0x7f7434196190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 >> 172.21.15.17:0/1649631623 conn(0x7f74340fb6f0 msgr2=0x7f74341045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:02.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 shutdown_connections 2024-02-14T15:03:02.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.428+0000 7f7438c15640 1 -- 172.21.15.17:0/1649631623 wait complete. 2024-02-14T15:03:02.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:03:02.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 -- 172.21.15.17:0/979250581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 msgr2=0x7f9afc109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 --2- 172.21.15.17:0/979250581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc109380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009ca0 tx=0x7f9af402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 -- 172.21.15.17:0/979250581 shutdown_connections 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 --2- 172.21.15.17:0/979250581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc109380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 --2- 172.21.15.17:0/979250581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc104600 0x7f9afc1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 --2- 172.21.15.17:0/979250581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9afc101cd0 0x7f9afc1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 -- 172.21.15.17:0/979250581 >> 172.21.15.17:0/979250581 conn(0x7f9afc0fb6c0 msgr2=0x7f9afc0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:02.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 -- 172.21.15.17:0/979250581 shutdown_connections 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.546+0000 7f9b0236b640 1 -- 172.21.15.17:0/979250581 wait complete. 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 Processor -- start 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 -- start start 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9afc101cd0 0x7f9afc101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc104600 0x7f9afc0ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc100920 con 0x7f9afc106f30 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc100a90 con 0x7f9afc104600 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b0236b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1a12f0 con 0x7f9afc101cd0 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc1001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57980/0 (socket says 172.21.15.17:57980) 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 -- 172.21.15.17:0/547539273 learned_addr learned my addr 172.21.15.17:0/547539273 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9afbfff640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9afc101cd0 0x7f9afc101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9afb7fe640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc104600 0x7f9afc0ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 -- 172.21.15.17:0/547539273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9afc101cd0 msgr2=0x7f9afc101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9afc101cd0 0x7f9afc101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 -- 172.21.15.17:0/547539273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc104600 msgr2=0x7f9afc0ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc104600 0x7f9afc0ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9b00908640 1 -- 172.21.15.17:0/547539273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4009900 con 0x7f9afc106f30 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.547+0000 7f9afbfff640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9afc101cd0 0x7f9afc101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.548+0000 7f9b00908640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc1001c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9af40098d0 tx=0x7f9af4034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.548+0000 7f9af97fa640 1 -- 172.21.15.17:0/547539273 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4032eb0 con 0x7f9afc106f30 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.548+0000 7f9af97fa640 1 -- 172.21.15.17:0/547539273 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9af4003e50 con 0x7f9afc106f30 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.548+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc1a1540 con 0x7f9afc106f30 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.548+0000 7f9af97fa640 1 -- 172.21.15.17:0/547539273 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40406b0 con 0x7f9afc106f30 2024-02-14T15:03:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.548+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc1a19f0 con 0x7f9afc106f30 2024-02-14T15:03:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.549+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc0fd1b0 con 0x7f9afc106f30 2024-02-14T15:03:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.550+0000 7f9af97fa640 1 -- 172.21.15.17:0/547539273 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9af400a040 con 0x7f9afc106f30 2024-02-14T15:03:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.550+0000 7f9af97fa640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ae40a20c0 0x7f9ae40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.550+0000 7f9afbfff640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ae40a20c0 0x7f9ae40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.550+0000 7f9af97fa640 1 -- 172.21.15.17:0/547539273 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f9af40f0fc0 con 0x7f9afc106f30 2024-02-14T15:03:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.551+0000 7f9afbfff640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ae40a20c0 0x7f9ae40a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009b40 tx=0x7f9af0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:02.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.555+0000 7f9af97fa640 1 -- 172.21.15.17:0/547539273 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9af40b9cd0 con 0x7f9afc106f30 2024-02-14T15:03:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.738+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9afc02ce50 con 0x7f9afc106f30 2024-02-14T15:03:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.739+0000 7f9af97fa640 1 -- 172.21.15.17:0/547539273 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9af4004860 con 0x7f9afc106f30 2024-02-14T15:03:02.740 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:03:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ae40a20c0 msgr2=0x7f9ae40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ae40a20c0 0x7f9ae40a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009b40 tx=0x7f9af0005e50 comp rx=0 tx=0).stop 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 msgr2=0x7f9afc1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc1001c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9af40098d0 tx=0x7f9af4034d40 comp rx=0 tx=0).stop 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 shutdown_connections 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f9ae40a20c0 0x7f9ae40a4580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9afc106f30 0x7f9afc1001c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9afc104600 0x7f9afc0ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 --2- 172.21.15.17:0/547539273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9afc101cd0 0x7f9afc101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 >> 172.21.15.17:0/547539273 conn(0x7f9afc0fb6c0 msgr2=0x7f9afc102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 shutdown_connections 2024-02-14T15:03:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.741+0000 7f9b0236b640 1 -- 172.21.15.17:0/547539273 wait complete. 2024-02-14T15:03:02.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 -- 172.21.15.17:0/1757513067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102df0 msgr2=0x7fb5f4109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/1757513067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102df0 0x7fb5f4109680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009ca0 tx=0x7fb5ec02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 -- 172.21.15.17:0/1757513067 shutdown_connections 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/1757513067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102df0 0x7fb5f4109680 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/1757513067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f4102430 0x7fb5f41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/1757513067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f4101240 0x7fb5f4101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 -- 172.21.15.17:0/1757513067 >> 172.21.15.17:0/1757513067 conn(0x7fb5f40fc7b0 msgr2=0x7fb5f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 -- 172.21.15.17:0/1757513067 shutdown_connections 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 -- 172.21.15.17:0/1757513067 wait complete. 2024-02-14T15:03:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.859+0000 7fb5f9c83640 1 Processor -- start 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f9c83640 1 -- start start 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f9c83640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f4101240 0x7fb5f41972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f9c83640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102430 0x7fb5f41977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f9c83640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f4102df0 0x7fb5f419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f9c83640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f419ed90 con 0x7fb5f4102430 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f9c83640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f419ef00 con 0x7fb5f4101240 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f9c83640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f419f0c0 con 0x7fb5f4102df0 2024-02-14T15:03:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102430 0x7fb5f41977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102430 0x7fb5f41977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58002/0 (socket says 172.21.15.17:58002) 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 -- 172.21.15.17:0/4167959691 learned_addr learned my addr 172.21.15.17:0/4167959691 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f37fe640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f4101240 0x7fb5f41972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f3fff640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f4102df0 0x7fb5f419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 -- 172.21.15.17:0/4167959691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f4102df0 msgr2=0x7fb5f419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f4102df0 0x7fb5f419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 -- 172.21.15.17:0/4167959691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f4101240 msgr2=0x7fb5f41972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f4101240 0x7fb5f41972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 -- 172.21.15.17:0/4167959691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec009900 con 0x7fb5f4102430 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f3fff640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f4102df0 0x7fb5f419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f2ffd640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102430 0x7fb5f41977e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c970 tx=0x7fb5e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.860+0000 7fb5f0ff9640 1 -- 172.21.15.17:0/4167959691 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007bb0 con 0x7fb5f4102430 2024-02-14T15:03:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.861+0000 7fb5f0ff9640 1 -- 172.21.15.17:0/4167959691 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5e8007d10 con 0x7fb5f4102430 2024-02-14T15:03:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.861+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f419f3a0 con 0x7fb5f4102430 2024-02-14T15:03:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.861+0000 7fb5f0ff9640 1 -- 172.21.15.17:0/4167959691 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800f450 con 0x7fb5f4102430 2024-02-14T15:03:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.861+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f419f860 con 0x7fb5f4102430 2024-02-14T15:03:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.862+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f41085e0 con 0x7fb5f4102430 2024-02-14T15:03:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.863+0000 7fb5f0ff9640 1 -- 172.21.15.17:0/4167959691 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb5e8004f50 con 0x7fb5f4102430 2024-02-14T15:03:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.864+0000 7fb5f0ff9640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5dc0a20c0 0x7fb5dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.864+0000 7fb5f37fe640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5dc0a20c0 0x7fb5dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.864+0000 7fb5f0ff9640 1 -- 172.21.15.17:0/4167959691 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fb5e80ce120 con 0x7fb5f4102430 2024-02-14T15:03:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.864+0000 7fb5f37fe640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5dc0a20c0 0x7fb5dc0a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4102290 tx=0x7fb5e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:02.868+0000 7fb5f0ff9640 1 -- 172.21.15.17:0/4167959691 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5e8096da0 con 0x7fb5f4102430 2024-02-14T15:03:03.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2353058275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:03.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2294982175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:03.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1649631623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:03.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:02 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/547539273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.047+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5f410cde0 con 0x7fb5f4102430 2024-02-14T15:03:03.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.047+0000 7fb5f0ff9640 1 -- 172.21.15.17:0/4167959691 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5e801d0a0 con 0x7fb5f4102430 2024-02-14T15:03:03.048 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5dc0a20c0 msgr2=0x7fb5dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5dc0a20c0 0x7fb5dc0a4580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4102290 tx=0x7fb5e400b390 comp rx=0 tx=0).stop 2024-02-14T15:03:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102430 msgr2=0x7fb5f41977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102430 0x7fb5f41977e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c970 tx=0x7fb5e800ce30 comp rx=0 tx=0).stop 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 shutdown_connections 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fb5dc0a20c0 0x7fb5dc0a4580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5f4102430 0x7fb5f41977e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f4102df0 0x7fb5f419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 --2- 172.21.15.17:0/4167959691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f4101240 0x7fb5f41972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 >> 172.21.15.17:0/4167959691 conn(0x7fb5f40fc7b0 msgr2=0x7fb5f41057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.049+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 shutdown_connections 2024-02-14T15:03:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.050+0000 7fb5f9c83640 1 -- 172.21.15.17:0/4167959691 wait complete. 2024-02-14T15:03:03.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:03:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2353058275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2294982175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1649631623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:02 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/547539273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.166+0000 7fba974eb640 1 -- 172.21.15.17:0/959435181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101340 msgr2=0x7fba901017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.166+0000 7fba974eb640 1 --2- 172.21.15.17:0/959435181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101340 0x7fba901017c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fba80009900 tx=0x7fba8002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.166+0000 7fba974eb640 1 -- 172.21.15.17:0/959435181 shutdown_connections 2024-02-14T15:03:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.166+0000 7fba974eb640 1 --2- 172.21.15.17:0/959435181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101340 0x7fba901017c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.166+0000 7fba974eb640 1 --2- 172.21.15.17:0/959435181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba90101d00 0x7fba90108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.166+0000 7fba974eb640 1 --2- 172.21.15.17:0/959435181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba90100150 0x7fba90100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.166+0000 7fba974eb640 1 -- 172.21.15.17:0/959435181 >> 172.21.15.17:0/959435181 conn(0x7fba900fb6c0 msgr2=0x7fba900fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 -- 172.21.15.17:0/959435181 shutdown_connections 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 -- 172.21.15.17:0/959435181 wait complete. 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 Processor -- start 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 -- start start 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba90100150 0x7fba90069090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba90101340 0x7fba9019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101d00 0x7fba9019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9019ee00 con 0x7fba90101d00 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9019ef70 con 0x7fba90100150 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba974eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9019f130 con 0x7fba90101340 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba95a88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101d00 0x7fba9019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba95a88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101d00 0x7fba9019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58012/0 (socket says 172.21.15.17:58012) 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.167+0000 7fba95a88640 1 -- 172.21.15.17:0/3812895372 learned_addr learned my addr 172.21.15.17:0/3812895372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:03.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba94a86640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba90101340 0x7fba9019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba95287640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba90100150 0x7fba90069090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba95a88640 1 -- 172.21.15.17:0/3812895372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba90101340 msgr2=0x7fba9019c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba95a88640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba90101340 0x7fba9019c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba95a88640 1 -- 172.21.15.17:0/3812895372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba90100150 msgr2=0x7fba90069090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba95a88640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba90100150 0x7fba90069090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba95a88640 1 -- 172.21.15.17:0/3812895372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80009560 con 0x7fba90101d00 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba95a88640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101d00 0x7fba9019e670 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009fd0 tx=0x7fba8c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba867fc640 1 -- 172.21.15.17:0/3812895372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00bea0 con 0x7fba90101d00 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9019f410 con 0x7fba90101d00 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba867fc640 1 -- 172.21.15.17:0/3812895372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba8c009940 con 0x7fba90101d00 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba867fc640 1 -- 172.21.15.17:0/3812895372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0106c0 con 0x7fba90101d00 2024-02-14T15:03:03.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.168+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9019f920 con 0x7fba90101d00 2024-02-14T15:03:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.169+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba901074f0 con 0x7fba90101d00 2024-02-14T15:03:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.170+0000 7fba867fc640 1 -- 172.21.15.17:0/3812895372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fba8c004240 con 0x7fba90101d00 2024-02-14T15:03:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.170+0000 7fba867fc640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fba7c0a20c0 0x7fba7c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.171+0000 7fba95287640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fba7c0a20c0 0x7fba7c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.171+0000 7fba867fc640 1 -- 172.21.15.17:0/3812895372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fba8c0cde20 con 0x7fba90101d00 2024-02-14T15:03:03.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.171+0000 7fba95287640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fba7c0a20c0 0x7fba7c0a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fba901011a0 tx=0x7fba88005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:03.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.175+0000 7fba867fc640 1 -- 172.21.15.17:0/3812895372 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba8c096a40 con 0x7fba90101d00 2024-02-14T15:03:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2353058275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2294982175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1649631623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:02 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/547539273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:03.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.357+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba9002cde0 con 0x7fba90101d00 2024-02-14T15:03:03.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.358+0000 7fba867fc640 1 -- 172.21.15.17:0/3812895372 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba8c017020 con 0x7fba90101d00 2024-02-14T15:03:03.359 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:03:03.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fba7c0a20c0 msgr2=0x7fba7c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fba7c0a20c0 0x7fba7c0a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fba901011a0 tx=0x7fba88005c60 comp rx=0 tx=0).stop 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101d00 msgr2=0x7fba9019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101d00 0x7fba9019e670 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009fd0 tx=0x7fba8c00b260 comp rx=0 tx=0).stop 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 shutdown_connections 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fba7c0a20c0 0x7fba7c0a4580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba90101d00 0x7fba9019e670 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba90101340 0x7fba9019c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 --2- 172.21.15.17:0/3812895372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba90100150 0x7fba90069090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 >> 172.21.15.17:0/3812895372 conn(0x7fba900fb6c0 msgr2=0x7fba901046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 shutdown_connections 2024-02-14T15:03:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.360+0000 7fba974eb640 1 -- 172.21.15.17:0/3812895372 wait complete. 2024-02-14T15:03:03.371 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:03:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 -- 172.21.15.17:0/2325608317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628106f30 msgr2=0x7f3628109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 --2- 172.21.15.17:0/2325608317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628106f30 0x7f3628109380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3620009ca0 tx=0x7f362002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 -- 172.21.15.17:0/2325608317 shutdown_connections 2024-02-14T15:03:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 --2- 172.21.15.17:0/2325608317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628106f30 0x7f3628109380 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 --2- 172.21.15.17:0/2325608317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3628104600 0x7f36281069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 --2- 172.21.15.17:0/2325608317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3628101cd0 0x7f36281040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 -- 172.21.15.17:0/2325608317 >> 172.21.15.17:0/2325608317 conn(0x7f36280fb6e0 msgr2=0x7f36280fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 -- 172.21.15.17:0/2325608317 shutdown_connections 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.476+0000 7f362f9f7640 1 -- 172.21.15.17:0/2325608317 wait complete. 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 Processor -- start 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 -- start start 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3628101cd0 0x7f3628197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628104600 0x7f36281977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3628106f30 0x7f362819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362819ed80 con 0x7f3628104600 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362819eef0 con 0x7f3628106f30 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362f9f7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362819f0b0 con 0x7f3628101cd0 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628104600 0x7f36281977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628104600 0x7f36281977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58022/0 (socket says 172.21.15.17:58022) 2024-02-14T15:03:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 -- 172.21.15.17:0/3005227278 learned_addr learned my addr 172.21.15.17:0/3005227278 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362df94640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3628106f30 0x7f362819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362d793640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3628101cd0 0x7f3628197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 -- 172.21.15.17:0/3005227278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3628101cd0 msgr2=0x7f3628197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3628101cd0 0x7f3628197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 -- 172.21.15.17:0/3005227278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3628106f30 msgr2=0x7f362819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3628106f30 0x7f362819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 -- 172.21.15.17:0/3005227278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620009900 con 0x7f3628104600 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.477+0000 7f362cf92640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628104600 0x7f36281977d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f361800c970 tx=0x7f361800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.478+0000 7f361e7fc640 1 -- 172.21.15.17:0/3005227278 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618007bb0 con 0x7f3628104600 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.478+0000 7f361e7fc640 1 -- 172.21.15.17:0/3005227278 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3618007d10 con 0x7f3628104600 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.478+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362819f390 con 0x7f3628104600 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.478+0000 7f361e7fc640 1 -- 172.21.15.17:0/3005227278 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800f450 con 0x7f3628104600 2024-02-14T15:03:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.478+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362819f8f0 con 0x7f3628104600 2024-02-14T15:03:03.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.479+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36280fd1d0 con 0x7f3628104600 2024-02-14T15:03:03.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.480+0000 7f361e7fc640 1 -- 172.21.15.17:0/3005227278 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3618004f20 con 0x7f3628104600 2024-02-14T15:03:03.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.480+0000 7f361e7fc640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36140a2140 0x7f36140a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.481+0000 7f361e7fc640 1 -- 172.21.15.17:0/3005227278 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f36180cdf80 con 0x7f3628104600 2024-02-14T15:03:03.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.481+0000 7f362d793640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36140a2140 0x7f36140a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.481+0000 7f362d793640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36140a2140 0x7f36140a4600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3624005fd0 tx=0x7f3624005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.484+0000 7f361e7fc640 1 -- 172.21.15.17:0/3005227278 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3618096c80 con 0x7f3628104600 2024-02-14T15:03:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.665+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f362810ceb0 con 0x7f3628104600 2024-02-14T15:03:03.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.665+0000 7f361e7fc640 1 -- 172.21.15.17:0/3005227278 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f361801d070 con 0x7f3628104600 2024-02-14T15:03:03.666 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36140a2140 msgr2=0x7f36140a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36140a2140 0x7f36140a4600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3624005fd0 tx=0x7f3624005e20 comp rx=0 tx=0).stop 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628104600 msgr2=0x7f36281977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628104600 0x7f36281977d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f361800c970 tx=0x7f361800ce30 comp rx=0 tx=0).stop 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 shutdown_connections 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f36140a2140 0x7f36140a4600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3628104600 0x7f36281977d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3628106f30 0x7f362819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 --2- 172.21.15.17:0/3005227278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3628101cd0 0x7f3628197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 >> 172.21.15.17:0/3005227278 conn(0x7f36280fb6e0 msgr2=0x7f3628102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 shutdown_connections 2024-02-14T15:03:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.667+0000 7f362f9f7640 1 -- 172.21.15.17:0/3005227278 wait complete. 2024-02-14T15:03:03.679 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:03:03.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 -- 172.21.15.17:0/1165224669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02141013b0 msgr2=0x7f0214101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1165224669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02141013b0 0x7f0214101830 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0204009900 tx=0x7f020402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 -- 172.21.15.17:0/1165224669 shutdown_connections 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1165224669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02141013b0 0x7f0214101830 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1165224669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0214101d70 0x7f0214108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1165224669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02141001c0 0x7f02141005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 -- 172.21.15.17:0/1165224669 >> 172.21.15.17:0/1165224669 conn(0x7f02140fb710 msgr2=0x7f02140fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 -- 172.21.15.17:0/1165224669 shutdown_connections 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 -- 172.21.15.17:0/1165224669 wait complete. 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 Processor -- start 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.781+0000 7f021bf1e640 1 -- start start 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02141001c0 0x7f021406a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02141013b0 0x7f021419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0214101d70 0x7f021419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021419ee00 con 0x7f0214101d70 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021419ef70 con 0x7f02141001c0 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021419f130 con 0x7f02141013b0 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0214101d70 0x7f021419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0214101d70 0x7f021419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58038/0 (socket says 172.21.15.17:58038) 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 -- 172.21.15.17:0/1441647307 learned_addr learned my addr 172.21.15.17:0/1441647307 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f02194b9640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02141013b0 0x7f021419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f0219cba640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02141001c0 0x7f021406a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 -- 172.21.15.17:0/1441647307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02141013b0 msgr2=0x7f021419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02141013b0 0x7f021419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 -- 172.21.15.17:0/1441647307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02141001c0 msgr2=0x7f021406a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02141001c0 0x7f021406a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 -- 172.21.15.17:0/1441647307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0204009560 con 0x7f0214101d70 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021a4bb640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0214101d70 0x7f021419e620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f020c00eaa0 tx=0x7f020c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f020affd640 1 -- 172.21.15.17:0/1441647307 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c00cdb0 con 0x7f0214101d70 2024-02-14T15:03:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021419f3b0 con 0x7f0214101d70 2024-02-14T15:03:03.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f020affd640 1 -- 172.21.15.17:0/1441647307 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f020c005020 con 0x7f0214101d70 2024-02-14T15:03:03.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f020affd640 1 -- 172.21.15.17:0/1441647307 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0116b0 con 0x7f0214101d70 2024-02-14T15:03:03.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.782+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021419f890 con 0x7f0214101d70 2024-02-14T15:03:03.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.783+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02141074e0 con 0x7f0214101d70 2024-02-14T15:03:03.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.784+0000 7f020affd640 1 -- 172.21.15.17:0/1441647307 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f020c005520 con 0x7f0214101d70 2024-02-14T15:03:03.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.785+0000 7f020affd640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02000a20c0 0x7f02000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:03.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.785+0000 7f0219cba640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02000a20c0 0x7f02000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:03.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.785+0000 7f020affd640 1 -- 172.21.15.17:0/1441647307 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f020c0ce630 con 0x7f0214101d70 2024-02-14T15:03:03.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.785+0000 7f0219cba640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02000a20c0 0x7f02000a4580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0210005fd0 tx=0x7f0210005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:03.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:03.789+0000 7f020affd640 1 -- 172.21.15.17:0/1441647307 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f020c0972b0 con 0x7f0214101d70 2024-02-14T15:03:04.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.008+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f021410bc80 con 0x7f0214101d70 2024-02-14T15:03:04.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.008+0000 7f020affd640 1 -- 172.21.15.17:0/1441647307 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f020c01f020 con 0x7f0214101d70 2024-02-14T15:03:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:04.010 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:01:15.551062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:03:04.010 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-02-14T15:03:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02000a20c0 msgr2=0x7f02000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02000a20c0 0x7f02000a4580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0210005fd0 tx=0x7f0210005850 comp rx=0 tx=0).stop 2024-02-14T15:03:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0214101d70 msgr2=0x7f021419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0214101d70 0x7f021419e620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f020c00eaa0 tx=0x7f020c00ef60 comp rx=0 tx=0).stop 2024-02-14T15:03:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 shutdown_connections 2024-02-14T15:03:04.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f02000a20c0 0x7f02000a4580 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0214101d70 0x7f021419e620 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02141013b0 0x7f021419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 --2- 172.21.15.17:0/1441647307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02141001c0 0x7f021406a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 >> 172.21.15.17:0/1441647307 conn(0x7f02140fb710 msgr2=0x7f02141046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 shutdown_connections 2024-02-14T15:03:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.010+0000 7f021bf1e640 1 -- 172.21.15.17:0/1441647307 wait complete. 2024-02-14T15:03:04.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:03 smithi017 ceph-mon[32549]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:03:04.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:03 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4167959691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:03 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3812895372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:03 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3005227278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.023 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:03:04.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 -- 172.21.15.17:0/689325075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102560 msgr2=0x7f17941029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 --2- 172.21.15.17:0/689325075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102560 0x7f17941029e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1780009900 tx=0x7f178002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:04.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 -- 172.21.15.17:0/689325075 shutdown_connections 2024-02-14T15:03:04.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 --2- 172.21.15.17:0/689325075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102560 0x7f17941029e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 --2- 172.21.15.17:0/689325075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1794102f20 0x7f17941097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 --2- 172.21.15.17:0/689325075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17940ff590 0x7f17940ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 -- 172.21.15.17:0/689325075 >> 172.21.15.17:0/689325075 conn(0x7f17940faea0 msgr2=0x7f17940fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 -- 172.21.15.17:0/689325075 shutdown_connections 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.124+0000 7f179ad25640 1 -- 172.21.15.17:0/689325075 wait complete. 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 Processor -- start 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 -- start start 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17940ff590 0x7f1794101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1794102560 0x7f1794100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102f20 0x7f1794100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17941011b0 con 0x7f1794102f20 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1794101320 con 0x7f17940ff590 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f179ad25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17941a12b0 con 0x7f1794102560 2024-02-14T15:03:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102f20 0x7f1794100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f1793fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1794102560 0x7f1794100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102f20 0x7f1794100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58062/0 (socket says 172.21.15.17:58062) 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 -- 172.21.15.17:0/1223252340 learned_addr learned my addr 172.21.15.17:0/1223252340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f1798ac1640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17940ff590 0x7f1794101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 -- 172.21.15.17:0/1223252340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1794102560 msgr2=0x7f1794100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1794102560 0x7f1794100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 -- 172.21.15.17:0/1223252340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17940ff590 msgr2=0x7f1794101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17940ff590 0x7f1794101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f17992c2640 1 -- 172.21.15.17:0/1223252340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780009560 con 0x7f1794102f20 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.125+0000 7f1798ac1640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17940ff590 0x7f1794101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.126+0000 7f17992c2640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102f20 0x7f1794100a50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f178c00cb00 tx=0x7f178c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.126+0000 7f1791ffb640 1 -- 172.21.15.17:0/1223252340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00ae90 con 0x7f1794102f20 2024-02-14T15:03:04.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.126+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17941a1560 con 0x7f1794102f20 2024-02-14T15:03:04.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.126+0000 7f1791ffb640 1 -- 172.21.15.17:0/1223252340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c005020 con 0x7f1794102f20 2024-02-14T15:03:04.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.126+0000 7f1791ffb640 1 -- 172.21.15.17:0/1223252340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00f6b0 con 0x7f1794102f20 2024-02-14T15:03:04.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.126+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17941a1a40 con 0x7f1794102f20 2024-02-14T15:03:04.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.127+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1794108690 con 0x7f1794102f20 2024-02-14T15:03:04.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.128+0000 7f1791ffb640 1 -- 172.21.15.17:0/1223252340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f178c005520 con 0x7f1794102f20 2024-02-14T15:03:04.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.128+0000 7f1791ffb640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17840a20c0 0x7f17840a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.129+0000 7f1791ffb640 1 -- 172.21.15.17:0/1223252340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f178c0ce000 con 0x7f1794102f20 2024-02-14T15:03:04.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.129+0000 7f1798ac1640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17840a20c0 0x7f17840a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.129+0000 7f1798ac1640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17840a20c0 0x7f17840a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17940697b0 tx=0x7f1788005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.133+0000 7f1791ffb640 1 -- 172.21.15.17:0/1223252340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f178c096c80 con 0x7f1794102f20 2024-02-14T15:03:04.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:03 smithi050 ceph-mon[36512]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:03:04.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:03 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4167959691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:03 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3812895372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:03 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3005227278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:03 smithi076 ceph-mon[37057]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:03:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:03 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4167959691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:03 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3812895372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:03 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3005227278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:04.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.316+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f179402ce50 con 0x7f1794102f20 2024-02-14T15:03:04.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.318+0000 7f1791ffb640 1 -- 172.21.15.17:0/1223252340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f178c09a340 con 0x7f1794102f20 2024-02-14T15:03:04.319 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:04.319 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:02:45.606961+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:03:04.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17840a20c0 msgr2=0x7f17840a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17840a20c0 0x7f17840a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17940697b0 tx=0x7f1788005db0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102f20 msgr2=0x7f1794100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102f20 0x7f1794100a50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f178c00cb00 tx=0x7f178c00a250 comp rx=0 tx=0).stop 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 shutdown_connections 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f17840a20c0 0x7f17840a4580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1794102f20 0x7f1794100a50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1794102560 0x7f1794100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 --2- 172.21.15.17:0/1223252340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17940ff590 0x7f1794101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 >> 172.21.15.17:0/1223252340 conn(0x7f17940faea0 msgr2=0x7f1794105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 shutdown_connections 2024-02-14T15:03:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.320+0000 7f179ad25640 1 -- 172.21.15.17:0/1223252340 wait complete. 2024-02-14T15:03:04.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-14T15:00:48.619843+0000', 'modified': '2024-02-14T15:01:15.551062+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24502, 'mds_2': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6835/3474801477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 3474801477}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 3474801477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.76:6835/1787917010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 1787917010}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 1787917010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6837/1706654420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 1706654420}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 1706654420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.76:6837/3638854935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3638854935}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3638854935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.76:6839/666966898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 666966898}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 666966898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6839/916171404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 916171404}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 916171404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-14T15:03:04.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-14T15:03:04.334 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-14T15:03:04.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 -- 172.21.15.17:0/3179232577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc102430 msgr2=0x7f86dc1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 --2- 172.21.15.17:0/3179232577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc102430 0x7f86dc1028b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:04.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 -- 172.21.15.17:0/3179232577 shutdown_connections 2024-02-14T15:03:04.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 --2- 172.21.15.17:0/3179232577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc102430 0x7f86dc1028b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 --2- 172.21.15.17:0/3179232577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86dc102df0 0x7f86dc109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 --2- 172.21.15.17:0/3179232577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86dc101240 0x7f86dc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 -- 172.21.15.17:0/3179232577 >> 172.21.15.17:0/3179232577 conn(0x7f86dc0fc7b0 msgr2=0x7f86dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 -- 172.21.15.17:0/3179232577 shutdown_connections 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.437+0000 7f86e1332640 1 -- 172.21.15.17:0/3179232577 wait complete. 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 Processor -- start 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 -- start start 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc101240 0x7f86dc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86dc102430 0x7f86dc197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86dc102df0 0x7f86dc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc19ede0 con 0x7f86dc101240 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc19ef50 con 0x7f86dc102df0 2024-02-14T15:03:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86e1332640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc19f110 con 0x7f86dc102430 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc101240 0x7f86dc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc101240 0x7f86dc1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58078/0 (socket says 172.21.15.17:58078) 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 -- 172.21.15.17:0/1309464485 learned_addr learned my addr 172.21.15.17:0/1309464485 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86d3fff640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86dc102430 0x7f86dc197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86db7fe640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86dc102df0 0x7f86dc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 -- 172.21.15.17:0/1309464485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86dc102430 msgr2=0x7f86dc197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86dc102430 0x7f86dc197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 -- 172.21.15.17:0/1309464485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86dc102df0 msgr2=0x7f86dc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86dc102df0 0x7f86dc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 -- 172.21.15.17:0/1309464485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4009560 con 0x7f86dc101240 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.438+0000 7f86daffd640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc101240 0x7f86dc1972f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00d770 tx=0x7f86cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.439+0000 7f86d8ff9640 1 -- 172.21.15.17:0/1309464485 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc009c30 con 0x7f86dc101240 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.439+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc19f3f0 con 0x7f86dc101240 2024-02-14T15:03:04.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.439+0000 7f86d8ff9640 1 -- 172.21.15.17:0/1309464485 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86cc00de80 con 0x7f86dc101240 2024-02-14T15:03:04.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.439+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc19f920 con 0x7f86dc101240 2024-02-14T15:03:04.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.439+0000 7f86d8ff9640 1 -- 172.21.15.17:0/1309464485 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc010460 con 0x7f86dc101240 2024-02-14T15:03:04.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.441+0000 7f86d8ff9640 1 -- 172.21.15.17:0/1309464485 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f86cc003f00 con 0x7f86dc101240 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.441+0000 7f86e3fd7740 1 Processor -- start 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.441+0000 7f86e3fd7740 1 -- start start 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.441+0000 7f86d8ff9640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f86c80a20c0 0x7f86c80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.441+0000 7f86d8ff9640 1 -- 172.21.15.17:0/1309464485 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f86cc0cdc10 con 0x7f86dc101240 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.441+0000 7f86db7fe640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f86c80a20c0 0x7f86c80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d710880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d715950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d718450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d6ccfd0 con 0x55ab0d710e50 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d67fcb0 con 0x55ab0d715fc0 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d67d1e0 con 0x55ab0d710460 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d715950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d715950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58094/0 (socket says 172.21.15.17:58094) 2024-02-14T15:03:04.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 -- 172.21.15.17:0/3341789768 learned_addr learned my addr 172.21.15.17:0/3341789768 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86daffd640 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d710880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86db7fe640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f86c80a20c0 0x7f86c80a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86d4005fd0 tx=0x7f86d4009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86db7fe640 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d718450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 -- 172.21.15.17:0/3341789768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 msgr2=0x55ab0d710880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d710880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 -- 172.21.15.17:0/3341789768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 msgr2=0x55ab0d718450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d718450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 -- 172.21.15.17:0/3341789768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0d63b100 con 0x55ab0d710e50 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86db7fe640 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d718450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d3fff640 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d715950 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9cb4acb5c28913 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d17fa640 1 -- 172.21.15.17:0/3341789768 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c402cbd0 con 0x55ab0d710e50 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d17fa640 1 -- 172.21.15.17:0/3341789768 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86c4004030 con 0x55ab0d710e50 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.442+0000 7f86d17fa640 1 -- 172.21.15.17:0/3341789768 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40052d0 con 0x55ab0d710e50 2024-02-14T15:03:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3341789768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 msgr2=0x55ab0d715950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d715950 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c4005c00 comp rx=0 tx=0).stop 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3341789768 shutdown_connections 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d715950 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d718450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3341789768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d710880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3341789768 >> 172.21.15.17:0/3341789768 conn(0x55ab0d70b9d0 msgr2=0x55ab0d70bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3341789768 shutdown_connections 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3341789768 wait complete. 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.443+0000 7f86e3fd7740 1 -- start start 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 1 Processor -- start 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d710460 0x55ab0d71a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 0x55ab0d71aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d715fc0 0x55ab0d721b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d6789c0 con 0x55ab0d715fc0 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d61dcd0 con 0x55ab0d710460 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d34c310 con 0x55ab0d710e50 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86e3fd7740 10 client.0 _ll_register_callbacks cb 0x55ab0d389ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d715fc0 0x55ab0d721b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d715fc0 0x55ab0d721b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58108/0 (socket says 172.21.15.17:58108) 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 -- 172.21.15.17:0/3108431049 learned_addr learned my addr 172.21.15.17:0/3108431049 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86d3fff640 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 0x55ab0d71aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86daffd640 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d710460 0x55ab0d71a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 -- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 msgr2=0x55ab0d71aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 0x55ab0d71aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 -- 172.21.15.17:0/3108431049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d710460 msgr2=0x55ab0d71a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d710460 0x55ab0d71a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 -- 172.21.15.17:0/3108431049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc109f80 con 0x55ab0d715fc0 2024-02-14T15:03:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86db7fe640 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d715fc0 0x55ab0d721b10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f86d4012b60 tx=0x7f86d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e4bcdf82c54c23 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.444+0000 7f86c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86c27fc640 1 -- 172.21.15.17:0/3108431049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d401f420 con 0x55ab0d715fc0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86c27fc640 1 -- 172.21.15.17:0/3108431049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f86d401f580 con 0x55ab0d715fc0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab0d3786c0 con 0x55ab0d715fc0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86c27fc640 1 -- 172.21.15.17:0/3108431049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d401f740 con 0x55ab0d715fc0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 10 client.15429 fetch_fsmap learned FSMap version 23 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ab0d677040 con 0x55ab0d715fc0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86c27fc640 1 -- 172.21.15.17:0/3108431049 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ab0d3786c0 con 0x55ab0d715fc0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86c27fc640 1 -- 172.21.15.17:0/3108431049 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86d4025020 con 0x55ab0d715fc0 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 10 client.15429 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 10 client.15429 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 20 client.15429 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d726be0 0x55ab0d725e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 4 client.15429 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab0d3c9270 con 0x55ab0d726be0 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.445+0000 7f86daffd640 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d726be0 0x55ab0d725e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.446+0000 7f86daffd640 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d726be0 0x55ab0d725e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3d2cf1c3aede753 server_cookie=855af77898f086aa in_seq=0 out_seq=0 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.446+0000 7f86c27fc640 10 client.15429 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.446+0000 7f86c27fc640 1 -- 172.21.15.17:0/3108431049 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab0d3c9270 con 0x55ab0d726be0 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.446+0000 7f86c27fc640 10 client.15429 handle_command_reply: tid=0 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.446+0000 7f86e3fd7740 1 client.15429 shutdown 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d715fc0 msgr2=0x55ab0d721b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d715fc0 0x55ab0d721b10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f86d4012b60 tx=0x7f86d4004a40 comp rx=0 tx=0).stop 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 shutdown_connections 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d726be0 0x55ab0d725e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d715fc0 0x55ab0d721b10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86c27fc640 0 client.15429 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 0x55ab0d71aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3108431049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d710460 0x55ab0d71a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 >> 172.21.15.17:0/3108431049 conn(0x55ab0d70b9d0 msgr2=0x55ab0d70ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 shutdown_connections 2024-02-14T15:03:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3108431049 wait complete. 2024-02-14T15:03:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.447+0000 7f86e3fd7740 20 client.15429 trim_cache size 0 max 16384 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.462+0000 7f86e3fd7740 1 Processor -- start 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.462+0000 7f86e3fd7740 1 -- start start 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710460 0x55ab0d70bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 0x55ab0d721a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d7118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d5fe490 con 0x55ab0d710460 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d6716c0 con 0x55ab0d715fc0 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d675620 con 0x55ab0d710e50 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710460 0x55ab0d70bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710460 0x55ab0d70bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58122/0 (socket says 172.21.15.17:58122) 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 -- 172.21.15.17:0/2696912068 learned_addr learned my addr 172.21.15.17:0/2696912068 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 -- 172.21.15.17:0/2696912068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 msgr2=0x55ab0d721a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 --2- 172.21.15.17:0/2696912068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 0x55ab0d721a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 -- 172.21.15.17:0/2696912068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 msgr2=0x55ab0d7118f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 --2- 172.21.15.17:0/2696912068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d7118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 -- 172.21.15.17:0/2696912068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0d613a90 con 0x55ab0d710460 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86daffd640 1 --2- 172.21.15.17:0/2696912068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710460 0x55ab0d70bdb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0cb730 tx=0x7f86cc0cb290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b621f9d808ae8873 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86d0ff9640 1 -- 172.21.15.17:0/2696912068 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc014070 con 0x55ab0d710460 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86d0ff9640 1 -- 172.21.15.17:0/2696912068 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86cc0cb4d0 con 0x55ab0d710460 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.463+0000 7f86d0ff9640 1 -- 172.21.15.17:0/2696912068 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc003710 con 0x55ab0d710460 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 -- 172.21.15.17:0/2696912068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710460 msgr2=0x55ab0d70bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/2696912068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710460 0x55ab0d70bdb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0cb730 tx=0x7f86cc0cb290 comp rx=0 tx=0).stop 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 -- 172.21.15.17:0/2696912068 shutdown_connections 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/2696912068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d7118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/2696912068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710e50 0x55ab0d721a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/2696912068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710460 0x55ab0d70bdb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 -- 172.21.15.17:0/2696912068 >> 172.21.15.17:0/2696912068 conn(0x55ab0d70b9d0 msgr2=0x55ab0d719130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 -- 172.21.15.17:0/2696912068 shutdown_connections 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 -- 172.21.15.17:0/2696912068 wait complete. 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 -- start start 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.464+0000 7f86e3fd7740 1 Processor -- start 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d71ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d71f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d720080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d7205c0 con 0x55ab0d710e50 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d720730 con 0x55ab0d715fc0 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0d7208f0 con 0x55ab0d710460 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 10 client.0 _ll_register_callbacks cb 0x7f86c4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d71f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d71f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58126/0 (socket says 172.21.15.17:58126) 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 -- 172.21.15.17:0/3209068253 learned_addr learned my addr 172.21.15.17:0/3209068253 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86daffd640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d71ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86db7fe640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d720080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 -- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 msgr2=0x55ab0d71ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d71ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 -- 172.21.15.17:0/3209068253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 msgr2=0x55ab0d720080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d720080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 -- 172.21.15.17:0/3209068253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc19f920 con 0x55ab0d710e50 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86daffd640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d71ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86d3fff640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d71f160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c402cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c0b53b508ba25 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86c27fc640 1 -- 172.21.15.17:0/3209068253 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4004df0 con 0x55ab0d710e50 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86c27fc640 1 -- 172.21.15.17:0/3209068253 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f86c4004f50 con 0x55ab0d710e50 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.465+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c4004030 con 0x55ab0d710e50 2024-02-14T15:03:04.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86c27fc640 1 -- 172.21.15.17:0/3209068253 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c4005110 con 0x55ab0d710e50 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 10 client.15435 fetch_fsmap learned FSMap version 23 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ab0d639960 con 0x55ab0d710e50 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86c27fc640 1 -- 172.21.15.17:0/3209068253 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86c4004030 con 0x55ab0d710e50 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86c27fc640 1 -- 172.21.15.17:0/3209068253 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86c403c050 con 0x55ab0d710e50 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 10 client.15435 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 10 client.15435 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 20 client.15435 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d781f80 0x55ab0d77e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 4 client.15435 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ab0d3b5610 con 0x55ab0d781f80 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.466+0000 7f86daffd640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d781f80 0x55ab0d77e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86daffd640 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d781f80 0x55ab0d77e7f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c8f7162a87edbc1 server_cookie=b6caadd68d8dcc31 in_seq=0 out_seq=0 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86c27fc640 10 client.15435 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:04.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86c27fc640 1 -- 172.21.15.17:0/3209068253 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55ab0d3b5610 con 0x55ab0d781f80 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86c27fc640 10 client.15435 handle_command_reply: tid=0 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 client.15435 shutdown 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 msgr2=0x55ab0d71f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d71f160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c402cd40 comp rx=0 tx=0).stop 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 shutdown_connections 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55ab0d781f80 0x55ab0d77e7f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab0d710e50 0x55ab0d71f160 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab0d715fc0 0x55ab0d720080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 --2- 172.21.15.17:0/3209068253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab0d710460 0x55ab0d71ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86c27fc640 0 client.15435 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 >> 172.21.15.17:0/3209068253 conn(0x55ab0d70b9d0 msgr2=0x55ab0d7118f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 shutdown_connections 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 1 -- 172.21.15.17:0/3209068253 wait complete. 2024-02-14T15:03:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.467+0000 7f86e3fd7740 20 client.15435 trim_cache size 0 max 16384 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 2, 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24481, 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 109.887693697, 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 23, 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-02-14T15:03:04.475 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 155.671751191 2024-02-14T15:03:04.476 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:04.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f86c80a20c0 msgr2=0x7f86c80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f86c80a20c0 0x7f86c80a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86d4005fd0 tx=0x7f86d4009600 comp rx=0 tx=0).stop 2024-02-14T15:03:04.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc101240 msgr2=0x7f86dc1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc101240 0x7f86dc1972f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00d770 tx=0x7f86cc00dc30 comp rx=0 tx=0).stop 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 shutdown_connections 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f86c80a20c0 0x7f86c80a4580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86dc102df0 0x7f86dc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86dc102430 0x7f86dc197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 --2- 172.21.15.17:0/1309464485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f86dc101240 0x7f86dc1972f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 >> 172.21.15.17:0/1309464485 conn(0x7f86dc0fc7b0 msgr2=0x7f86dc1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 shutdown_connections 2024-02-14T15:03:04.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.468+0000 7f86e1332640 1 -- 172.21.15.17:0/1309464485 wait complete. 2024-02-14T15:03:04.479 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 -- 172.21.15.17:0/2933752339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 msgr2=0x7f8abc108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/2933752339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc108590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009ca0 tx=0x7f8ab402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 -- 172.21.15.17:0/2933752339 shutdown_connections 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/2933752339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc108590 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/2933752339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8abc101340 0x7f8abc1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/2933752339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8abc100150 0x7f8abc100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 -- 172.21.15.17:0/2933752339 >> 172.21.15.17:0/2933752339 conn(0x7f8abc0fb6c0 msgr2=0x7f8abc0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 -- 172.21.15.17:0/2933752339 shutdown_connections 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 -- 172.21.15.17:0/2933752339 wait complete. 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.582+0000 7f8ac28f9640 1 Processor -- start 2024-02-14T15:03:04.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac28f9640 1 -- start start 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac28f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8abc100150 0x7f8abc10aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac28f9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8abc101340 0x7f8abc109050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac28f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc109590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac28f9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc109cf0 con 0x7f8abc101d00 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac28f9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc109e60 con 0x7f8abc101340 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac28f9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1a1320 con 0x7f8abc100150 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58142/0 (socket says 172.21.15.17:58142) 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 -- 172.21.15.17:0/1754269543 learned_addr learned my addr 172.21.15.17:0/1754269543 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8abc101340 0x7f8abc109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8abbfff640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8abc100150 0x7f8abc10aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 -- 172.21.15.17:0/1754269543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8abc100150 msgr2=0x7f8abc10aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8abc100150 0x7f8abc10aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 -- 172.21.15.17:0/1754269543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8abc101340 msgr2=0x7f8abc109050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8abc101340 0x7f8abc109050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 -- 172.21.15.17:0/1754269543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009900 con 0x7f8abc101d00 2024-02-14T15:03:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.583+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc109590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4033000 tx=0x7f8ab4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.584+0000 7f8ab97fa640 1 -- 172.21.15.17:0/1754269543 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40329b0 con 0x7f8abc101d00 2024-02-14T15:03:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.584+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc1a1570 con 0x7f8abc101d00 2024-02-14T15:03:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.584+0000 7f8ab97fa640 1 -- 172.21.15.17:0/1754269543 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ab4003e50 con 0x7f8abc101d00 2024-02-14T15:03:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.584+0000 7f8ab97fa640 1 -- 172.21.15.17:0/1754269543 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4040450 con 0x7f8abc101d00 2024-02-14T15:03:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.584+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc1a1a20 con 0x7f8abc101d00 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ab97fa640 1 -- 172.21.15.17:0/1754269543 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8ab400a040 con 0x7f8abc101d00 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 Processor -- start 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 -- start start 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ab97fa640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8aa40a20c0 0x7f8aa40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4460 0x55be880d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4e50 0x55be880d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be88090fd0 con 0x55be880d4460 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be88043cb0 con 0x55be880d4e50 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.586+0000 7f8ac559d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880411e0 con 0x55be880d9fc0 2024-02-14T15:03:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8aa40a20c0 0x7f8aa40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4460 0x55be880d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4460 0x55be880d4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58150/0 (socket says 172.21.15.17:58150) 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 -- 172.21.15.17:0/3597253685 learned_addr learned my addr 172.21.15.17:0/3597253685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8ab97fa640 1 -- 172.21.15.17:0/1754269543 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f8ab40f0e30 con 0x7f8abc101d00 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4e50 0x55be880d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 -- 172.21.15.17:0/3597253685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 msgr2=0x55be880dc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880dc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 -- 172.21.15.17:0/3597253685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4e50 msgr2=0x55be880d9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4e50 0x55be880d9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 -- 172.21.15.17:0/3597253685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be87fff100 con 0x55be880d4460 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8aa40a20c0 0x7f8aa40a4580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8abc10a780 tx=0x7f8ab0013880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8abbfff640 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4460 0x55be880d4880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009c20 tx=0x7f8aac0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27aefd8a048760f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8aa27fc640 1 -- 172.21.15.17:0/3597253685 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac009720 con 0x55be880d4460 2024-02-14T15:03:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8aa27fc640 1 -- 172.21.15.17:0/3597253685 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8aac012930 con 0x55be880d4460 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.587+0000 7f8aa27fc640 1 -- 172.21.15.17:0/3597253685 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00f9e0 con 0x55be880d4460 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 -- 172.21.15.17:0/3597253685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4460 msgr2=0x55be880d4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4460 0x55be880d4880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009c20 tx=0x7f8aac0117b0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 -- 172.21.15.17:0/3597253685 shutdown_connections 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4e50 0x55be880d9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3597253685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4460 0x55be880d4880 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 -- 172.21.15.17:0/3597253685 >> 172.21.15.17:0/3597253685 conn(0x55be880cf9d0 msgr2=0x55be880cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 -- 172.21.15.17:0/3597253685 shutdown_connections 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 -- 172.21.15.17:0/3597253685 wait complete. 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 -- start start 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.588+0000 7f8ac559d740 1 Processor -- start 2024-02-14T15:03:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880de610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4e50 0x55be880deb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d9fc0 0x55be880e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8803c9c0 con 0x55be880d9fc0 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be87fe1cd0 con 0x55be880d4460 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be87d10310 con 0x55be880d4e50 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 10 client.0 _ll_register_callbacks cb 0x55be87d4dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d9fc0 0x55be880e5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d9fc0 0x55be880e5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58158/0 (socket says 172.21.15.17:58158) 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 -- 172.21.15.17:0/3397341493 learned_addr learned my addr 172.21.15.17:0/3397341493 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4e50 0x55be880deb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8abbfff640 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880de610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 -- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4e50 msgr2=0x55be880deb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4e50 0x55be880deb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 -- 172.21.15.17:0/3397341493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 msgr2=0x55be880de610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880de610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 -- 172.21.15.17:0/3397341493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac009880 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d9fc0 0x55be880e5b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40ee580 tx=0x7f8ab402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae008755116a5d3e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8a9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/3397341493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40a14e0 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/3397341493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f8ab40bdeb0 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.589+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be87d3c6c0 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/3397341493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab40bd7b0 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 10 client.15447 fetch_fsmap learned FSMap version 23 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55be8803b040 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/3397341493 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55be87d3c6c0 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/3397341493 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8ab40a16c0 con 0x55be880d9fc0 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 10 client.15447 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 10 client.15447 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 20 client.15447 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be880ea9b0 0x55be880eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 4 client.15447 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be87d8d270 con 0x55be880ea9b0 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.590+0000 7f8abbfff640 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be880ea9b0 0x55be880eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8a9b7fe640 10 client.15447 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8abbfff640 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be880ea9b0 0x55be880eee30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8da7eb05dfd05e95 server_cookie=d5a7b1d2927b89fe in_seq=0 out_seq=0 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/3397341493 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be87d8d270 con 0x55be880ea9b0 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8a9b7fe640 10 client.15447 handle_command_reply: tid=0 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 client.15447 shutdown 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d9fc0 msgr2=0x55be880e5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d9fc0 0x55be880e5b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40ee580 tx=0x7f8ab402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 shutdown_connections 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be880ea9b0 0x55be880eee30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d9fc0 0x55be880e5b90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4e50 0x55be880deb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8a9b7fe640 0 client.15447 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 --2- 172.21.15.17:0/3397341493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880de610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.591+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 >> 172.21.15.17:0/3397341493 conn(0x55be880cf9d0 msgr2=0x55be880cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.592+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 shutdown_connections 2024-02-14T15:03:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.592+0000 7f8ac559d740 1 -- 172.21.15.17:0/3397341493 wait complete. 2024-02-14T15:03:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.592+0000 7f8ac559d740 20 client.15447 trim_cache size 0 max 16384 2024-02-14T15:03:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 Processor -- start 2024-02-14T15:03:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 -- start start 2024-02-14T15:03:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880dc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880d6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be87fc2490 con 0x55be880d4e50 2024-02-14T15:03:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880d0970 con 0x55be880d4460 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac559d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880dde60 con 0x55be880d9fc0 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58160/0 (socket says 172.21.15.17:58160) 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2963479426 learned_addr learned my addr 172.21.15.17:0/2963479426 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abbfff640 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880dc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8ac0e96640 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2963479426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 msgr2=0x55be880d6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880d6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2963479426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 msgr2=0x55be880dc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880dc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2963479426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be87fd7a90 con 0x55be880d4e50 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abbfff640 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880dc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.608+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e5b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0004030 tx=0x7f8ab0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743d0a9ad6142148 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8aa1ffb640 1 -- 172.21.15.17:0/2963479426 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0005220 con 0x55be880d4e50 2024-02-14T15:03:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8aa1ffb640 1 -- 172.21.15.17:0/2963479426 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ab0025070 con 0x55be880d4e50 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8aa1ffb640 1 -- 172.21.15.17:0/2963479426 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0003af0 con 0x55be880d4e50 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 -- 172.21.15.17:0/2963479426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 msgr2=0x55be880e5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e5b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0004030 tx=0x7f8ab0005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 -- 172.21.15.17:0/2963479426 shutdown_connections 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e5b50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d9fc0 0x55be880d6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2963479426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d4460 0x55be880dc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 -- 172.21.15.17:0/2963479426 >> 172.21.15.17:0/2963479426 conn(0x55be880cf9d0 msgr2=0x55be880cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 -- 172.21.15.17:0/2963479426 shutdown_connections 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.609+0000 7f8ac559d740 1 -- 172.21.15.17:0/2963479426 wait complete. 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 -- start start 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 Processor -- start 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4460 0x55be880e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d9fc0 0x55be88140af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880e9f20 con 0x55be880d4e50 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880e5d30 con 0x55be880d9fc0 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880e5ef0 con 0x55be880d4460 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8ac559d740 10 client.0 _ll_register_callbacks cb 0x7f8aac009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58176/0 (socket says 172.21.15.17:58176) 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2650566401 learned_addr learned my addr 172.21.15.17:0/2650566401 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abbfff640 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4460 0x55be880e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4460 msgr2=0x55be880e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4460 0x55be880e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2650566401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d9fc0 msgr2=0x55be88140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d9fc0 0x55be88140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 -- 172.21.15.17:0/2650566401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab00121c0 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abbfff640 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4460 0x55be880e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8abb7fe640 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e2d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0013d80 tx=0x7f8ab0005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddefde7be637d5b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.610+0000 7f8a9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/2650566401 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0003bc0 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/2650566401 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f8ab0003d20 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be880ea060 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/2650566401 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab000f540 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 10 client.15453 fetch_fsmap learned FSMap version 23 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55be87ffd960 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/2650566401 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55be880ea060 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/2650566401 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8ab002a860 con 0x55be880d4e50 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 10 client.15453 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 10 client.15453 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 20 client.15453 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:04.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be88146120 0x55be881425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 4 client.15453 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be880eb380 con 0x55be88146120 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.611+0000 7f8abbfff640 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be88146120 0x55be881425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8abbfff640 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be88146120 0x55be881425a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=241452b054309f2e server_cookie=991f654d53ec5983 in_seq=0 out_seq=0 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8a9b7fe640 10 client.15453 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8a9b7fe640 1 -- 172.21.15.17:0/2650566401 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55be880eb380 con 0x55be88146120 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8a9b7fe640 10 client.15453 handle_command_reply: tid=0 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 client.15453 shutdown 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 msgr2=0x55be880e2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e2d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0013d80 tx=0x7f8ab0005580 comp rx=0 tx=0).stop 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 shutdown_connections 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55be88146120 0x55be881425a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be880d4e50 0x55be880e2d10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be880d9fc0 0x55be88140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 --2- 172.21.15.17:0/2650566401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be880d4460 0x55be880e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8a9b7fe640 0 client.15453 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 >> 172.21.15.17:0/2650566401 conn(0x55be880cf9d0 msgr2=0x55be880de530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 shutdown_connections 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 1 -- 172.21.15.17:0/2650566401 wait complete. 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.612+0000 7f8ac559d740 20 client.15453 trim_cache size 0 max 16384 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24502, 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-02-14T15:03:04.620 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-02-14T15:03:04.621 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 111.03915394400001, 2024-02-14T15:03:04.621 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 23, 2024-02-14T15:03:04.621 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-02-14T15:03:04.621 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-02-14T15:03:04.621 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 150.14970994800001 2024-02-14T15:03:04.621 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8aa40a20c0 msgr2=0x7f8aa40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8aa40a20c0 0x7f8aa40a4580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8abc10a780 tx=0x7f8ab0013880 comp rx=0 tx=0).stop 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 msgr2=0x7f8abc109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc109590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4033000 tx=0x7f8ab4009360 comp rx=0 tx=0).stop 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 shutdown_connections 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8aa40a20c0 0x7f8aa40a4580 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8abc101d00 0x7f8abc109590 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8abc101340 0x7f8abc109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 --2- 172.21.15.17:0/1754269543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8abc100150 0x7f8abc10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 >> 172.21.15.17:0/1754269543 conn(0x7f8abc0fb6c0 msgr2=0x7f8abc104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 shutdown_connections 2024-02-14T15:03:04.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.613+0000 7f8ac28f9640 1 -- 172.21.15.17:0/1754269543 wait complete. 2024-02-14T15:03:04.624 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-14T15:03:04.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 -- 172.21.15.17:0/3479559628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 msgr2=0x7eff8c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 --2- 172.21.15.17:0/3479559628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c1028a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff80009900 tx=0x7eff8002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 -- 172.21.15.17:0/3479559628 shutdown_connections 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 --2- 172.21.15.17:0/3479559628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c1028a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 --2- 172.21.15.17:0/3479559628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c102de0 0x7eff8c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 --2- 172.21.15.17:0/3479559628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff8c101230 0x7eff8c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 -- 172.21.15.17:0/3479559628 >> 172.21.15.17:0/3479559628 conn(0x7eff8c0fc780 msgr2=0x7eff8c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 -- 172.21.15.17:0/3479559628 shutdown_connections 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 -- 172.21.15.17:0/3479559628 wait complete. 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 Processor -- start 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 -- start start 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.728+0000 7eff91691640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff8c101230 0x7eff8c074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff91691640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff91691640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c102de0 0x7eff8c073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff91691640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0742c0 con 0x7eff8c102420 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff91691640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c074430 con 0x7eff8c102de0 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff91691640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c1a1350 con 0x7eff8c101230 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58204/0 (socket says 172.21.15.17:58204) 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 -- 172.21.15.17:0/1695113377 learned_addr learned my addr 172.21.15.17:0/1695113377 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c102de0 0x7eff8c073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8affd640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff8c101230 0x7eff8c074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 -- 172.21.15.17:0/1695113377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff8c101230 msgr2=0x7eff8c074fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff8c101230 0x7eff8c074fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 -- 172.21.15.17:0/1695113377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c102de0 msgr2=0x7eff8c073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c102de0 0x7eff8c073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 -- 172.21.15.17:0/1695113377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80009560 con 0x7eff8c102420 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c102de0 0x7eff8c073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c073620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff80036040 tx=0x7eff800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff73fff640 1 -- 172.21.15.17:0/1695113377 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800044e0 con 0x7eff8c102420 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c1a15a0 con 0x7eff8c102420 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff73fff640 1 -- 172.21.15.17:0/1695113377 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff800054e0 con 0x7eff8c102420 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff73fff640 1 -- 172.21.15.17:0/1695113377 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80003ec0 con 0x7eff8c102420 2024-02-14T15:03:04.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.729+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c1a1ab0 con 0x7eff8c102420 2024-02-14T15:03:04.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.731+0000 7eff73fff640 1 -- 172.21.15.17:0/1695113377 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7eff800035f0 con 0x7eff8c102420 2024-02-14T15:03:04.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 Processor -- start 2024-02-14T15:03:04.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 -- start start 2024-02-14T15:03:04.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff73fff640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7eff740a20c0 0x7eff740a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf176880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf17b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf17e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf132fd0 con 0x55dbbf176460 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf0e5cb0 con 0x55dbbf176e50 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff94259740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf0e31e0 con 0x55dbbf17bfc0 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7eff740a20c0 0x7eff740a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff73fff640 1 -- 172.21.15.17:0/1695113377 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7eff800f0440 con 0x7eff8c102420 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf176880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf176880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58220/0 (socket says 172.21.15.17:58220) 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 -- 172.21.15.17:0/3139503239 learned_addr learned my addr 172.21.15.17:0/3139503239 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf17b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf17e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 -- 172.21.15.17:0/3139503239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 msgr2=0x55dbbf17e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf17e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 -- 172.21.15.17:0/3139503239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 msgr2=0x55dbbf17b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf17b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8affd640 1 -- 172.21.15.17:0/3139503239 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbf0a1100 con 0x55dbbf176460 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.732+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7eff740a20c0 0x7eff740a4580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff84005fd0 tx=0x7eff84014b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff8affd640 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf176880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bc00 tx=0x7eff7c00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949f118bdbf81a76 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff717fa640 1 -- 172.21.15.17:0/3139503239 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00fd90 con 0x55dbbf176460 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff717fa640 1 -- 172.21.15.17:0/3139503239 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff7c01a940 con 0x55dbbf176460 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff717fa640 1 -- 172.21.15.17:0/3139503239 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011940 con 0x55dbbf176460 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff94259740 1 -- 172.21.15.17:0/3139503239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 msgr2=0x55dbbf176880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff94259740 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf176880 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bc00 tx=0x7eff7c00f360 comp rx=0 tx=0).stop 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff94259740 1 -- 172.21.15.17:0/3139503239 shutdown_connections 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff94259740 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf17e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff94259740 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf17b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff94259740 1 --2- 172.21.15.17:0/3139503239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf176880 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.733+0000 7eff94259740 1 -- 172.21.15.17:0/3139503239 >> 172.21.15.17:0/3139503239 conn(0x55dbbf1719d0 msgr2=0x55dbbf171dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 -- 172.21.15.17:0/3139503239 shutdown_connections 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 -- 172.21.15.17:0/3139503239 wait complete. 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 -- start start 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 Processor -- start 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176460 0x55dbbf1805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf180b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf17bfc0 0x55dbbf187b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf0de9c0 con 0x55dbbf17bfc0 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf083cd0 con 0x55dbbf176e50 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbedb2310 con 0x55dbbf176460 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff94259740 10 client.0 _ll_register_callbacks cb 0x55dbbedefac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf17bfc0 0x55dbbf187b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf17bfc0 0x55dbbf187b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58236/0 (socket says 172.21.15.17:58236) 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8b7fe640 1 -- 172.21.15.17:0/317200840 learned_addr learned my addr 172.21.15.17:0/317200840 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8affd640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176460 0x55dbbf1805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8b7fe640 1 -- 172.21.15.17:0/317200840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176460 msgr2=0x55dbbf1805e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf180b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176460 0x55dbbf1805e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.734+0000 7eff8b7fe640 1 -- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 msgr2=0x55dbbf180b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf180b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff8b7fe640 1 -- 172.21.15.17:0/317200840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c00b8e0 con 0x55dbbf17bfc0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf180b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf17bfc0 0x55dbbf187b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff84012c20 tx=0x7eff84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3241269dddca7a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff6eff5640 1 -- 172.21.15.17:0/317200840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400fbf0 con 0x55dbbf17bfc0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff6eff5640 1 -- 172.21.15.17:0/317200840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7eff8400fd50 con 0x55dbbf17bfc0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbbedde6c0 con 0x55dbbf17bfc0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff6eff5640 1 -- 172.21.15.17:0/317200840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff84021420 con 0x55dbbf17bfc0 2024-02-14T15:03:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 10 client.15465 fetch_fsmap learned FSMap version 23 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dbbf0dd040 con 0x55dbbf17bfc0 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff6eff5640 1 -- 172.21.15.17:0/317200840 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dbbedde6c0 con 0x55dbbf17bfc0 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff6eff5640 1 -- 172.21.15.17:0/317200840 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7eff84029690 con 0x55dbbf17bfc0 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 10 client.15465 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 10 client.15465 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 20 client.15465 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf18cc40 0x55dbbf18be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 4 client.15465 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.735+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbbee2f270 con 0x55dbbf18cc40 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.736+0000 7eff8affd640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf18cc40 0x55dbbf18be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.736+0000 7eff6eff5640 10 client.15465 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.736+0000 7eff8affd640 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf18cc40 0x55dbbf18be50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb3c7d8cb3e3c02 server_cookie=ddd471add5985a54 in_seq=0 out_seq=0 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff6eff5640 1 -- 172.21.15.17:0/317200840 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbbee2f270 con 0x55dbbf18cc40 2024-02-14T15:03:04.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff6eff5640 10 client.15465 handle_command_reply: tid=0 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 client.15465 shutdown 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf17bfc0 msgr2=0x55dbbf187b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf17bfc0 0x55dbbf187b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7eff84012c20 tx=0x7eff84004ab0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 shutdown_connections 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf18cc40 0x55dbbf18be50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf17bfc0 0x55dbbf187b60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176e50 0x55dbbf180b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 --2- 172.21.15.17:0/317200840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176460 0x55dbbf1805e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 >> 172.21.15.17:0/317200840 conn(0x55dbbf1719d0 msgr2=0x55dbbf170c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff6eff5640 0 client.15465 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 shutdown_connections 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 1 -- 172.21.15.17:0/317200840 wait complete. 2024-02-14T15:03:04.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.737+0000 7eff94259740 20 client.15465 trim_cache size 0 max 16384 2024-02-14T15:03:04.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.752+0000 7eff94259740 1 Processor -- start 2024-02-14T15:03:04.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff94259740 1 -- start start 2024-02-14T15:03:04.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff94259740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176460 0x55dbbf17e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff94259740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176e50 0x55dbbf187a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff94259740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff94259740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf064490 con 0x55dbbf176e50 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff94259740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf18aa50 con 0x55dbbf176460 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff94259740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbee2f270 con 0x55dbbf17bfc0 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176e50 0x55dbbf187a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176e50 0x55dbbf187a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58248/0 (socket says 172.21.15.17:58248) 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 -- 172.21.15.17:0/328843876 learned_addr learned my addr 172.21.15.17:0/328843876 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 -- 172.21.15.17:0/328843876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 msgr2=0x55dbbf1703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8affd640 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176460 0x55dbbf17e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 -- 172.21.15.17:0/328843876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176460 msgr2=0x55dbbf17e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176460 0x55dbbf17e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 -- 172.21.15.17:0/328843876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbf079a90 con 0x55dbbf176e50 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8affd640 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176460 0x55dbbf17e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176e50 0x55dbbf187a80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7eff80006010 tx=0x7eff800edac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab11a6ff7bcb73f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff70ff9640 1 -- 172.21.15.17:0/328843876 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800bce20 con 0x55dbbf176e50 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff70ff9640 1 -- 172.21.15.17:0/328843876 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff800c0f70 con 0x55dbbf176e50 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.753+0000 7eff70ff9640 1 -- 172.21.15.17:0/328843876 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800b9490 con 0x55dbbf176e50 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 -- 172.21.15.17:0/328843876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176e50 msgr2=0x55dbbf187a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176e50 0x55dbbf187a80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7eff80006010 tx=0x7eff800edac0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 -- 172.21.15.17:0/328843876 shutdown_connections 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176e50 0x55dbbf187a80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 --2- 172.21.15.17:0/328843876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf176460 0x55dbbf17e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 -- 172.21.15.17:0/328843876 >> 172.21.15.17:0/328843876 conn(0x55dbbf1719d0 msgr2=0x55dbbf178310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 -- 172.21.15.17:0/328843876 shutdown_connections 2024-02-14T15:03:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 -- 172.21.15.17:0/328843876 wait complete. 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 -- start start 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.754+0000 7eff94259740 1 Processor -- start 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf184d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176e50 0x55dbbf185250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1e2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf185970 con 0x55dbbf176460 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf1e2db0 con 0x55dbbf17bfc0 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbf1e2f70 con 0x55dbbf176e50 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 10 client.0 _ll_register_callbacks cb 0x7eff7c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf184d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf184d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58264/0 (socket says 172.21.15.17:58264) 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 -- 172.21.15.17:0/3775551622 learned_addr learned my addr 172.21.15.17:0/3775551622 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176e50 0x55dbbf185250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 -- 172.21.15.17:0/3775551622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176e50 msgr2=0x55dbbf185250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176e50 0x55dbbf185250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 -- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf17bfc0 msgr2=0x55dbbf1e2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1e2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 -- 172.21.15.17:0/3775551622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c1a1ab0 con 0x55dbbf176460 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176e50 0x55dbbf185250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8b7fe640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff8affd640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf184d10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bbd0 tx=0x7eff7c00ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1206bd6ea8e2a195 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff6eff5640 1 -- 172.21.15.17:0/3775551622 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c020070 con 0x55dbbf176460 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff6eff5640 1 -- 172.21.15.17:0/3775551622 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7eff7c01aab0 con 0x55dbbf176460 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbbf17f480 con 0x55dbbf176460 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff6eff5640 1 -- 172.21.15.17:0/3775551622 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff7c01ac70 con 0x55dbbf176460 2024-02-14T15:03:04.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 10 client.15471 fetch_fsmap learned FSMap version 23 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.755+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dbbf09f960 con 0x55dbbf176460 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff6eff5640 1 -- 172.21.15.17:0/3775551622 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dbbf17f480 con 0x55dbbf176460 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff6eff5640 1 -- 172.21.15.17:0/3775551622 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7eff7c005040 con 0x55dbbf176460 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff94259740 10 client.15471 fetch_fsmap finished waiting for FSMap version 23 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff94259740 10 client.15471 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff94259740 20 client.15471 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff94259740 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf1e8270 0x55dbbf1e4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff94259740 4 client.15471 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dbbf18d610 con 0x55dbbf1e8270 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf1e8270 0x55dbbf1e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff6eff5640 10 client.15471 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.756+0000 7eff8a7fc640 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf1e8270 0x55dbbf1e4610 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993a798db71d35c4 server_cookie=540b4b4812caf79e in_seq=0 out_seq=0 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff6eff5640 1 -- 172.21.15.17:0/3775551622 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55dbbf18d610 con 0x55dbbf1e8270 2024-02-14T15:03:04.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff6eff5640 10 client.15471 handle_command_reply: tid=0 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 client.15471 shutdown 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 msgr2=0x55dbbf184d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf184d10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bbd0 tx=0x7eff7c00ff40 comp rx=0 tx=0).stop 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 shutdown_connections 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dbbf1e8270 0x55dbbf1e4610 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbbf17bfc0 0x55dbbf1e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbbf176e50 0x55dbbf185250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff6eff5640 0 client.15471 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 --2- 172.21.15.17:0/3775551622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbf176460 0x55dbbf184d10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 >> 172.21.15.17:0/3775551622 conn(0x55dbbf1719d0 msgr2=0x55dbbf174c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 shutdown_connections 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 1 -- 172.21.15.17:0/3775551622 wait complete. 2024-02-14T15:03:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff94259740 20 client.15471 trim_cache size 0 max 16384 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "867ff6ea-cb48-11ee-95b9-87774f69a715", 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24509, 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 135.49259964500001, 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 23, 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 93, 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 89, 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 141.60133433199999 2024-02-14T15:03:04.766 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7eff740a20c0 msgr2=0x7eff740a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7eff740a20c0 0x7eff740a4580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff84005fd0 tx=0x7eff84014b70 comp rx=0 tx=0).stop 2024-02-14T15:03:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 msgr2=0x7eff8c073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c073620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff80036040 tx=0x7eff800042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 shutdown_connections 2024-02-14T15:03:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7eff740a20c0 0x7eff740a4580 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff8c102420 0x7eff8c073620 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff8c102de0 0x7eff8c073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 --2- 172.21.15.17:0/1695113377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff8c101230 0x7eff8c074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.757+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 >> 172.21.15.17:0/1695113377 conn(0x7eff8c0fc780 msgr2=0x7eff8c105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.758+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 shutdown_connections 2024-02-14T15:03:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:04.758+0000 7eff91691640 1 -- 172.21.15.17:0/1695113377 wait complete. 2024-02-14T15:03:04.769 INFO:teuthology.run_tasks:Running task kclient... 2024-02-14T15:03:04.783 INFO:tasks.kclient:Mounting kernel clients... 2024-02-14T15:03:04.783 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-14T15:03:04.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:04.783 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-02-14T15:03:04.798 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-02-14T15:03:04.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:04.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-14T15:03:04.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:04.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-02-14T15:03:04.876 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1441647307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:04.876 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1223252340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:04.936 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-02-14T15:03:04.939 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-14T15:03:04.939 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:04.939 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph fs ls 2024-02-14T15:03:05.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1441647307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:05.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1223252340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:05.185 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:03:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1441647307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1223252340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:06.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:05 smithi017 ceph-mon[32549]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:03:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:05 smithi050 ceph-mon[36512]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:03:06.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:05 smithi076 ceph-mon[37057]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:03:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 -- 172.21.15.17:0/1438228103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a806ae20 msgr2=0x7f11a810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 --2- 172.21.15.17:0/1438228103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a806ae20 0x7f11a810ec40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f11a000b0a0 tx=0x7f11a000b3b0 comp rx=0 tx=0).stop 2024-02-14T15:03:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 -- 172.21.15.17:0/1438228103 shutdown_connections 2024-02-14T15:03:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 --2- 172.21.15.17:0/1438228103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a806ae20 0x7f11a810ec40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 --2- 172.21.15.17:0/1438228103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11a8107f00 0x7f11a81082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 --2- 172.21.15.17:0/1438228103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a806a460 0x7f11a806a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 -- 172.21.15.17:0/1438228103 >> 172.21.15.17:0/1438228103 conn(0x7f11a8077a80 msgr2=0x7f11a8077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 -- 172.21.15.17:0/1438228103 shutdown_connections 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.904+0000 7f11aed49700 1 -- 172.21.15.17:0/1438228103 wait complete. 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 Processor -- start 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 -- start start 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11a806a460 0x7f11a8100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a806ae20 0x7f11a8101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a8107f00 0x7f11a8101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81a6720 con 0x7f11a8107f00 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81a6890 con 0x7f11a806a460 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11aed49700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81a6a50 con 0x7f11a806ae20 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11ad2e6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a8107f00 0x7f11a8101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11ad2e6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a8107f00 0x7f11a8101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58272/0 (socket says 172.21.15.17:58272) 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11ad2e6700 1 -- 172.21.15.17:0/2961894980 learned_addr learned my addr 172.21.15.17:0/2961894980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11a7fff700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a806ae20 0x7f11a8101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.905+0000 7f11acae5700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11a806a460 0x7f11a8100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11ad2e6700 1 -- 172.21.15.17:0/2961894980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a806ae20 msgr2=0x7f11a8101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11ad2e6700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a806ae20 0x7f11a8101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11ad2e6700 1 -- 172.21.15.17:0/2961894980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11a806a460 msgr2=0x7f11a8100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11ad2e6700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11a806a460 0x7f11a8100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11ad2e6700 1 -- 172.21.15.17:0/2961894980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0009d00 con 0x7f11a8107f00 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11ad2e6700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a8107f00 0x7f11a8101a50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f11a000b070 tx=0x7f11a0004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11a5ffb700 1 -- 172.21.15.17:0/2961894980 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000e040 con 0x7f11a8107f00 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11a5ffb700 1 -- 172.21.15.17:0/2961894980 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11a0007850 con 0x7f11a8107f00 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11a5ffb700 1 -- 172.21.15.17:0/2961894980 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a001b850 con 0x7f11a8107f00 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a81a6cd0 con 0x7f11a8107f00 2024-02-14T15:03:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.906+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a81a7180 con 0x7f11a8107f00 2024-02-14T15:03:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.907+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a810a2a0 con 0x7f11a8107f00 2024-02-14T15:03:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.908+0000 7f11a5ffb700 1 -- 172.21.15.17:0/2961894980 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f11a001da80 con 0x7f11a8107f00 2024-02-14T15:03:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.909+0000 7f11a5ffb700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f11900a2150 0x7f11900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.909+0000 7f11acae5700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f11900a2150 0x7f11900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.909+0000 7f11a5ffb700 1 -- 172.21.15.17:0/2961894980 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f11a00d0240 con 0x7f11a8107f00 2024-02-14T15:03:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.909+0000 7f11acae5700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f11900a2150 0x7f11900a4610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f11a8103220 tx=0x7f119800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:06.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:06.913+0000 7f11a5ffb700 1 -- 172.21.15.17:0/2961894980 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11a0098f50 con 0x7f11a8107f00 2024-02-14T15:03:07.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.160+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f11a81034d0 con 0x7f11a8107f00 2024-02-14T15:03:07.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.160+0000 7f11a5ffb700 1 -- 172.21.15.17:0/2961894980 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7f11a009c610 con 0x7f11a8107f00 2024-02-14T15:03:07.161 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-14T15:03:07.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f11900a2150 msgr2=0x7f11900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:07.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f11900a2150 0x7f11900a4610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f11a8103220 tx=0x7f119800b410 comp rx=0 tx=0).stop 2024-02-14T15:03:07.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a8107f00 msgr2=0x7f11a8101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:07.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a8107f00 0x7f11a8101a50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f11a000b070 tx=0x7f11a0004e40 comp rx=0 tx=0).stop 2024-02-14T15:03:07.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 shutdown_connections 2024-02-14T15:03:07.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f11900a2150 0x7f11900a4610 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:07.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a8107f00 0x7f11a8101a50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:07.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11a806a460 0x7f11a8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:07.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 --2- 172.21.15.17:0/2961894980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a806ae20 0x7f11a8101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:07.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.162+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 >> 172.21.15.17:0/2961894980 conn(0x7f11a8077a80 msgr2=0x7f11a80ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:07.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.163+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 shutdown_connections 2024-02-14T15:03:07.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:07.163+0000 7f11aed49700 1 -- 172.21.15.17:0/2961894980 wait complete. 2024-02-14T15:03:07.835 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-14T15:03:07.835 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-14T15:03:07.835 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-02-14T15:03:07.835 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-14T15:03:07.835 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:07.836 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-14T15:03:07.836 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-14T15:03:07.836 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-14T15:03:07.836 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-14T15:03:07.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:07.836 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-02-14T15:03:07.853 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-14T15:03:07.853 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-14T15:03:07.853 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-02-14T15:03:07.853 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-02-14T15:03:07.853 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42089sec preferred_lft 42089sec 2024-02-14T15:03:07.854 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-02-14T15:03:07.855 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-02-14T15:03:07.855 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-14T15:03:07.855 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-02-14T15:03:07.855 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-02-14T15:03:07.855 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-14T15:03:07.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:07.855 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:03:07.855 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:03:07.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-02-14T15:03:07.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-02-14T15:03:07.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-02-14T15:03:07.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-14T15:03:07.856 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:03:07.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:07.968 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-14T15:03:07.995 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-02-14T15:03:07.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:07.997 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-02-14T15:03:08.053 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-02-14T15:03:08.054 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-14T15:03:08.054 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-14T15:03:08.054 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-14T15:03:08.054 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-14T15:03:08.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.054 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:03:08.054 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:03:08.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-14T15:03:08.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-14T15:03:08.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-14T15:03:08.054 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:03:08.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:07 smithi017 ceph-mon[32549]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:03:08.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:07 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2961894980' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-14T15:03:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:07 smithi050 ceph-mon[36512]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:03:08.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:07 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2961894980' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-14T15:03:08.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.149 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-02-14T15:03:08.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.204 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-02-14T15:03:08.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:07 smithi076 ceph-mon[37057]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-14T15:03:08.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:07 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2961894980' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-14T15:03:08.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.259 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:03:08.259 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:03:08.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-14T15:03:08.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-14T15:03:08.259 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:03:08.341 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-14T15:03:08.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-14T15:03:08.341 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:03:08.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.466 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-02-14T15:03:08.467 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-02-14T15:03:08.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-02-14T15:03:08.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-02-14T15:03:08.467 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-02-14T15:03:08.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-14T15:03:08.508 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:08.562 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-14T15:03:08.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-14T15:03:08.562 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:08.617 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-14T15:03:08.617 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-14T15:03:08.617 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f 2024-02-14T15:03:08.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.617 DEBUG:teuthology.orchestra.run.smithi017:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-02-14T15:03:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-02-14T15:03:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-02-14T15:03:08.701 INFO:teuthology.orchestra.run.smithi017.stdout:invalid new device string format 2024-02-14T15:03:08.865 INFO:teuthology.orchestra.run.smithi017.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-02-14T15:03:08.865 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-02-14T15:03:08.865 INFO:teuthology.orchestra.run.smithi017.stdout:invalid new device string format 2024-02-14T15:03:08.866 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: resolved to: "172.21.15.17:6789,172.21.15.50:6789,172.21.15.76:6789" 2024-02-14T15:03:08.866 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.17:6789,172.21.15.50:6789,172.21.15.76:6789:/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f 2024-02-14T15:03:08.866 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=867ff6ea-cb48-11ee-95b9-87774f69a715" will pass to kernel 2024-02-14T15:03:08.866 INFO:teuthology.orchestra.run.smithi017.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-14T15:03:08.866 INFO:teuthology.orchestra.run.smithi017.stdout: You just mounted a file system that supports labels which does not 2024-02-14T15:03:08.866 INFO:teuthology.orchestra.run.smithi017.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-14T15:03:08.867 INFO:teuthology.orchestra.run.smithi017.stdout: applications will generate AVC messages and not be allowed access to 2024-02-14T15:03:08.867 INFO:teuthology.orchestra.run.smithi017.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-14T15:03:08.868 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-14T15:03:08.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.868 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:08.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:08.936 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-14T15:03:08.993 INFO:teuthology.orchestra.run.smithi017.stdout:client25037 2024-02-14T15:03:08.993 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:03:08.993 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-14T15:03:09.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:09.049 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-14T15:03:09.105 INFO:teuthology.orchestra.run.smithi017.stdout:client25037 2024-02-14T15:03:09.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:09.106 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-14T15:03:09.162 INFO:teuthology.orchestra.run.smithi017.stdout:client25037 2024-02-14T15:03:09.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:09.162 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715.client25037/status') 2024-02-14T15:03:09.228 INFO:teuthology.orchestra.run.smithi017.stdout:instance: client.25037 (3)172.21.15.17:0/1804619088 2024-02-14T15:03:09.228 INFO:teuthology.orchestra.run.smithi017.stdout:blocklisted: false 2024-02-14T15:03:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:0+1 records in 2024-02-14T15:03:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:0+1 records out 2024-02-14T15:03:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:71 bytes copied, 2.2764e-05 s, 3.1 MB/s 2024-02-14T15:03:09.230 INFO:teuthology.run_tasks:Running task exec... 2024-02-14T15:03:09.242 INFO:teuthology.task.exec:Executing custom commands... 2024-02-14T15:03:09.242 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:03:09.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-02-14T15:03:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 -- 172.21.15.17:0/1862362523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 msgr2=0x7f32c40ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 --2- 172.21.15.17:0/1862362523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c40ffd40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009900 tx=0x7f32c002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 -- 172.21.15.17:0/1862362523 shutdown_connections 2024-02-14T15:03:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 --2- 172.21.15.17:0/1862362523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c41014d0 0x7f32c4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 --2- 172.21.15.17:0/1862362523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c4100b10 0x7f32c4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 --2- 172.21.15.17:0/1862362523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c40ffd40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 -- 172.21.15.17:0/1862362523 >> 172.21.15.17:0/1862362523 conn(0x7f32c40fae90 msgr2=0x7f32c40fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 -- 172.21.15.17:0/1862362523 shutdown_connections 2024-02-14T15:03:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.373+0000 7f32cb684640 1 -- 172.21.15.17:0/1862362523 wait complete. 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 Processor -- start 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 -- start start 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c410b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c4100b10 0x7f32c41096b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c41014d0 0x7f32c4109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c410a350 con 0x7f32c40ff920 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c410a4c0 con 0x7f32c4100b10 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32cb684640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c41a13a0 con 0x7f32c41014d0 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c410b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c410b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58290/0 (socket says 172.21.15.17:58290) 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 -- 172.21.15.17:0/1983655204 learned_addr learned my addr 172.21.15.17:0/1983655204 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9c21640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c41014d0 0x7f32c4109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 -- 172.21.15.17:0/1983655204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c41014d0 msgr2=0x7f32c4109bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c41014d0 0x7f32c4109bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c8c1f640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c4100b10 0x7f32c41096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 -- 172.21.15.17:0/1983655204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c4100b10 msgr2=0x7f32c41096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c4100b10 0x7f32c41096b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.374+0000 7f32c9420640 1 -- 172.21.15.17:0/1983655204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c0009560 con 0x7f32c40ff920 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.375+0000 7f32c9420640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c410b070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32c0000c00 tx=0x7f32c0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.375+0000 7f32ba7fc640 1 -- 172.21.15.17:0/1983655204 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00043d0 con 0x7f32c40ff920 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.375+0000 7f32ba7fc640 1 -- 172.21.15.17:0/1983655204 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32c00052d0 con 0x7f32c40ff920 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.375+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c41a15f0 con 0x7f32c40ff920 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.375+0000 7f32ba7fc640 1 -- 172.21.15.17:0/1983655204 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0003d70 con 0x7f32c40ff920 2024-02-14T15:03:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.375+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c41a1b50 con 0x7f32c40ff920 2024-02-14T15:03:09.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.376+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c4106c40 con 0x7f32c40ff920 2024-02-14T15:03:09.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.377+0000 7f32ba7fc640 1 -- 172.21.15.17:0/1983655204 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f32c0005440 con 0x7f32c40ff920 2024-02-14T15:03:09.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.377+0000 7f32ba7fc640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f32b00a2130 0x7f32b00a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.377+0000 7f32c8c1f640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f32b00a2130 0x7f32b00a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:09.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.377+0000 7f32ba7fc640 1 -- 172.21.15.17:0/1983655204 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f32c00f0240 con 0x7f32c40ff920 2024-02-14T15:03:09.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.378+0000 7f32c8c1f640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f32b00a2130 0x7f32b00a45f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f32c410ade0 tx=0x7f32b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.382+0000 7f32ba7fc640 1 -- 172.21.15.17:0/1983655204 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32c00b8e60 con 0x7f32c40ff920 2024-02-14T15:03:09.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.606+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f32c402cde0 con 0x7f32c40ff920 2024-02-14T15:03:09.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.776+0000 7f32ba7fc640 1 -- 172.21.15.17:0/1983655204 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f32c00bc520 con 0x7f32c40ff920 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f32b00a2130 msgr2=0x7f32b00a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f32b00a2130 0x7f32b00a45f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f32c410ade0 tx=0x7f32b4005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 msgr2=0x7f32c410b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c410b070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32c0000c00 tx=0x7f32c0005e50 comp rx=0 tx=0).stop 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 shutdown_connections 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f32b00a2130 0x7f32b00a45f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c41014d0 0x7f32c4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32c4100b10 0x7f32c41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 --2- 172.21.15.17:0/1983655204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32c40ff920 0x7f32c410b070 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 >> 172.21.15.17:0/1983655204 conn(0x7f32c40fae90 msgr2=0x7f32c4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:09.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 shutdown_connections 2024-02-14T15:03:09.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.778+0000 7f32cb684640 1 -- 172.21.15.17:0/1983655204 wait complete. 2024-02-14T15:03:09.790 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-02-14T15:03:09.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 -- 172.21.15.17:0/2210081272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10101d30 msgr2=0x7faf101085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 --2- 172.21.15.17:0/2210081272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10101d30 0x7faf101085c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faf08009ca0 tx=0x7faf0802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 -- 172.21.15.17:0/2210081272 shutdown_connections 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 --2- 172.21.15.17:0/2210081272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10101d30 0x7faf101085c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 --2- 172.21.15.17:0/2210081272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf10101370 0x7faf101017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 --2- 172.21.15.17:0/2210081272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf10100180 0x7faf101005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 -- 172.21.15.17:0/2210081272 >> 172.21.15.17:0/2210081272 conn(0x7faf100fb6f0 msgr2=0x7faf100fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 -- 172.21.15.17:0/2210081272 shutdown_connections 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 -- 172.21.15.17:0/2210081272 wait complete. 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 Processor -- start 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.895+0000 7faf178ed640 1 -- start start 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10100180 0x7faf101972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf10101370 0x7faf101977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf10101d30 0x7faf1019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1019eda0 con 0x7faf10100180 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1019ef10 con 0x7faf10101370 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1019f0d0 con 0x7faf10101d30 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10100180 0x7faf101972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10100180 0x7faf101972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58310/0 (socket says 172.21.15.17:58310) 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 -- 172.21.15.17:0/2733078898 learned_addr learned my addr 172.21.15.17:0/2733078898 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf14e88640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf10101370 0x7faf101977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 -- 172.21.15.17:0/2733078898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf10101d30 msgr2=0x7faf1019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf10101d30 0x7faf1019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 -- 172.21.15.17:0/2733078898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf10101370 msgr2=0x7faf101977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf10101370 0x7faf101977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 -- 172.21.15.17:0/2733078898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf08009900 con 0x7faf10100180 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf14e88640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf10101370 0x7faf101977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf15689640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10100180 0x7faf101972b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e9c0 tx=0x7faf0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf027fc640 1 -- 172.21.15.17:0/2733078898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04009700 con 0x7faf10100180 2024-02-14T15:03:09.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf027fc640 1 -- 172.21.15.17:0/2733078898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf0400cd80 con 0x7faf10100180 2024-02-14T15:03:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1019f3b0 con 0x7faf10100180 2024-02-14T15:03:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf027fc640 1 -- 172.21.15.17:0/2733078898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04010430 con 0x7faf10100180 2024-02-14T15:03:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.896+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1019f910 con 0x7faf10100180 2024-02-14T15:03:09.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.897+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf101074a0 con 0x7faf10100180 2024-02-14T15:03:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.898+0000 7faf027fc640 1 -- 172.21.15.17:0/2733078898 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7faf04003680 con 0x7faf10100180 2024-02-14T15:03:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.899+0000 7faf027fc640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faefc0a2130 0x7faefc0a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.899+0000 7faf14e88640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faefc0a2130 0x7faefc0a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.899+0000 7faf027fc640 1 -- 172.21.15.17:0/2733078898 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7faf04014070 con 0x7faf10100180 2024-02-14T15:03:09.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.900+0000 7faf14e88640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faefc0a2130 0x7faefc0a45f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faf101987c0 tx=0x7faf0c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:09.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:09.904+0000 7faf027fc640 1 -- 172.21.15.17:0/2733078898 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf04097100 con 0x7faf10100180 2024-02-14T15:03:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.113+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7faf1002cfe0 con 0x7faefc0a2130 2024-02-14T15:03:10.119 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:09 smithi017 ceph-mon[32549]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:03:10.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:09 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1983655204' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-14T15:03:10.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.134+0000 7faf027fc640 1 -- 172.21.15.17:0/2733078898 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7faf1002cfe0 con 0x7faefc0a2130 2024-02-14T15:03:10.135 INFO:teuthology.orchestra.run.smithi017.stdout:{} 2024-02-14T15:03:10.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faefc0a2130 msgr2=0x7faefc0a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faefc0a2130 0x7faefc0a45f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faf101987c0 tx=0x7faf0c005d30 comp rx=0 tx=0).stop 2024-02-14T15:03:10.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10100180 msgr2=0x7faf101972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10100180 0x7faf101972b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e9c0 tx=0x7faf0400ee80 comp rx=0 tx=0).stop 2024-02-14T15:03:10.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 shutdown_connections 2024-02-14T15:03:10.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7faefc0a2130 0x7faefc0a45f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf10101d30 0x7faf1019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf10101370 0x7faf101977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 --2- 172.21.15.17:0/2733078898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf10100180 0x7faf101972b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 >> 172.21.15.17:0/2733078898 conn(0x7faf100fb6f0 msgr2=0x7faf10104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:10.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 shutdown_connections 2024-02-14T15:03:10.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.136+0000 7faf178ed640 1 -- 172.21.15.17:0/2733078898 wait complete. 2024-02-14T15:03:10.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:09 smithi050 ceph-mon[36512]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:03:10.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:09 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1983655204' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-14T15:03:10.148 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-14T15:03:10.159 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-14T15:03:10.168 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:03:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:09 smithi076 ceph-mon[37057]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-14T15:03:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:09 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1983655204' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 -- 172.21.15.17:0/966072666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102e10 msgr2=0x7fc31c1096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 --2- 172.21.15.17:0/966072666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102e10 0x7fc31c1096a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc314009ca0 tx=0x7fc31402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 -- 172.21.15.17:0/966072666 shutdown_connections 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 --2- 172.21.15.17:0/966072666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102e10 0x7fc31c1096a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 --2- 172.21.15.17:0/966072666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc31c102450 0x7fc31c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 --2- 172.21.15.17:0/966072666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc31c101260 0x7fc31c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 -- 172.21.15.17:0/966072666 >> 172.21.15.17:0/966072666 conn(0x7fc31c0fc810 msgr2=0x7fc31c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 -- 172.21.15.17:0/966072666 shutdown_connections 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 -- 172.21.15.17:0/966072666 wait complete. 2024-02-14T15:03:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.270+0000 7fc322993640 1 Processor -- start 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc322993640 1 -- start start 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc322993640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc31c101260 0x7fc31c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc322993640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102450 0x7fc31c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc322993640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc31c102e10 0x7fc31c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc322993640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c19ed50 con 0x7fc31c102450 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc322993640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c19eec0 con 0x7fc31c102e10 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc322993640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c19f080 con 0x7fc31c101260 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102450 0x7fc31c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102450 0x7fc31c1977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58358/0 (socket says 172.21.15.17:58358) 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 -- 172.21.15.17:0/1589982019 learned_addr learned my addr 172.21.15.17:0/1589982019 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31bfff640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc31c101260 0x7fc31c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc320f30640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc31c102e10 0x7fc31c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 -- 172.21.15.17:0/1589982019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc31c101260 msgr2=0x7fc31c197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc31c101260 0x7fc31c197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 -- 172.21.15.17:0/1589982019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc31c102e10 msgr2=0x7fc31c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc31c102e10 0x7fc31c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc31b7fe640 1 -- 172.21.15.17:0/1589982019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc314009900 con 0x7fc31c102450 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.271+0000 7fc320f30640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc31c102e10 0x7fc31c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.272+0000 7fc31b7fe640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102450 0x7fc31c1977a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc31000d770 tx=0x7fc31000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.272+0000 7fc3197fa640 1 -- 172.21.15.17:0/1589982019 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310009c30 con 0x7fc31c102450 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.272+0000 7fc3197fa640 1 -- 172.21.15.17:0/1589982019 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc31000de80 con 0x7fc31c102450 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.272+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c19f360 con 0x7fc31c102450 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.272+0000 7fc3197fa640 1 -- 172.21.15.17:0/1589982019 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310004e80 con 0x7fc31c102450 2024-02-14T15:03:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.272+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c19f820 con 0x7fc31c102450 2024-02-14T15:03:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.273+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31c108520 con 0x7fc31c102450 2024-02-14T15:03:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.274+0000 7fc3197fa640 1 -- 172.21.15.17:0/1589982019 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc310005020 con 0x7fc31c102450 2024-02-14T15:03:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.274+0000 7fc3197fa640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3040a20c0 0x7fc3040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.275+0000 7fc31bfff640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3040a20c0 0x7fc3040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.275+0000 7fc3197fa640 1 -- 172.21.15.17:0/1589982019 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fc3100cda10 con 0x7fc31c102450 2024-02-14T15:03:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.275+0000 7fc31bfff640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3040a20c0 0x7fc3040a4580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc31c1022b0 tx=0x7fc30c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.279+0000 7fc3197fa640 1 -- 172.21.15.17:0/1589982019 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc310096690 con 0x7fc31c102450 2024-02-14T15:03:10.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.462+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc31c02ce50 con 0x7fc31c102450 2024-02-14T15:03:10.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.463+0000 7fc3197fa640 1 -- 172.21.15.17:0/1589982019 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fc310099d50 con 0x7fc31c102450 2024-02-14T15:03:10.464 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:10.465 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:02:45.606961+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:03:10.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.465+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3040a20c0 msgr2=0x7fc3040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.465+0000 7fc322993640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3040a20c0 0x7fc3040a4580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc31c1022b0 tx=0x7fc30c00b390 comp rx=0 tx=0).stop 2024-02-14T15:03:10.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.465+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102450 msgr2=0x7fc31c1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.465+0000 7fc322993640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102450 0x7fc31c1977a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc31000d770 tx=0x7fc31000dc30 comp rx=0 tx=0).stop 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 shutdown_connections 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc3040a20c0 0x7fc3040a4580 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c102450 0x7fc31c1977a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc31c102e10 0x7fc31c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 --2- 172.21.15.17:0/1589982019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc31c101260 0x7fc31c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 >> 172.21.15.17:0/1589982019 conn(0x7fc31c0fc810 msgr2=0x7fc31c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 shutdown_connections 2024-02-14T15:03:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.466+0000 7fc322993640 1 -- 172.21.15.17:0/1589982019 wait complete. 2024-02-14T15:03:10.476 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:10.118+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:10.476 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:10.118+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:10.477 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:10.118+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:10.477 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:10.118+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:10.477 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:10.118+0000 7f09c9e73700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:10.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:03:10.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 -- 172.21.15.17:0/4292050948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101340 msgr2=0x7fc5941017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 --2- 172.21.15.17:0/4292050948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101340 0x7fc5941017c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc588009900 tx=0x7fc58802bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 -- 172.21.15.17:0/4292050948 shutdown_connections 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 --2- 172.21.15.17:0/4292050948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101340 0x7fc5941017c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 --2- 172.21.15.17:0/4292050948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc594101d00 0x7fc594108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 --2- 172.21.15.17:0/4292050948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc594100150 0x7fc594100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 -- 172.21.15.17:0/4292050948 >> 172.21.15.17:0/4292050948 conn(0x7fc5940fb6c0 msgr2=0x7fc5940fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 -- 172.21.15.17:0/4292050948 shutdown_connections 2024-02-14T15:03:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.583+0000 7fc598a55640 1 -- 172.21.15.17:0/4292050948 wait complete. 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 Processor -- start 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 -- start start 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc594100150 0x7fc594197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc594101340 0x7fc594197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101d00 0x7fc59419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59419ee20 con 0x7fc594101d00 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59419ef90 con 0x7fc594101340 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc598a55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59419f150 con 0x7fc594100150 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101d00 0x7fc59419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101d00 0x7fc59419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58824/0 (socket says 172.21.15.17:58824) 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 -- 172.21.15.17:0/898972133 learned_addr learned my addr 172.21.15.17:0/898972133 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc591d9b640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc594101340 0x7fc594197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc59259c640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc594100150 0x7fc594197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 -- 172.21.15.17:0/898972133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc594100150 msgr2=0x7fc594197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc594100150 0x7fc594197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 -- 172.21.15.17:0/898972133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc594101340 msgr2=0x7fc594197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc594101340 0x7fc594197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.584+0000 7fc592d9d640 1 -- 172.21.15.17:0/898972133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc588009560 con 0x7fc594101d00 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.585+0000 7fc592d9d640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101d00 0x7fc59419e8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009fd0 tx=0x7fc58c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.585+0000 7fc5837fe640 1 -- 172.21.15.17:0/898972133 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00bea0 con 0x7fc594101d00 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.585+0000 7fc5837fe640 1 -- 172.21.15.17:0/898972133 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc58c009940 con 0x7fc594101d00 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.585+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59419f430 con 0x7fc594101d00 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.585+0000 7fc5837fe640 1 -- 172.21.15.17:0/898972133 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0106c0 con 0x7fc594101d00 2024-02-14T15:03:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.585+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59419f910 con 0x7fc594101d00 2024-02-14T15:03:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.586+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59410bb60 con 0x7fc594101d00 2024-02-14T15:03:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.587+0000 7fc5837fe640 1 -- 172.21.15.17:0/898972133 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc58c004240 con 0x7fc594101d00 2024-02-14T15:03:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.588+0000 7fc5837fe640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc57c0a2140 0x7fc57c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.588+0000 7fc59259c640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc57c0a2140 0x7fc57c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.588+0000 7fc5837fe640 1 -- 172.21.15.17:0/898972133 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fc58c0cdbf0 con 0x7fc594101d00 2024-02-14T15:03:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.588+0000 7fc59259c640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc57c0a2140 0x7fc57c0a4600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc584005fd0 tx=0x7fc58400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:10.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.593+0000 7fc5837fe640 1 -- 172.21.15.17:0/898972133 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc58c096920 con 0x7fc594101d00 2024-02-14T15:03:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.775+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc59419fd90 con 0x7fc594101d00 2024-02-14T15:03:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.776+0000 7fc5837fe640 1 -- 172.21.15.17:0/898972133 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc58c099fe0 con 0x7fc594101d00 2024-02-14T15:03:10.776 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:03:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc57c0a2140 msgr2=0x7fc57c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc57c0a2140 0x7fc57c0a4600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc584005fd0 tx=0x7fc58400b390 comp rx=0 tx=0).stop 2024-02-14T15:03:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101d00 msgr2=0x7fc59419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101d00 0x7fc59419e8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009fd0 tx=0x7fc58c00b260 comp rx=0 tx=0).stop 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 shutdown_connections 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc57c0a2140 0x7fc57c0a4600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc594101d00 0x7fc59419e8e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc594101340 0x7fc594197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 --2- 172.21.15.17:0/898972133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc594100150 0x7fc594197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.777+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 >> 172.21.15.17:0/898972133 conn(0x7fc5940fb6c0 msgr2=0x7fc5940fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.778+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 shutdown_connections 2024-02-14T15:03:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.778+0000 7fc598a55640 1 -- 172.21.15.17:0/898972133 wait complete. 2024-02-14T15:03:10.790 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 -- 172.21.15.17:0/2699143861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 msgr2=0x7f43cc1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 --2- 172.21.15.17:0/2699143861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1017c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009900 tx=0x7f43b802bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 -- 172.21.15.17:0/2699143861 shutdown_connections 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 --2- 172.21.15.17:0/2699143861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1017c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 --2- 172.21.15.17:0/2699143861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43cc101d00 0x7f43cc108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 --2- 172.21.15.17:0/2699143861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43cc100150 0x7f43cc100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 -- 172.21.15.17:0/2699143861 >> 172.21.15.17:0/2699143861 conn(0x7f43cc0fb6c0 msgr2=0x7f43cc0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 -- 172.21.15.17:0/2699143861 shutdown_connections 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.894+0000 7f43d2fac640 1 -- 172.21.15.17:0/2699143861 wait complete. 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 Processor -- start 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 -- start start 2024-02-14T15:03:10.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43cc100150 0x7f43cc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43cc101d00 0x7f43cc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19edd0 con 0x7f43cc101340 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19ef40 con 0x7f43cc101d00 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d2fac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19f100 con 0x7f43cc100150 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58846/0 (socket says 172.21.15.17:58846) 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 -- 172.21.15.17:0/1017081377 learned_addr learned my addr 172.21.15.17:0/1017081377 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d0d48640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43cc100150 0x7f43cc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d1549640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43cc101d00 0x7f43cc19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 -- 172.21.15.17:0/1017081377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43cc100150 msgr2=0x7f43cc1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43cc100150 0x7f43cc1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 -- 172.21.15.17:0/1017081377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43cc101d00 msgr2=0x7f43cc19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:10.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43cc101d00 0x7f43cc19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 -- 172.21.15.17:0/1017081377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009560 con 0x7f43cc101340 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43d1549640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43cc101d00 0x7f43cc19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.895+0000 7f43cbfff640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1978e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f43b8006010 tx=0x7f43b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.896+0000 7f43c9ffb640 1 -- 172.21.15.17:0/1017081377 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80044e0 con 0x7f43cc101340 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.896+0000 7f43c9ffb640 1 -- 172.21.15.17:0/1017081377 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b80054e0 con 0x7f43cc101340 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.896+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc19f380 con 0x7f43cc101340 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.896+0000 7f43c9ffb640 1 -- 172.21.15.17:0/1017081377 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b803e450 con 0x7f43cc101340 2024-02-14T15:03:10.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.896+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc19f890 con 0x7f43cc101340 2024-02-14T15:03:10.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.897+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc10bb60 con 0x7f43cc101340 2024-02-14T15:03:10.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.898+0000 7f43c9ffb640 1 -- 172.21.15.17:0/1017081377 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f43b80035f0 con 0x7f43cc101340 2024-02-14T15:03:10.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.898+0000 7f43c9ffb640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43bc0a20c0 0x7f43bc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:10.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.898+0000 7f43c9ffb640 1 -- 172.21.15.17:0/1017081377 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f43b80f0310 con 0x7f43cc101340 2024-02-14T15:03:10.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.898+0000 7f43d0d48640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43bc0a20c0 0x7f43bc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:10.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.899+0000 7f43d0d48640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43bc0a20c0 0x7f43bc0a4580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43c0005fd0 tx=0x7f43c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:10.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:10.903+0000 7f43c9ffb640 1 -- 172.21.15.17:0/1017081377 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43b80b9030 con 0x7f43cc101340 2024-02-14T15:03:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.084+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43cc19fab0 con 0x7f43cc101340 2024-02-14T15:03:11.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.084+0000 7f43c9ffb640 1 -- 172.21.15.17:0/1017081377 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43b80bc6f0 con 0x7f43cc101340 2024-02-14T15:03:11.085 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.086+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43bc0a20c0 msgr2=0x7f43bc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43bc0a20c0 0x7f43bc0a4580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43c0005fd0 tx=0x7f43c000b410 comp rx=0 tx=0).stop 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 msgr2=0x7f43cc1978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1978e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f43b8006010 tx=0x7f43b80042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 shutdown_connections 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f43bc0a20c0 0x7f43bc0a4580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43cc101340 0x7f43cc1978e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43cc101d00 0x7f43cc19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 --2- 172.21.15.17:0/1017081377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43cc100150 0x7f43cc1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 >> 172.21.15.17:0/1017081377 conn(0x7f43cc0fb6c0 msgr2=0x7f43cc0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 shutdown_connections 2024-02-14T15:03:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.087+0000 7f43d2fac640 1 -- 172.21.15.17:0/1017081377 wait complete. 2024-02-14T15:03:11.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1983655204' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-14T15:03:11.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:03:11.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-mon[32549]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:11.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1589982019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:11.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:10 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/898972133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:11.098 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:03:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:10 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1983655204' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-14T15:03:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:10 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:03:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:10 smithi050 ceph-mon[36512]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:10 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1589982019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:11.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:10 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/898972133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:11.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 -- 172.21.15.17:0/3935873510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec106f30 msgr2=0x7f56ec109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 --2- 172.21.15.17:0/3935873510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec106f30 0x7f56ec109380 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009ca0 tx=0x7f56dc02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 -- 172.21.15.17:0/3935873510 shutdown_connections 2024-02-14T15:03:11.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 --2- 172.21.15.17:0/3935873510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec106f30 0x7f56ec109380 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 --2- 172.21.15.17:0/3935873510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56ec104600 0x7f56ec1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 --2- 172.21.15.17:0/3935873510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec101cd0 0x7f56ec1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 -- 172.21.15.17:0/3935873510 >> 172.21.15.17:0/3935873510 conn(0x7f56ec0fb6c0 msgr2=0x7f56ec0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:11.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 -- 172.21.15.17:0/3935873510 shutdown_connections 2024-02-14T15:03:11.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.203+0000 7f56f11f4640 1 -- 172.21.15.17:0/3935873510 wait complete. 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 Processor -- start 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 -- start start 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56ec101cd0 0x7f56ec1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec104600 0x7f56ec1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec106f30 0x7f56ec19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec19ed90 con 0x7f56ec104600 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec19ef00 con 0x7f56ec106f30 2024-02-14T15:03:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56f11f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec19f0c0 con 0x7f56ec101cd0 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56ea59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec104600 0x7f56ec1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56ea59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec104600 0x7f56ec1977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58854/0 (socket says 172.21.15.17:58854) 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.204+0000 7f56ea59c640 1 -- 172.21.15.17:0/2954143726 learned_addr learned my addr 172.21.15.17:0/2954143726 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56ead9d640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56ec101cd0 0x7f56ec1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56eb59e640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec106f30 0x7f56ec19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56ea59c640 1 -- 172.21.15.17:0/2954143726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56ec101cd0 msgr2=0x7f56ec1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56ea59c640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56ec101cd0 0x7f56ec1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56ea59c640 1 -- 172.21.15.17:0/2954143726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec106f30 msgr2=0x7f56ec19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56ea59c640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec106f30 0x7f56ec19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56ea59c640 1 -- 172.21.15.17:0/2954143726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc009900 con 0x7f56ec104600 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56ea59c640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec104600 0x7f56ec1977e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d740 tx=0x7f56e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56d3fff640 1 -- 172.21.15.17:0/2954143726 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0009c30 con 0x7f56ec104600 2024-02-14T15:03:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec19f3a0 con 0x7f56ec104600 2024-02-14T15:03:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56d3fff640 1 -- 172.21.15.17:0/2954143726 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56e000de50 con 0x7f56ec104600 2024-02-14T15:03:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56d3fff640 1 -- 172.21.15.17:0/2954143726 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0010460 con 0x7f56ec104600 2024-02-14T15:03:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.205+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec19f8b0 con 0x7f56ec104600 2024-02-14T15:03:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.206+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec0fd1b0 con 0x7f56ec104600 2024-02-14T15:03:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.207+0000 7f56d3fff640 1 -- 172.21.15.17:0/2954143726 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f56e0004f10 con 0x7f56ec104600 2024-02-14T15:03:11.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.208+0000 7f56d3fff640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56d40a20c0 0x7f56d40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.208+0000 7f56d3fff640 1 -- 172.21.15.17:0/2954143726 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f56e00cdde0 con 0x7f56ec104600 2024-02-14T15:03:11.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.208+0000 7f56ead9d640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56d40a20c0 0x7f56d40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.208+0000 7f56ead9d640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56d40a20c0 0x7f56d40a4580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009bd0 tx=0x7f56e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:11.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.212+0000 7f56d3fff640 1 -- 172.21.15.17:0/2954143726 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56e0096a60 con 0x7f56ec104600 2024-02-14T15:03:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:10 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1983655204' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-14T15:03:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:10 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:03:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:10 smithi076 ceph-mon[37057]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:11.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:10 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1589982019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:11.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:10 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/898972133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.393+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56ec10cd80 con 0x7f56ec104600 2024-02-14T15:03:11.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.393+0000 7f56d3fff640 1 -- 172.21.15.17:0/2954143726 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56e00170a0 con 0x7f56ec104600 2024-02-14T15:03:11.394 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:03:11.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56d40a20c0 msgr2=0x7f56d40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56d40a20c0 0x7f56d40a4580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009bd0 tx=0x7f56e4005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:11.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec104600 msgr2=0x7f56ec1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec104600 0x7f56ec1977e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d740 tx=0x7f56e000dc00 comp rx=0 tx=0).stop 2024-02-14T15:03:11.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 shutdown_connections 2024-02-14T15:03:11.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f56d40a20c0 0x7f56d40a4580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56ec104600 0x7f56ec1977e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec106f30 0x7f56ec19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 --2- 172.21.15.17:0/2954143726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56ec101cd0 0x7f56ec1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 >> 172.21.15.17:0/2954143726 conn(0x7f56ec0fb6c0 msgr2=0x7f56ec102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:11.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 shutdown_connections 2024-02-14T15:03:11.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.395+0000 7f56f11f4640 1 -- 172.21.15.17:0/2954143726 wait complete. 2024-02-14T15:03:11.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:03:11.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 -- 172.21.15.17:0/2947749404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 msgr2=0x7ff4b41040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/2947749404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b41040c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009900 tx=0x7ff4b002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:11.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 -- 172.21.15.17:0/2947749404 shutdown_connections 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/2947749404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b4106f30 0x7ff4b4109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/2947749404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4104600 0x7ff4b41069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/2947749404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b41040c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 -- 172.21.15.17:0/2947749404 >> 172.21.15.17:0/2947749404 conn(0x7ff4b40fb6c0 msgr2=0x7ff4b40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 -- 172.21.15.17:0/2947749404 shutdown_connections 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 -- 172.21.15.17:0/2947749404 wait complete. 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 Processor -- start 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.512+0000 7ff4bb9af640 1 -- start start 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b4101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b4104600 0x7ff4b40ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4106f30 0x7ff4b41001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4100920 con 0x7ff4b4101cd0 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4100a90 con 0x7ff4b4104600 2024-02-14T15:03:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b41a1360 con 0x7ff4b4106f30 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b4101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b4101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58870/0 (socket says 172.21.15.17:58870) 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 -- 172.21.15.17:0/1392485356 learned_addr learned my addr 172.21.15.17:0/1392485356 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b8f4a640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b4104600 0x7ff4b40ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 -- 172.21.15.17:0/1392485356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4106f30 msgr2=0x7ff4b41001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4106f30 0x7ff4b41001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 -- 172.21.15.17:0/1392485356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b4104600 msgr2=0x7ff4b40ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b4104600 0x7ff4b40ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 -- 172.21.15.17:0/1392485356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0009560 con 0x7ff4b4101cd0 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4b974b640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b4101640 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0036040 tx=0x7ff4b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4aa7fc640 1 -- 172.21.15.17:0/1392485356 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00044e0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4aa7fc640 1 -- 172.21.15.17:0/1392485356 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b00054e0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b41a15b0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4aa7fc640 1 -- 172.21.15.17:0/1392485356 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0003ec0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.513+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b41a1ac0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.514+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40fd1b0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.515+0000 7ff4aa7fc640 1 -- 172.21.15.17:0/1392485356 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff4b00035f0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.516+0000 7ff4aa7fc640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.516+0000 7ff4b8f4a640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.516+0000 7ff4aa7fc640 1 -- 172.21.15.17:0/1392485356 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7ff4b00f0440 con 0x7ff4b4101cd0 2024-02-14T15:03:11.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.516+0000 7ff4b8f4a640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4005d00 tx=0x7ff4a4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:11.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.520+0000 7ff4aa7fc640 1 -- 172.21.15.17:0/1392485356 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4b00b90c0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.700+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff4b402ceb0 con 0x7ff4b4101cd0 2024-02-14T15:03:11.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.701+0000 7ff4aa7fc640 1 -- 172.21.15.17:0/1392485356 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff4b00bc780 con 0x7ff4b4101cd0 2024-02-14T15:03:11.702 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 msgr2=0x7ff4a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4005d00 tx=0x7ff4a4005c50 comp rx=0 tx=0).stop 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 msgr2=0x7ff4b4101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b4101640 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0036040 tx=0x7ff4b00042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 shutdown_connections 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7ff4a00a20c0 0x7ff4a00a4580 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4106f30 0x7ff4b41001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4b4104600 0x7ff4b40ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 --2- 172.21.15.17:0/1392485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b4101cd0 0x7ff4b4101640 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 >> 172.21.15.17:0/1392485356 conn(0x7ff4b40fb6c0 msgr2=0x7ff4b4102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 shutdown_connections 2024-02-14T15:03:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.703+0000 7ff4bb9af640 1 -- 172.21.15.17:0/1392485356 wait complete. 2024-02-14T15:03:11.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:03:11.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 -- 172.21.15.17:0/1683211750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228100160 msgr2=0x7fc228100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 --2- 172.21.15.17:0/1683211750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228100160 0x7fc228100580 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc220009900 tx=0x7fc22002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 -- 172.21.15.17:0/1683211750 shutdown_connections 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 --2- 172.21.15.17:0/1683211750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc228101d10 0x7fc2281085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 --2- 172.21.15.17:0/1683211750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc228101350 0x7fc2281017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 --2- 172.21.15.17:0/1683211750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228100160 0x7fc228100580 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 -- 172.21.15.17:0/1683211750 >> 172.21.15.17:0/1683211750 conn(0x7fc2280fb6d0 msgr2=0x7fc2280fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 -- 172.21.15.17:0/1683211750 shutdown_connections 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.816+0000 7fc22cea4640 1 -- 172.21.15.17:0/1683211750 wait complete. 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 Processor -- start 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 -- start start 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc228100160 0x7fc228197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228101350 0x7fc228197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc228101d10 0x7fc22819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22819edf0 con 0x7fc228101350 2024-02-14T15:03:11.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22819ef60 con 0x7fc228101d10 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc22cea4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22819f120 con 0x7fc228100160 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc225d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228101350 0x7fc228197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.817+0000 7fc225d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228101350 0x7fc228197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58888/0 (socket says 172.21.15.17:58888) 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc225d9b640 1 -- 172.21.15.17:0/4038918145 learned_addr learned my addr 172.21.15.17:0/4038918145 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc22659c640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc228100160 0x7fc228197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc226d9d640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc228101d10 0x7fc22819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc225d9b640 1 -- 172.21.15.17:0/4038918145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc228100160 msgr2=0x7fc228197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc225d9b640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc228100160 0x7fc228197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc225d9b640 1 -- 172.21.15.17:0/4038918145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc228101d10 msgr2=0x7fc22819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc225d9b640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc228101d10 0x7fc22819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc225d9b640 1 -- 172.21.15.17:0/4038918145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc220009560 con 0x7fc228101350 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc226d9d640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc228101d10 0x7fc22819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc225d9b640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228101350 0x7fc228197840 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d770 tx=0x7fc21000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:11.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc21f7fe640 1 -- 172.21.15.17:0/4038918145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210009c30 con 0x7fc228101350 2024-02-14T15:03:11.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22819f400 con 0x7fc228101350 2024-02-14T15:03:11.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc21f7fe640 1 -- 172.21.15.17:0/4038918145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc21000de80 con 0x7fc228101350 2024-02-14T15:03:11.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc21f7fe640 1 -- 172.21.15.17:0/4038918145 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210004e80 con 0x7fc228101350 2024-02-14T15:03:11.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.818+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22819f910 con 0x7fc228101350 2024-02-14T15:03:11.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.819+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc228107480 con 0x7fc228101350 2024-02-14T15:03:11.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.820+0000 7fc21f7fe640 1 -- 172.21.15.17:0/4038918145 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc210005020 con 0x7fc228101350 2024-02-14T15:03:11.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.821+0000 7fc21f7fe640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc2140a20c0 0x7fc2140a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:11.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.821+0000 7fc22659c640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc2140a20c0 0x7fc2140a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:11.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.821+0000 7fc21f7fe640 1 -- 172.21.15.17:0/4038918145 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fc2100cda90 con 0x7fc228101350 2024-02-14T15:03:11.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.821+0000 7fc22659c640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc2140a20c0 0x7fc2140a4580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc220005ab0 tx=0x7fc220005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:11.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:11.825+0000 7fc21f7fe640 1 -- 172.21.15.17:0/4038918145 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2100966b0 con 0x7fc228101350 2024-02-14T15:03:12.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.008+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc22802cde0 con 0x7fc228101350 2024-02-14T15:03:12.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.009+0000 7fc21f7fe640 1 -- 172.21.15.17:0/4038918145 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fc210099d70 con 0x7fc228101350 2024-02-14T15:03:12.010 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:12.010 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:02:45.606961+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:03:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc2140a20c0 msgr2=0x7fc2140a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc2140a20c0 0x7fc2140a4580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc220005ab0 tx=0x7fc220005fd0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228101350 msgr2=0x7fc228197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228101350 0x7fc228197840 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d770 tx=0x7fc21000dc30 comp rx=0 tx=0).stop 2024-02-14T15:03:12.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 shutdown_connections 2024-02-14T15:03:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fc2140a20c0 0x7fc2140a4580 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc228101350 0x7fc228197840 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc228101d10 0x7fc22819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 --2- 172.21.15.17:0/4038918145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc228100160 0x7fc228197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 >> 172.21.15.17:0/4038918145 conn(0x7fc2280fb6d0 msgr2=0x7fc228104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 shutdown_connections 2024-02-14T15:03:12.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.011+0000 7fc22cea4640 1 -- 172.21.15.17:0/4038918145 wait complete. 2024-02-14T15:03:12.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:11 smithi017 ceph-mon[32549]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-14T15:03:12.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:11 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1017081377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:11 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2954143726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:11 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1392485356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.023 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:03:12.024 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:03:12.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 -- 172.21.15.17:0/2057360686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044104600 msgr2=0x7f90441069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 --2- 172.21.15.17:0/2057360686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044104600 0x7f90441069f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9034009900 tx=0x7f903402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:12.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 -- 172.21.15.17:0/2057360686 shutdown_connections 2024-02-14T15:03:12.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 --2- 172.21.15.17:0/2057360686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044104600 0x7f90441069f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 --2- 172.21.15.17:0/2057360686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9044106f30 0x7f9044109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 --2- 172.21.15.17:0/2057360686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044101cd0 0x7f90441040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 -- 172.21.15.17:0/2057360686 >> 172.21.15.17:0/2057360686 conn(0x7f90440fb6e0 msgr2=0x7f90440fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 -- 172.21.15.17:0/2057360686 shutdown_connections 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.133+0000 7f904a593640 1 -- 172.21.15.17:0/2057360686 wait complete. 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 Processor -- start 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 -- start start 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044101cd0 0x7f90441972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9044104600 0x7f9044197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044106f30 0x7f904419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904419edc0 con 0x7f9044106f30 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904419ef30 con 0x7f9044104600 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f904a593640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904419f0f0 con 0x7f9044101cd0 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f9048b30640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044106f30 0x7f904419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f9048b30640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044106f30 0x7f904419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58906/0 (socket says 172.21.15.17:58906) 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f9048b30640 1 -- 172.21.15.17:0/3232422340 learned_addr learned my addr 172.21.15.17:0/3232422340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f90437fe640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9044104600 0x7f9044197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.134+0000 7f9043fff640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044101cd0 0x7f90441972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f9048b30640 1 -- 172.21.15.17:0/3232422340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044101cd0 msgr2=0x7f90441972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f9048b30640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044101cd0 0x7f90441972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f9048b30640 1 -- 172.21.15.17:0/3232422340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9044104600 msgr2=0x7f9044197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f9048b30640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9044104600 0x7f9044197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f9048b30640 1 -- 172.21.15.17:0/3232422340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9034009560 con 0x7f9044106f30 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f9043fff640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044101cd0 0x7f90441972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f9048b30640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044106f30 0x7f904419e880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f903c00eb10 tx=0x7f903c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f90417fa640 1 -- 172.21.15.17:0/3232422340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c004b20 con 0x7f9044106f30 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f90417fa640 1 -- 172.21.15.17:0/3232422340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f903c004c80 con 0x7f9044106f30 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904419f3d0 con 0x7f9044106f30 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f90417fa640 1 -- 172.21.15.17:0/3232422340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c010650 con 0x7f9044106f30 2024-02-14T15:03:12.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.135+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904419f8b0 con 0x7f9044106f30 2024-02-14T15:03:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.136+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90440fd1d0 con 0x7f9044106f30 2024-02-14T15:03:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.137+0000 7f90417fa640 1 -- 172.21.15.17:0/3232422340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f903c005020 con 0x7f9044106f30 2024-02-14T15:03:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.138+0000 7f90417fa640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f902c0a2140 0x7f902c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.138+0000 7f9043fff640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f902c0a2140 0x7f902c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.138+0000 7f90417fa640 1 -- 172.21.15.17:0/3232422340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f903c014070 con 0x7f9044106f30 2024-02-14T15:03:12.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.138+0000 7f9043fff640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f902c0a2140 0x7f902c0a4600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9038009b40 tx=0x7f9038005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:12.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:11 smithi050 ceph-mon[36512]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-14T15:03:12.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:11 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1017081377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:11 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2954143726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:11 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1392485356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.142+0000 7f90417fa640 1 -- 172.21.15.17:0/3232422340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f903c097240 con 0x7f9044106f30 2024-02-14T15:03:12.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:11 smithi076 ceph-mon[37057]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-14T15:03:12.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:11 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1017081377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:11 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2954143726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:11 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1392485356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.324+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f904402ce50 con 0x7f9044106f30 2024-02-14T15:03:12.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.325+0000 7f90417fa640 1 -- 172.21.15.17:0/3232422340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f903c09a900 con 0x7f9044106f30 2024-02-14T15:03:12.326 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:03:12.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.326+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f902c0a2140 msgr2=0x7f902c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.326+0000 7f904a593640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f902c0a2140 0x7f902c0a4600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9038009b40 tx=0x7f9038005e50 comp rx=0 tx=0).stop 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044106f30 msgr2=0x7f904419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044106f30 0x7f904419e880 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f903c00eb10 tx=0x7f903c00c2d0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 shutdown_connections 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f902c0a2140 0x7f902c0a4600 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9044106f30 0x7f904419e880 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9044104600 0x7f9044197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 --2- 172.21.15.17:0/3232422340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044101cd0 0x7f90441972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 >> 172.21.15.17:0/3232422340 conn(0x7f90440fb6e0 msgr2=0x7f9044102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 shutdown_connections 2024-02-14T15:03:12.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.327+0000 7f904a593640 1 -- 172.21.15.17:0/3232422340 wait complete. 2024-02-14T15:03:12.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:03:12.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 -- 172.21.15.17:0/1273061240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c102450 msgr2=0x7fdb4c1028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 --2- 172.21.15.17:0/1273061240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c102450 0x7fdb4c1028d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009900 tx=0x7fdb3802bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:12.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 -- 172.21.15.17:0/1273061240 shutdown_connections 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 --2- 172.21.15.17:0/1273061240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c102450 0x7fdb4c1028d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 --2- 172.21.15.17:0/1273061240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb4c102e10 0x7fdb4c1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 --2- 172.21.15.17:0/1273061240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb4c101260 0x7fdb4c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 -- 172.21.15.17:0/1273061240 >> 172.21.15.17:0/1273061240 conn(0x7fdb4c0fc7f0 msgr2=0x7fdb4c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 -- 172.21.15.17:0/1273061240 shutdown_connections 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 -- 172.21.15.17:0/1273061240 wait complete. 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 Processor -- start 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.444+0000 7fdb5321b640 1 -- start start 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb5321b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c101260 0x7fdb4c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb5321b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb4c102450 0x7fdb4c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb5321b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb4c102e10 0x7fdb4c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb5321b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c19edb0 con 0x7fdb4c101260 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb5321b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c19ef20 con 0x7fdb4c102450 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb5321b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c19f0e0 con 0x7fdb4c102e10 2024-02-14T15:03:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c101260 0x7fdb4c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c101260 0x7fdb4c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58934/0 (socket says 172.21.15.17:58934) 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 -- 172.21.15.17:0/3357887392 learned_addr learned my addr 172.21.15.17:0/3357887392 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb4bfff640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb4c102450 0x7fdb4c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb517b8640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb4c102e10 0x7fdb4c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 -- 172.21.15.17:0/3357887392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb4c102e10 msgr2=0x7fdb4c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb4c102e10 0x7fdb4c19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 -- 172.21.15.17:0/3357887392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb4c102450 msgr2=0x7fdb4c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb4c102450 0x7fdb4c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 -- 172.21.15.17:0/3357887392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38009560 con 0x7fdb4c101260 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb517b8640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb4c102e10 0x7fdb4c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb4bfff640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb4c102450 0x7fdb4c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb50fb7640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c101260 0x7fdb4c1972c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000d950 tx=0x7fdb4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb49ffb640 1 -- 172.21.15.17:0/3357887392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40009c30 con 0x7fdb4c101260 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb49ffb640 1 -- 172.21.15.17:0/3357887392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb40009d90 con 0x7fdb4c101260 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c19f3c0 con 0x7fdb4c101260 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.445+0000 7fdb49ffb640 1 -- 172.21.15.17:0/3357887392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000f720 con 0x7fdb4c101260 2024-02-14T15:03:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.446+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c19f8d0 con 0x7fdb4c101260 2024-02-14T15:03:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.447+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c108520 con 0x7fdb4c101260 2024-02-14T15:03:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.448+0000 7fdb49ffb640 1 -- 172.21.15.17:0/3357887392 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdb40010a60 con 0x7fdb4c101260 2024-02-14T15:03:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.448+0000 7fdb49ffb640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdb3c0a2140 0x7fdb3c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.448+0000 7fdb4bfff640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdb3c0a2140 0x7fdb3c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.448+0000 7fdb49ffb640 1 -- 172.21.15.17:0/3357887392 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fdb400cddf0 con 0x7fdb4c101260 2024-02-14T15:03:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.449+0000 7fdb4bfff640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdb3c0a2140 0x7fdb3c0a4600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c1987d0 tx=0x7fdb38005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.452+0000 7fdb49ffb640 1 -- 172.21.15.17:0/3357887392 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb40096a10 con 0x7fdb4c101260 2024-02-14T15:03:12.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.645+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb4c02ce50 con 0x7fdb4c101260 2024-02-14T15:03:12.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.645+0000 7fdb49ffb640 1 -- 172.21.15.17:0/3357887392 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb4009a0d0 con 0x7fdb4c101260 2024-02-14T15:03:12.646 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdb3c0a2140 msgr2=0x7fdb3c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdb3c0a2140 0x7fdb3c0a4600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c1987d0 tx=0x7fdb38005be0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c101260 msgr2=0x7fdb4c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c101260 0x7fdb4c1972c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000d950 tx=0x7fdb4000de10 comp rx=0 tx=0).stop 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 shutdown_connections 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdb3c0a2140 0x7fdb3c0a4600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb4c102e10 0x7fdb4c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb4c102450 0x7fdb4c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 --2- 172.21.15.17:0/3357887392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb4c101260 0x7fdb4c1972c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 >> 172.21.15.17:0/3357887392 conn(0x7fdb4c0fc7f0 msgr2=0x7fdb4c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 shutdown_connections 2024-02-14T15:03:12.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.648+0000 7fdb5321b640 1 -- 172.21.15.17:0/3357887392 wait complete. 2024-02-14T15:03:12.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 -- 172.21.15.17:0/1782245752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8102df0 msgr2=0x7fdad8109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 --2- 172.21.15.17:0/1782245752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8102df0 0x7fdad8109680 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009ca0 tx=0x7fdad002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 -- 172.21.15.17:0/1782245752 shutdown_connections 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 --2- 172.21.15.17:0/1782245752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8102df0 0x7fdad8109680 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 --2- 172.21.15.17:0/1782245752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdad8102430 0x7fdad81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 --2- 172.21.15.17:0/1782245752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdad8101240 0x7fdad8101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 -- 172.21.15.17:0/1782245752 >> 172.21.15.17:0/1782245752 conn(0x7fdad80fc7b0 msgr2=0x7fdad80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 -- 172.21.15.17:0/1782245752 shutdown_connections 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 -- 172.21.15.17:0/1782245752 wait complete. 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 Processor -- start 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.766+0000 7fdade9d0640 1 -- start start 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8101240 0x7fdad8197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdad8102430 0x7fdad8197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdad8102df0 0x7fdad819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad819ecf0 con 0x7fdad8101240 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad819ee60 con 0x7fdad8102df0 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad819f020 con 0x7fdad8102430 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8101240 0x7fdad8197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8101240 0x7fdad8197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58952/0 (socket says 172.21.15.17:58952) 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 -- 172.21.15.17:0/2441812074 learned_addr learned my addr 172.21.15.17:0/2441812074 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdadcf6d640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdad8102df0 0x7fdad819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad77fe640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdad8102430 0x7fdad8197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 -- 172.21.15.17:0/2441812074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdad8102430 msgr2=0x7fdad8197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdad8102430 0x7fdad8197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 -- 172.21.15.17:0/2441812074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdad8102df0 msgr2=0x7fdad819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdad8102df0 0x7fdad819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 -- 172.21.15.17:0/2441812074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0009900 con 0x7fdad8101240 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad7fff640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8101240 0x7fdad8197200 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c970 tx=0x7fdac000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:12.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0007c10 con 0x7fdad8101240 2024-02-14T15:03:12.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdac0007d70 con 0x7fdad8101240 2024-02-14T15:03:12.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad819f300 con 0x7fdad8101240 2024-02-14T15:03:12.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac000f8a0 con 0x7fdad8101240 2024-02-14T15:03:12.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.767+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad819f810 con 0x7fdad8101240 2024-02-14T15:03:12.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.768+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad8108500 con 0x7fdad8101240 2024-02-14T15:03:12.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.770+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdac0018af0 con 0x7fdad8101240 2024-02-14T15:03:12.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.770+0000 7fdad57fa640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdac40a20c0 0x7fdac40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:12.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.770+0000 7fdad77fe640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdac40a20c0 0x7fdac40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:12.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.770+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fdac0013070 con 0x7fdad8101240 2024-02-14T15:03:12.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.771+0000 7fdad77fe640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdac40a20c0 0x7fdac40a4580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005fd0 tx=0x7fdac8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:12.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.774+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdac0097360 con 0x7fdad8101240 2024-02-14T15:03:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.796+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fdac0097730 con 0x7fdad8101240 2024-02-14T15:03:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.961+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdad810cc50 con 0x7fdad8101240 2024-02-14T15:03:12.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.961+0000 7fdad57fa640 1 -- 172.21.15.17:0/2441812074 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdac00c2580 con 0x7fdad8101240 2024-02-14T15:03:12.962 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:03:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdac40a20c0 msgr2=0x7fdac40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdac40a20c0 0x7fdac40a4580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005fd0 tx=0x7fdac8005850 comp rx=0 tx=0).stop 2024-02-14T15:03:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8101240 msgr2=0x7fdad8197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8101240 0x7fdad8197200 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c970 tx=0x7fdac000ce30 comp rx=0 tx=0).stop 2024-02-14T15:03:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 shutdown_connections 2024-02-14T15:03:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fdac40a20c0 0x7fdac40a4580 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdad8102df0 0x7fdad819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdad8102430 0x7fdad8197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 --2- 172.21.15.17:0/2441812074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad8101240 0x7fdad8197200 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 >> 172.21.15.17:0/2441812074 conn(0x7fdad80fc7b0 msgr2=0x7fdad81056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 shutdown_connections 2024-02-14T15:03:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:12.963+0000 7fdade9d0640 1 -- 172.21.15.17:0/2441812074 wait complete. 2024-02-14T15:03:12.974 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:12 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4038918145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:12.974 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:12 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3232422340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.974 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:12 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3357887392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:12.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:03:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 -- 172.21.15.17:0/2073570095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 msgr2=0x7f73cc1008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 --2- 172.21.15.17:0/2073570095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc1008f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009900 tx=0x7f73b802bcd0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 -- 172.21.15.17:0/2073570095 shutdown_connections 2024-02-14T15:03:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 --2- 172.21.15.17:0/2073570095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc1008f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 --2- 172.21.15.17:0/2073570095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc100e30 0x7f73cc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 --2- 172.21.15.17:0/2073570095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73cc0ffb30 0x7f73cc0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 -- 172.21.15.17:0/2073570095 >> 172.21.15.17:0/2073570095 conn(0x7f73cc0fb690 msgr2=0x7f73cc0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 -- 172.21.15.17:0/2073570095 shutdown_connections 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.080+0000 7f73d3015640 1 -- 172.21.15.17:0/2073570095 wait complete. 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 Processor -- start 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 -- start start 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc0ffb30 0x7f73cc1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73cc100e30 0x7f73cc1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc1a0fd0 con 0x7f73cc100490 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc1a1140 con 0x7f73cc0ffb30 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d3015640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc1a1300 con 0x7f73cc100e30 2024-02-14T15:03:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58968/0 (socket says 172.21.15.17:58968) 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 -- 172.21.15.17:0/4087931561 learned_addr learned my addr 172.21.15.17:0/4087931561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d0db1640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc0ffb30 0x7f73cc1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 -- 172.21.15.17:0/4087931561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73cc100e30 msgr2=0x7f73cc1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d15b2640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73cc100e30 0x7f73cc1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73cc100e30 0x7f73cc1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 -- 172.21.15.17:0/4087931561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc0ffb30 msgr2=0x7f73cc1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc0ffb30 0x7f73cc1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 -- 172.21.15.17:0/4087931561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b8009560 con 0x7f73cc100490 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73d0db1640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc0ffb30 0x7f73cc1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.081+0000 7f73cbfff640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc199a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f73b8006010 tx=0x7f73b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.082+0000 7f73c9ffb640 1 -- 172.21.15.17:0/4087931561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b80044e0 con 0x7f73cc100490 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.082+0000 7f73c9ffb640 1 -- 172.21.15.17:0/4087931561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73b80054e0 con 0x7f73cc100490 2024-02-14T15:03:13.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.082+0000 7f73c9ffb640 1 -- 172.21.15.17:0/4087931561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b803e450 con 0x7f73cc100490 2024-02-14T15:03:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.082+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc1a1580 con 0x7f73cc100490 2024-02-14T15:03:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.082+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc1a1ae0 con 0x7f73cc100490 2024-02-14T15:03:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.083+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc10aa60 con 0x7f73cc100490 2024-02-14T15:03:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.084+0000 7f73c9ffb640 1 -- 172.21.15.17:0/4087931561 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f73b80035f0 con 0x7f73cc100490 2024-02-14T15:03:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.084+0000 7f73c9ffb640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f73bc0a21d0 0x7f73bc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.085+0000 7f73c9ffb640 1 -- 172.21.15.17:0/4087931561 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f73b80f0ae0 con 0x7f73cc100490 2024-02-14T15:03:13.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.085+0000 7f73d0db1640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f73bc0a21d0 0x7f73bc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.085+0000 7f73d0db1640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f73bc0a21d0 0x7f73bc0a4690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009b40 tx=0x7f73c0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:13.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.089+0000 7f73c9ffb640 1 -- 172.21.15.17:0/4087931561 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73b80b96b0 con 0x7f73cc100490 2024-02-14T15:03:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:12 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4038918145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:13.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:12 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3232422340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:13.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:12 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3357887392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:12 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4038918145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:12 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3232422340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:12 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3357887392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.270+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f73cc02cdb0 con 0x7f73cc100490 2024-02-14T15:03:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.271+0000 7f73c9ffb640 1 -- 172.21.15.17:0/4087931561 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f73b80bcd70 con 0x7f73cc100490 2024-02-14T15:03:13.271 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f73bc0a21d0 msgr2=0x7f73bc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f73bc0a21d0 0x7f73bc0a4690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009b40 tx=0x7f73c0005e50 comp rx=0 tx=0).stop 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 msgr2=0x7f73cc199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc199a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f73b8006010 tx=0x7f73b80042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 shutdown_connections 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f73bc0a21d0 0x7f73bc0a4690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73cc100490 0x7f73cc199a20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73cc100e30 0x7f73cc1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 --2- 172.21.15.17:0/4087931561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73cc0ffb30 0x7f73cc1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 >> 172.21.15.17:0/4087931561 conn(0x7f73cc0fb690 msgr2=0x7f73cc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 shutdown_connections 2024-02-14T15:03:13.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.273+0000 7f73d3015640 1 -- 172.21.15.17:0/4087931561 wait complete. 2024-02-14T15:03:13.285 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-14T15:03:13.286 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 -- 172.21.15.17:0/1922668001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 msgr2=0x7f6c7c101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 --2- 172.21.15.17:0/1922668001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c101830 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009900 tx=0x7f6c6c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 -- 172.21.15.17:0/1922668001 shutdown_connections 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 --2- 172.21.15.17:0/1922668001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c101830 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 --2- 172.21.15.17:0/1922668001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c7c101d70 0x7f6c7c108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 --2- 172.21.15.17:0/1922668001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c7c1001c0 0x7f6c7c1005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 -- 172.21.15.17:0/1922668001 >> 172.21.15.17:0/1922668001 conn(0x7f6c7c0fb710 msgr2=0x7f6c7c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 -- 172.21.15.17:0/1922668001 shutdown_connections 2024-02-14T15:03:13.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 -- 172.21.15.17:0/1922668001 wait complete. 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.388+0000 7f6c820eb640 1 Processor -- start 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c820eb640 1 -- start start 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c820eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c7c1001c0 0x7f6c7c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c820eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c820eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c7c101d70 0x7f6c7c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c820eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c19ed70 con 0x7f6c7c1013b0 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c820eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c19eee0 con 0x7f6c7c101d70 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c820eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c19f0a0 con 0x7f6c7c1001c0 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c1977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58998/0 (socket says 172.21.15.17:58998) 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 -- 172.21.15.17:0/76462526 learned_addr learned my addr 172.21.15.17:0/76462526 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7bfff640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c7c101d70 0x7f6c7c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7b7fe640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c7c1001c0 0x7f6c7c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 -- 172.21.15.17:0/76462526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c7c1001c0 msgr2=0x7f6c7c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c7c1001c0 0x7f6c7c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 -- 172.21.15.17:0/76462526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c7c101d70 msgr2=0x7f6c7c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c7c101d70 0x7f6c7c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.389+0000 7f6c7affd640 1 -- 172.21.15.17:0/76462526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009560 con 0x7f6c7c1013b0 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.390+0000 7f6c7affd640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c1977c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c006010 tx=0x7f6c6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.390+0000 7f6c78ff9640 1 -- 172.21.15.17:0/76462526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0044e0 con 0x7f6c7c1013b0 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.390+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c19f320 con 0x7f6c7c1013b0 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.390+0000 7f6c78ff9640 1 -- 172.21.15.17:0/76462526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c0054e0 con 0x7f6c7c1013b0 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.390+0000 7f6c78ff9640 1 -- 172.21.15.17:0/76462526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c03e450 con 0x7f6c7c1013b0 2024-02-14T15:03:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.390+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c19f880 con 0x7f6c7c1013b0 2024-02-14T15:03:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.391+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c10bb60 con 0x7f6c7c1013b0 2024-02-14T15:03:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.392+0000 7f6c78ff9640 1 -- 172.21.15.17:0/76462526 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6c6c0035f0 con 0x7f6c7c1013b0 2024-02-14T15:03:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.392+0000 7f6c78ff9640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c640a2150 0x7f6c640a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.393+0000 7f6c7b7fe640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c640a2150 0x7f6c640a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.393+0000 7f6c78ff9640 1 -- 172.21.15.17:0/76462526 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f6c6c0f0ab0 con 0x7f6c7c1013b0 2024-02-14T15:03:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.393+0000 7f6c7b7fe640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c640a2150 0x7f6c640a4610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005db0 tx=0x7f6c70005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:13.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.396+0000 7f6c78ff9640 1 -- 172.21.15.17:0/76462526 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c6c0b9680 con 0x7f6c7c1013b0 2024-02-14T15:03:13.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.615+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c7c10bcf0 con 0x7f6c7c1013b0 2024-02-14T15:03:13.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.616+0000 7f6c78ff9640 1 -- 172.21.15.17:0/76462526 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f6c6c0bcd40 con 0x7f6c7c1013b0 2024-02-14T15:03:13.617 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:13.617 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:03:09.607099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:03:13.617 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c640a2150 msgr2=0x7f6c640a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c640a2150 0x7f6c640a4610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005db0 tx=0x7f6c70005d40 comp rx=0 tx=0).stop 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 msgr2=0x7f6c7c1977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c1977c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c006010 tx=0x7f6c6c0042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 shutdown_connections 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f6c640a2150 0x7f6c640a4610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7c1013b0 0x7f6c7c1977c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c7c101d70 0x7f6c7c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 --2- 172.21.15.17:0/76462526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c7c1001c0 0x7f6c7c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 >> 172.21.15.17:0/76462526 conn(0x7f6c7c0fb710 msgr2=0x7f6c7c069950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 shutdown_connections 2024-02-14T15:03:13.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.618+0000 7f6c820eb640 1 -- 172.21.15.17:0/76462526 wait complete. 2024-02-14T15:03:13.630 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-14T15:03:13.630 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-14T15:03:13.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-14T15:03:13.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- 172.21.15.17:0/2159288989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 msgr2=0x7f76941085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 --2- 172.21.15.17:0/2159288989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f76941085a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7684009900 tx=0x7f768402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- 172.21.15.17:0/2159288989 shutdown_connections 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 --2- 172.21.15.17:0/2159288989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f76941085a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 --2- 172.21.15.17:0/2159288989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7694101350 0x7f76941017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 --2- 172.21.15.17:0/2159288989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694100160 0x7f7694100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- 172.21.15.17:0/2159288989 >> 172.21.15.17:0/2159288989 conn(0x7f76940fb6d0 msgr2=0x7f76940fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- 172.21.15.17:0/2159288989 shutdown_connections 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- 172.21.15.17:0/2159288989 wait complete. 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 Processor -- start 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- start start 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7694100160 0x7f7694197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694101350 0x7f76941977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f769419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769419ed50 con 0x7f7694101d10 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769419eec0 con 0x7f7694100160 2024-02-14T15:03:13.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f769af8c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769419f080 con 0x7f7694101350 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.732+0000 7f7699529640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f769419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f769419e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59022/0 (socket says 172.21.15.17:59022) 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 -- 172.21.15.17:0/1528794956 learned_addr learned my addr 172.21.15.17:0/1528794956 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7693fff640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694101350 0x7f76941977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7698d28640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7694100160 0x7f7694197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 -- 172.21.15.17:0/1528794956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694101350 msgr2=0x7f76941977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694101350 0x7f76941977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 -- 172.21.15.17:0/1528794956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7694100160 msgr2=0x7f7694197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7694100160 0x7f7694197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 -- 172.21.15.17:0/1528794956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7684009560 con 0x7f7694101d10 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7698d28640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7694100160 0x7f7694197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7699529640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f769419e810 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7684009230 tx=0x7f7684004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:13.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7691ffb640 1 -- 172.21.15.17:0/1528794956 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7684004df0 con 0x7f7694101d10 2024-02-14T15:03:13.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7691ffb640 1 -- 172.21.15.17:0/1528794956 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7684005430 con 0x7f7694101d10 2024-02-14T15:03:13.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769419f300 con 0x7f7694101d10 2024-02-14T15:03:13.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f7691ffb640 1 -- 172.21.15.17:0/1528794956 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7684003de0 con 0x7f7694101d10 2024-02-14T15:03:13.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.733+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769419f860 con 0x7f7694101d10 2024-02-14T15:03:13.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.734+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769410bb70 con 0x7f7694101d10 2024-02-14T15:03:13.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.735+0000 7f7691ffb640 1 -- 172.21.15.17:0/1528794956 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f76840055a0 con 0x7f7694101d10 2024-02-14T15:03:13.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.736+0000 7f7691ffb640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f767c0a21d0 0x7f767c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:13.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.736+0000 7f7691ffb640 1 -- 172.21.15.17:0/1528794956 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f76840f0ea0 con 0x7f7694101d10 2024-02-14T15:03:13.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.736+0000 7f7698d28640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f767c0a21d0 0x7f767c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:13.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.736+0000 7f7698d28640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f767c0a21d0 0x7f767c0a4690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f76941011b0 tx=0x7f768c009400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:13.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.740+0000 7f7691ffb640 1 -- 172.21.15.17:0/1528794956 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76840b9b90 con 0x7f7694101d10 2024-02-14T15:03:13.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.919+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76940fd820 con 0x7f767c0a21d0 2024-02-14T15:03:13.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.932+0000 7f7691ffb640 1 -- 172.21.15.17:0/1528794956 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428031 (secure 0 0 0) 0x7f76940fd820 con 0x7f767c0a21d0 2024-02-14T15:03:13.938 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:13.941 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-02-14T15:03:13.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f767c0a21d0 msgr2=0x7f767c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f767c0a21d0 0x7f767c0a4690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f76941011b0 tx=0x7f768c009400 comp rx=0 tx=0).stop 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 msgr2=0x7f769419e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f769419e810 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7684009230 tx=0x7f7684004270 comp rx=0 tx=0).stop 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 shutdown_connections 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f767c0a21d0 0x7f767c0a4690 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7694101d10 0x7f769419e810 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694101350 0x7f76941977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 --2- 172.21.15.17:0/1528794956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7694100160 0x7f7694197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 >> 172.21.15.17:0/1528794956 conn(0x7f76940fb6d0 msgr2=0x7f76940fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 shutdown_connections 2024-02-14T15:03:13.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:13.936+0000 7f769af8c640 1 -- 172.21.15.17:0/1528794956 wait complete. 2024-02-14T15:03:13.947 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:13 smithi017 ceph-mon[32549]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-14T15:03:13.947 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:13 smithi017 ceph-mon[32549]: mgrmap e19: x(active, since 9m), standbys: y, z 2024-02-14T15:03:13.947 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2441812074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:13.947 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/4087931561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:13.947 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:13 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/76462526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:13.948 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-02-14T15:03:12.696051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658657,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":846,"num_read_kb":545,"num_write":353,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1244674,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388604,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553220,"statfs":{"total":1151940427776,"available":1151542497280,"internally_reserved":0,"allocated":14778368,"data_stored":8176899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008748"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620302+0000","last_change":"2024-02-14T15:00:49.281057+0000","last_active":"2024-02-14T15:02:45.620302+0000","last_peered":"2024-02-14T15:02:45.620302+0000","last_clean":"2024-02-14T15:02:45.620302+0000","last_became_active":"2024-02-14T15:00:49.280642+0000","last_became_peered":"2024-02-14T15:00:49.280642+0000","last_unstale":"2024-02-14T15:02:45.620302+0000","last_undegraded":"2024-02-14T15:02:45.620302+0000","last_fullsized":"2024-02-14T15:02:45.620302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:03.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512248+0000","last_change":"2024-02-14T15:00:49.282794+0000","last_active":"2024-02-14T15:02:46.512248+0000","last_peered":"2024-02-14T15:02:46.512248+0000","last_clean":"2024-02-14T15:02:46.512248+0000","last_became_active":"2024-02-14T15:00:49.281441+0000","last_became_peered":"2024-02-14T15:00:49.281441+0000","last_unstale":"2024-02-14T15:02:46.512248+0000","last_undegraded":"2024-02-14T15:02:46.512248+0000","last_fullsized":"2024-02-14T15:02:46.512248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:57:36.577281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512181+0000","last_change":"2024-02-14T15:00:49.286253+0000","last_active":"2024-02-14T15:02:46.512181+0000","last_peered":"2024-02-14T15:02:46.512181+0000","last_clean":"2024-02-14T15:02:46.512181+0000","last_became_active":"2024-02-14T15:00:49.285950+0000","last_became_peered":"2024-02-14T15:00:49.285950+0000","last_unstale":"2024-02-14T15:02:46.512181+0000","last_undegraded":"2024-02-14T15:02:46.512181+0000","last_fullsized":"2024-02-14T15:02:46.512181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:53.395339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617486+0000","last_change":"2024-02-14T15:00:49.272764+0000","last_active":"2024-02-14T15:02:45.617486+0000","last_peered":"2024-02-14T15:02:45.617486+0000","last_clean":"2024-02-14T15:02:45.617486+0000","last_became_active":"2024-02-14T15:00:49.272430+0000","last_became_peered":"2024-02-14T15:00:49.272430+0000","last_unstale":"2024-02-14T15:02:45.617486+0000","last_undegraded":"2024-02-14T15:02:45.617486+0000","last_fullsized":"2024-02-14T15:02:45.617486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:36.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766939+0000","last_change":"2024-02-14T15:00:49.275028+0000","last_active":"2024-02-14T15:02:45.766939+0000","last_peered":"2024-02-14T15:02:45.766939+0000","last_clean":"2024-02-14T15:02:45.766939+0000","last_became_active":"2024-02-14T15:00:49.274686+0000","last_became_peered":"2024-02-14T15:00:49.274686+0000","last_unstale":"2024-02-14T15:02:45.766939+0000","last_undegraded":"2024-02-14T15:02:45.766939+0000","last_fullsized":"2024-02-14T15:02:45.766939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:21:23.465321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.705981+0000","last_change":"2024-02-14T15:00:49.287144+0000","last_active":"2024-02-14T15:02:45.705981+0000","last_peered":"2024-02-14T15:02:45.705981+0000","last_clean":"2024-02-14T15:02:45.705981+0000","last_became_active":"2024-02-14T15:00:49.286767+0000","last_became_peered":"2024-02-14T15:00:49.286767+0000","last_unstale":"2024-02-14T15:02:45.705981+0000","last_undegraded":"2024-02-14T15:02:45.705981+0000","last_fullsized":"2024-02-14T15:02:45.705981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:45:09.391595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767289+0000","last_change":"2024-02-14T15:00:49.274442+0000","last_active":"2024-02-14T15:02:45.767289+0000","last_peered":"2024-02-14T15:02:45.767289+0000","last_clean":"2024-02-14T15:02:45.767289+0000","last_became_active":"2024-02-14T15:00:49.274122+0000","last_became_peered":"2024-02-14T15:00:49.274122+0000","last_unstale":"2024-02-14T15:02:45.767289+0000","last_undegraded":"2024-02-14T15:02:45.767289+0000","last_fullsized":"2024-02-14T15:02:45.767289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:55.974889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.712957+0000","last_change":"2024-02-14T15:00:49.285432+0000","last_active":"2024-02-14T15:02:45.712957+0000","last_peered":"2024-02-14T15:02:45.712957+0000","last_clean":"2024-02-14T15:02:45.712957+0000","last_became_active":"2024-02-14T15:00:49.283703+0000","last_became_peered":"2024-02-14T15:00:49.283703+0000","last_unstale":"2024-02-14T15:02:45.712957+0000","last_undegraded":"2024-02-14T15:02:45.712957+0000","last_fullsized":"2024-02-14T15:02:45.712957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:14.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767120+0000","last_change":"2024-02-14T15:00:49.283908+0000","last_active":"2024-02-14T15:02:45.767120+0000","last_peered":"2024-02-14T15:02:45.767120+0000","last_clean":"2024-02-14T15:02:45.767120+0000","last_became_active":"2024-02-14T15:00:49.283131+0000","last_became_peered":"2024-02-14T15:00:49.283131+0000","last_unstale":"2024-02-14T15:02:45.767120+0000","last_undegraded":"2024-02-14T15:02:45.767120+0000","last_fullsized":"2024-02-14T15:02:45.767120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:42.247624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706471+0000","last_change":"2024-02-14T15:00:49.287229+0000","last_active":"2024-02-14T15:02:45.706471+0000","last_peered":"2024-02-14T15:02:45.706471+0000","last_clean":"2024-02-14T15:02:45.706471+0000","last_became_active":"2024-02-14T15:00:49.286791+0000","last_became_peered":"2024-02-14T15:00:49.286791+0000","last_unstale":"2024-02-14T15:02:45.706471+0000","last_undegraded":"2024-02-14T15:02:45.706471+0000","last_fullsized":"2024-02-14T15:02:45.706471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:33:01.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767467+0000","last_change":"2024-02-14T15:00:49.281669+0000","last_active":"2024-02-14T15:02:45.767467+0000","last_peered":"2024-02-14T15:02:45.767467+0000","last_clean":"2024-02-14T15:02:45.767467+0000","last_became_active":"2024-02-14T15:00:49.281298+0000","last_became_peered":"2024-02-14T15:00:49.281298+0000","last_unstale":"2024-02-14T15:02:45.767467+0000","last_undegraded":"2024-02-14T15:02:45.767467+0000","last_fullsized":"2024-02-14T15:02:45.767467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:54:40.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617437+0000","last_change":"2024-02-14T15:00:49.289679+0000","last_active":"2024-02-14T15:02:45.617437+0000","last_peered":"2024-02-14T15:02:45.617437+0000","last_clean":"2024-02-14T15:02:45.617437+0000","last_became_active":"2024-02-14T15:00:49.288767+0000","last_became_peered":"2024-02-14T15:00:49.288767+0000","last_unstale":"2024-02-14T15:02:45.617437+0000","last_undegraded":"2024-02-14T15:02:45.617437+0000","last_fullsized":"2024-02-14T15:02:45.617437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:16:20.181517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616867+0000","last_change":"2024-02-14T15:00:49.257471+0000","last_active":"2024-02-14T15:02:45.616867+0000","last_peered":"2024-02-14T15:02:45.616867+0000","last_clean":"2024-02-14T15:02:45.616867+0000","last_became_active":"2024-02-14T15:00:49.256766+0000","last_became_peered":"2024-02-14T15:00:49.256766+0000","last_unstale":"2024-02-14T15:02:45.616867+0000","last_undegraded":"2024-02-14T15:02:45.616867+0000","last_fullsized":"2024-02-14T15:02:45.616867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:14:00.525621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511859+0000","last_change":"2024-02-14T15:00:49.278462+0000","last_active":"2024-02-14T15:02:46.511859+0000","last_peered":"2024-02-14T15:02:46.511859+0000","last_clean":"2024-02-14T15:02:46.511859+0000","last_became_active":"2024-02-14T15:00:49.278114+0000","last_became_peered":"2024-02-14T15:00:49.278114+0000","last_unstale":"2024-02-14T15:02:46.511859+0000","last_undegraded":"2024-02-14T15:02:46.511859+0000","last_fullsized":"2024-02-14T15:02:46.511859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:02:16.601359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511865+0000","last_change":"2024-02-14T15:00:49.270653+0000","last_active":"2024-02-14T15:02:46.511865+0000","last_peered":"2024-02-14T15:02:46.511865+0000","last_clean":"2024-02-14T15:02:46.511865+0000","last_became_active":"2024-02-14T15:00:49.270334+0000","last_became_peered":"2024-02-14T15:00:49.270334+0000","last_unstale":"2024-02-14T15:02:46.511865+0000","last_undegraded":"2024-02-14T15:02:46.511865+0000","last_fullsized":"2024-02-14T15:02:46.511865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:13:53.840909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766103+0000","last_change":"2024-02-14T15:00:49.280597+0000","last_active":"2024-02-14T15:02:45.766103+0000","last_peered":"2024-02-14T15:02:45.766103+0000","last_clean":"2024-02-14T15:02:45.766103+0000","last_became_active":"2024-02-14T15:00:49.279612+0000","last_became_peered":"2024-02-14T15:00:49.279612+0000","last_unstale":"2024-02-14T15:02:45.766103+0000","last_undegraded":"2024-02-14T15:02:45.766103+0000","last_fullsized":"2024-02-14T15:02:45.766103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:11.676608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620327+0000","last_change":"2024-02-14T15:00:49.281422+0000","last_active":"2024-02-14T15:02:45.620327+0000","last_peered":"2024-02-14T15:02:45.620327+0000","last_clean":"2024-02-14T15:02:45.620327+0000","last_became_active":"2024-02-14T15:00:49.281048+0000","last_became_peered":"2024-02-14T15:00:49.281048+0000","last_unstale":"2024-02-14T15:02:45.620327+0000","last_undegraded":"2024-02-14T15:02:45.620327+0000","last_fullsized":"2024-02-14T15:02:45.620327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:04.994390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766686+0000","last_change":"2024-02-14T15:00:49.288245+0000","last_active":"2024-02-14T15:02:45.766686+0000","last_peered":"2024-02-14T15:02:45.766686+0000","last_clean":"2024-02-14T15:02:45.766686+0000","last_became_active":"2024-02-14T15:00:49.287908+0000","last_became_peered":"2024-02-14T15:00:49.287908+0000","last_unstale":"2024-02-14T15:02:45.766686+0000","last_undegraded":"2024-02-14T15:02:45.766686+0000","last_fullsized":"2024-02-14T15:02:45.766686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:52.397826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621696+0000","last_change":"2024-02-14T15:00:49.275428+0000","last_active":"2024-02-14T15:02:45.621696+0000","last_peered":"2024-02-14T15:02:45.621696+0000","last_clean":"2024-02-14T15:02:45.621696+0000","last_became_active":"2024-02-14T15:00:49.273464+0000","last_became_peered":"2024-02-14T15:00:49.273464+0000","last_unstale":"2024-02-14T15:02:45.621696+0000","last_undegraded":"2024-02-14T15:02:45.621696+0000","last_fullsized":"2024-02-14T15:02:45.621696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:34.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621145+0000","last_change":"2024-02-14T15:00:53.587219+0000","last_active":"2024-02-14T15:02:45.621145+0000","last_peered":"2024-02-14T15:02:45.621145+0000","last_clean":"2024-02-14T15:02:45.621145+0000","last_became_active":"2024-02-14T15:00:53.586917+0000","last_became_peered":"2024-02-14T15:00:53.586917+0000","last_unstale":"2024-02-14T15:02:45.621145+0000","last_undegraded":"2024-02-14T15:02:45.621145+0000","last_fullsized":"2024-02-14T15:02:45.621145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:39:38.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766753+0000","last_change":"2024-02-14T15:00:48.269243+0000","last_active":"2024-02-14T15:02:45.766753+0000","last_peered":"2024-02-14T15:02:45.766753+0000","last_clean":"2024-02-14T15:02:45.766753+0000","last_became_active":"2024-02-14T15:00:48.268883+0000","last_became_peered":"2024-02-14T15:00:48.268883+0000","last_unstale":"2024-02-14T15:02:45.766753+0000","last_undegraded":"2024-02-14T15:02:45.766753+0000","last_fullsized":"2024-02-14T15:02:45.766753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:08.530092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713654+0000","last_change":"2024-02-14T15:00:49.282458+0000","last_active":"2024-02-14T15:02:45.713654+0000","last_peered":"2024-02-14T15:02:45.713654+0000","last_clean":"2024-02-14T15:02:45.713654+0000","last_became_active":"2024-02-14T15:00:49.281982+0000","last_became_peered":"2024-02-14T15:00:49.281982+0000","last_unstale":"2024-02-14T15:02:45.713654+0000","last_undegraded":"2024-02-14T15:02:45.713654+0000","last_fullsized":"2024-02-14T15:02:45.713654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:51:33.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617171+0000","last_change":"2024-02-14T15:00:53.586578+0000","last_active":"2024-02-14T15:02:45.617171+0000","last_peered":"2024-02-14T15:02:45.617171+0000","last_clean":"2024-02-14T15:02:45.617171+0000","last_became_active":"2024-02-14T15:00:53.586109+0000","last_became_peered":"2024-02-14T15:00:53.586109+0000","last_unstale":"2024-02-14T15:02:45.617171+0000","last_undegraded":"2024-02-14T15:02:45.617171+0000","last_fullsized":"2024-02-14T15:02:45.617171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:07.332435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713729+0000","last_change":"2024-02-14T15:00:48.281953+0000","last_active":"2024-02-14T15:02:45.713729+0000","last_peered":"2024-02-14T15:02:45.713729+0000","last_clean":"2024-02-14T15:02:45.713729+0000","last_became_active":"2024-02-14T15:00:48.281590+0000","last_became_peered":"2024-02-14T15:00:48.281590+0000","last_unstale":"2024-02-14T15:02:45.713729+0000","last_undegraded":"2024-02-14T15:02:45.713729+0000","last_fullsized":"2024-02-14T15:02:45.713729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:22.991037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512389+0000","last_change":"2024-02-14T15:00:49.284729+0000","last_active":"2024-02-14T15:02:46.512389+0000","last_peered":"2024-02-14T15:02:46.512389+0000","last_clean":"2024-02-14T15:02:46.512389+0000","last_became_active":"2024-02-14T15:00:49.284013+0000","last_became_peered":"2024-02-14T15:00:49.284013+0000","last_unstale":"2024-02-14T15:02:46.512389+0000","last_undegraded":"2024-02-14T15:02:46.512389+0000","last_fullsized":"2024-02-14T15:02:46.512389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:03:31.313032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617441+0000","last_change":"2024-02-14T15:00:53.589825+0000","last_active":"2024-02-14T15:02:45.617441+0000","last_peered":"2024-02-14T15:02:45.617441+0000","last_clean":"2024-02-14T15:02:45.617441+0000","last_became_active":"2024-02-14T15:00:53.589424+0000","last_became_peered":"2024-02-14T15:00:53.589424+0000","last_unstale":"2024-02-14T15:02:45.617441+0000","last_undegraded":"2024-02-14T15:02:45.617441+0000","last_fullsized":"2024-02-14T15:02:45.617441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:24.522152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767437+0000","last_change":"2024-02-14T15:00:48.276884+0000","last_active":"2024-02-14T15:02:45.767437+0000","last_peered":"2024-02-14T15:02:45.767437+0000","last_clean":"2024-02-14T15:02:45.767437+0000","last_became_active":"2024-02-14T15:00:48.276576+0000","last_became_peered":"2024-02-14T15:00:48.276576+0000","last_unstale":"2024-02-14T15:02:45.767437+0000","last_undegraded":"2024-02-14T15:02:45.767437+0000","last_fullsized":"2024-02-14T15:02:45.767437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:25.793368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766543+0000","last_change":"2024-02-14T15:00:49.277602+0000","last_active":"2024-02-14T15:02:45.766543+0000","last_peered":"2024-02-14T15:02:45.766543+0000","last_clean":"2024-02-14T15:02:45.766543+0000","last_became_active":"2024-02-14T15:00:49.277249+0000","last_became_peered":"2024-02-14T15:00:49.277249+0000","last_unstale":"2024-02-14T15:02:45.766543+0000","last_undegraded":"2024-02-14T15:02:45.766543+0000","last_fullsized":"2024-02-14T15:02:45.766543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.347962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617245+0000","last_change":"2024-02-14T15:00:53.587838+0000","last_active":"2024-02-14T15:02:45.617245+0000","last_peered":"2024-02-14T15:02:45.617245+0000","last_clean":"2024-02-14T15:02:45.617245+0000","last_became_active":"2024-02-14T15:00:53.587012+0000","last_became_peered":"2024-02-14T15:00:53.587012+0000","last_unstale":"2024-02-14T15:02:45.617245+0000","last_undegraded":"2024-02-14T15:02:45.617245+0000","last_fullsized":"2024-02-14T15:02:45.617245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:32:12.276578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767139+0000","last_change":"2024-02-14T15:00:48.273529+0000","last_active":"2024-02-14T15:02:45.767139+0000","last_peered":"2024-02-14T15:02:45.767139+0000","last_clean":"2024-02-14T15:02:45.767139+0000","last_became_active":"2024-02-14T15:00:48.272555+0000","last_became_peered":"2024-02-14T15:00:48.272555+0000","last_unstale":"2024-02-14T15:02:45.767139+0000","last_undegraded":"2024-02-14T15:02:45.767139+0000","last_fullsized":"2024-02-14T15:02:45.767139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:19.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618504+0000","last_change":"2024-02-14T15:00:49.278834+0000","last_active":"2024-02-14T15:02:45.618504+0000","last_peered":"2024-02-14T15:02:45.618504+0000","last_clean":"2024-02-14T15:02:45.618504+0000","last_became_active":"2024-02-14T15:00:49.277095+0000","last_became_peered":"2024-02-14T15:00:49.277095+0000","last_unstale":"2024-02-14T15:02:45.618504+0000","last_undegraded":"2024-02-14T15:02:45.618504+0000","last_fullsized":"2024-02-14T15:02:45.618504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:27.291974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.768818+0000","last_change":"2024-02-14T15:02:45.768818+0000","last_active":"2024-02-14T15:02:45.768818+0000","last_peered":"2024-02-14T15:02:45.768818+0000","last_clean":"2024-02-14T15:02:45.768818+0000","last_became_active":"2024-02-14T15:02:45.768283+0000","last_became_peered":"2024-02-14T15:02:45.768283+0000","last_unstale":"2024-02-14T15:02:45.768818+0000","last_undegraded":"2024-02-14T15:02:45.768818+0000","last_fullsized":"2024-02-14T15:02:45.768818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:59:14.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.743886+0000","last_change":"2024-02-14T15:00:48.276146+0000","last_active":"2024-02-14T15:03:09.743886+0000","last_peered":"2024-02-14T15:03:09.743886+0000","last_clean":"2024-02-14T15:03:09.743886+0000","last_became_active":"2024-02-14T15:00:48.275833+0000","last_became_peered":"2024-02-14T15:00:48.275833+0000","last_unstale":"2024-02-14T15:03:09.743886+0000","last_undegraded":"2024-02-14T15:03:09.743886+0000","last_fullsized":"2024-02-14T15:03:09.743886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:46.649486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766980+0000","last_change":"2024-02-14T15:00:49.267285+0000","last_active":"2024-02-14T15:02:45.766980+0000","last_peered":"2024-02-14T15:02:45.766980+0000","last_clean":"2024-02-14T15:02:45.766980+0000","last_became_active":"2024-02-14T15:00:49.265592+0000","last_became_peered":"2024-02-14T15:00:49.265592+0000","last_unstale":"2024-02-14T15:02:45.766980+0000","last_undegraded":"2024-02-14T15:02:45.766980+0000","last_fullsized":"2024-02-14T15:02:45.766980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:18.469835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621465+0000","last_change":"2024-02-14T15:00:53.582085+0000","last_active":"2024-02-14T15:02:45.621465+0000","last_peered":"2024-02-14T15:02:45.621465+0000","last_clean":"2024-02-14T15:02:45.621465+0000","last_became_active":"2024-02-14T15:00:53.581706+0000","last_became_peered":"2024-02-14T15:00:53.581706+0000","last_unstale":"2024-02-14T15:02:45.621465+0000","last_undegraded":"2024-02-14T15:02:45.621465+0000","last_fullsized":"2024-02-14T15:02:45.621465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:32.481142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.705924+0000","last_change":"2024-02-14T15:00:48.268975+0000","last_active":"2024-02-14T15:02:45.705924+0000","last_peered":"2024-02-14T15:02:45.705924+0000","last_clean":"2024-02-14T15:02:45.705924+0000","last_became_active":"2024-02-14T15:00:48.268547+0000","last_became_peered":"2024-02-14T15:00:48.268547+0000","last_unstale":"2024-02-14T15:02:45.705924+0000","last_undegraded":"2024-02-14T15:02:45.705924+0000","last_fullsized":"2024-02-14T15:02:45.705924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:13.937917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618357+0000","last_change":"2024-02-14T15:00:49.276909+0000","last_active":"2024-02-14T15:02:45.618357+0000","last_peered":"2024-02-14T15:02:45.618357+0000","last_clean":"2024-02-14T15:02:45.618357+0000","last_became_active":"2024-02-14T15:00:49.276533+0000","last_became_peered":"2024-02-14T15:00:49.276533+0000","last_unstale":"2024-02-14T15:02:45.618357+0000","last_undegraded":"2024-02-14T15:02:45.618357+0000","last_fullsized":"2024-02-14T15:02:45.618357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:35:31.321455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620174+0000","last_change":"2024-02-14T15:00:53.592223+0000","last_active":"2024-02-14T15:02:45.620174+0000","last_peered":"2024-02-14T15:02:45.620174+0000","last_clean":"2024-02-14T15:02:45.620174+0000","last_became_active":"2024-02-14T15:00:53.591883+0000","last_became_peered":"2024-02-14T15:00:53.591883+0000","last_unstale":"2024-02-14T15:02:45.620174+0000","last_undegraded":"2024-02-14T15:02:45.620174+0000","last_fullsized":"2024-02-14T15:02:45.620174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:10.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616486+0000","last_change":"2024-02-14T15:00:48.274485+0000","last_active":"2024-02-14T15:02:45.616486+0000","last_peered":"2024-02-14T15:02:45.616486+0000","last_clean":"2024-02-14T15:02:45.616486+0000","last_became_active":"2024-02-14T15:00:48.274160+0000","last_became_peered":"2024-02-14T15:00:48.274160+0000","last_unstale":"2024-02-14T15:02:45.616486+0000","last_undegraded":"2024-02-14T15:02:45.616486+0000","last_fullsized":"2024-02-14T15:02:45.616486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:00:47.105069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713108+0000","last_change":"2024-02-14T15:00:49.272577+0000","last_active":"2024-02-14T15:02:45.713108+0000","last_peered":"2024-02-14T15:02:45.713108+0000","last_clean":"2024-02-14T15:02:45.713108+0000","last_became_active":"2024-02-14T15:00:49.271670+0000","last_became_peered":"2024-02-14T15:00:49.271670+0000","last_unstale":"2024-02-14T15:02:45.713108+0000","last_undegraded":"2024-02-14T15:02:45.713108+0000","last_fullsized":"2024-02-14T15:02:45.713108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:19.968389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512199+0000","last_change":"2024-02-14T15:00:53.581212+0000","last_active":"2024-02-14T15:02:46.512199+0000","last_peered":"2024-02-14T15:02:46.512199+0000","last_clean":"2024-02-14T15:02:46.512199+0000","last_became_active":"2024-02-14T15:00:53.580782+0000","last_became_peered":"2024-02-14T15:00:53.580782+0000","last_unstale":"2024-02-14T15:02:46.512199+0000","last_undegraded":"2024-02-14T15:02:46.512199+0000","last_fullsized":"2024-02-14T15:02:46.512199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:14.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617597+0000","last_change":"2024-02-14T15:00:48.273572+0000","last_active":"2024-02-14T15:02:45.617597+0000","last_peered":"2024-02-14T15:02:45.617597+0000","last_clean":"2024-02-14T15:02:45.617597+0000","last_became_active":"2024-02-14T15:00:48.273138+0000","last_became_peered":"2024-02-14T15:00:48.273138+0000","last_unstale":"2024-02-14T15:02:45.617597+0000","last_undegraded":"2024-02-14T15:02:45.617597+0000","last_fullsized":"2024-02-14T15:02:45.617597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:04.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766140+0000","last_change":"2024-02-14T15:00:49.288226+0000","last_active":"2024-02-14T15:02:45.766140+0000","last_peered":"2024-02-14T15:02:45.766140+0000","last_clean":"2024-02-14T15:02:45.766140+0000","last_became_active":"2024-02-14T15:00:49.287874+0000","last_became_peered":"2024-02-14T15:00:49.287874+0000","last_unstale":"2024-02-14T15:02:45.766140+0000","last_undegraded":"2024-02-14T15:02:45.766140+0000","last_fullsized":"2024-02-14T15:02:45.766140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:24:02.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713194+0000","last_change":"2024-02-14T15:00:53.590543+0000","last_active":"2024-02-14T15:02:45.713194+0000","last_peered":"2024-02-14T15:02:45.713194+0000","last_clean":"2024-02-14T15:02:45.713194+0000","last_became_active":"2024-02-14T15:00:53.589072+0000","last_became_peered":"2024-02-14T15:00:53.589072+0000","last_unstale":"2024-02-14T15:02:45.713194+0000","last_undegraded":"2024-02-14T15:02:45.713194+0000","last_fullsized":"2024-02-14T15:02:45.713194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:13.685655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":146,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:06.739070+0000","last_change":"2024-02-14T15:00:48.272027+0000","last_active":"2024-02-14T15:03:06.739070+0000","last_peered":"2024-02-14T15:03:06.739070+0000","last_clean":"2024-02-14T15:03:06.739070+0000","last_became_active":"2024-02-14T15:00:48.271587+0000","last_became_peered":"2024-02-14T15:00:48.271587+0000","last_unstale":"2024-02-14T15:03:06.739070+0000","last_undegraded":"2024-02-14T15:03:06.739070+0000","last_fullsized":"2024-02-14T15:03:06.739070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:33:03.566189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766999+0000","last_change":"2024-02-14T15:00:49.278936+0000","last_active":"2024-02-14T15:02:45.766999+0000","last_peered":"2024-02-14T15:02:45.766999+0000","last_clean":"2024-02-14T15:02:45.766999+0000","last_became_active":"2024-02-14T15:00:49.278318+0000","last_became_peered":"2024-02-14T15:00:49.278318+0000","last_unstale":"2024-02-14T15:02:45.766999+0000","last_undegraded":"2024-02-14T15:02:45.766999+0000","last_fullsized":"2024-02-14T15:02:45.766999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:01.327303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766017+0000","last_change":"2024-02-14T15:00:53.580340+0000","last_active":"2024-02-14T15:02:45.766017+0000","last_peered":"2024-02-14T15:02:45.766017+0000","last_clean":"2024-02-14T15:02:45.766017+0000","last_became_active":"2024-02-14T15:00:53.579742+0000","last_became_peered":"2024-02-14T15:00:53.579742+0000","last_unstale":"2024-02-14T15:02:45.766017+0000","last_undegraded":"2024-02-14T15:02:45.766017+0000","last_fullsized":"2024-02-14T15:02:45.766017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:38:33.182328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767303+0000","last_change":"2024-02-14T15:00:48.271891+0000","last_active":"2024-02-14T15:02:45.767303+0000","last_peered":"2024-02-14T15:02:45.767303+0000","last_clean":"2024-02-14T15:02:45.767303+0000","last_became_active":"2024-02-14T15:00:48.271419+0000","last_became_peered":"2024-02-14T15:00:48.271419+0000","last_unstale":"2024-02-14T15:02:45.767303+0000","last_undegraded":"2024-02-14T15:02:45.767303+0000","last_fullsized":"2024-02-14T15:02:45.767303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:03:04.725647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512325+0000","last_change":"2024-02-14T15:00:49.277025+0000","last_active":"2024-02-14T15:02:46.512325+0000","last_peered":"2024-02-14T15:02:46.512325+0000","last_clean":"2024-02-14T15:02:46.512325+0000","last_became_active":"2024-02-14T15:00:49.276694+0000","last_became_peered":"2024-02-14T15:00:49.276694+0000","last_unstale":"2024-02-14T15:02:46.512325+0000","last_undegraded":"2024-02-14T15:02:46.512325+0000","last_fullsized":"2024-02-14T15:02:46.512325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:43:07.249407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617568+0000","last_change":"2024-02-14T15:01:44.607053+0000","last_active":"2024-02-14T15:02:45.617568+0000","last_peered":"2024-02-14T15:02:45.617568+0000","last_clean":"2024-02-14T15:02:45.617568+0000","last_became_active":"2024-02-14T15:01:44.606596+0000","last_became_peered":"2024-02-14T15:01:44.606596+0000","last_unstale":"2024-02-14T15:02:45.617568+0000","last_undegraded":"2024-02-14T15:02:45.617568+0000","last_fullsized":"2024-02-14T15:02:45.617568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:35.949953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618389+0000","last_change":"2024-02-14T15:00:48.272866+0000","last_active":"2024-02-14T15:02:45.618389+0000","last_peered":"2024-02-14T15:02:45.618389+0000","last_clean":"2024-02-14T15:02:45.618389+0000","last_became_active":"2024-02-14T15:00:48.272520+0000","last_became_peered":"2024-02-14T15:00:48.272520+0000","last_unstale":"2024-02-14T15:02:45.618389+0000","last_undegraded":"2024-02-14T15:02:45.618389+0000","last_fullsized":"2024-02-14T15:02:45.618389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:39.434577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:49.280994+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:49.280205+0000","last_became_peered":"2024-02-14T15:00:49.280205+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:53:48.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766150+0000","last_change":"2024-02-14T15:00:53.582599+0000","last_active":"2024-02-14T15:02:45.766150+0000","last_peered":"2024-02-14T15:02:45.766150+0000","last_clean":"2024-02-14T15:02:45.766150+0000","last_became_active":"2024-02-14T15:00:53.582231+0000","last_became_peered":"2024-02-14T15:00:53.582231+0000","last_unstale":"2024-02-14T15:02:45.766150+0000","last_undegraded":"2024-02-14T15:02:45.766150+0000","last_fullsized":"2024-02-14T15:02:45.766150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:19.126264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":319,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:07.740623+0000","last_change":"2024-02-14T15:01:44.615393+0000","last_active":"2024-02-14T15:03:07.740623+0000","last_peered":"2024-02-14T15:03:07.740623+0000","last_clean":"2024-02-14T15:03:07.740623+0000","last_became_active":"2024-02-14T15:01:44.604294+0000","last_became_peered":"2024-02-14T15:01:44.604294+0000","last_unstale":"2024-02-14T15:03:07.740623+0000","last_undegraded":"2024-02-14T15:03:07.740623+0000","last_fullsized":"2024-02-14T15:03:07.740623+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:00.459860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":180,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767208+0000","last_change":"2024-02-14T15:00:49.270144+0000","last_active":"2024-02-14T15:02:45.767208+0000","last_peered":"2024-02-14T15:02:45.767208+0000","last_clean":"2024-02-14T15:02:45.767208+0000","last_became_active":"2024-02-14T15:00:49.269806+0000","last_became_peered":"2024-02-14T15:00:49.269806+0000","last_unstale":"2024-02-14T15:02:45.767208+0000","last_undegraded":"2024-02-14T15:02:45.767208+0000","last_fullsized":"2024-02-14T15:02:45.767208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:42:18.559226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618862+0000","last_change":"2024-02-14T15:00:53.582755+0000","last_active":"2024-02-14T15:02:45.618862+0000","last_peered":"2024-02-14T15:02:45.618862+0000","last_clean":"2024-02-14T15:02:45.618862+0000","last_became_active":"2024-02-14T15:00:53.582375+0000","last_became_peered":"2024-02-14T15:00:53.582375+0000","last_unstale":"2024-02-14T15:02:45.618862+0000","last_undegraded":"2024-02-14T15:02:45.618862+0000","last_fullsized":"2024-02-14T15:02:45.618862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:11:49.115136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621377+0000","last_change":"2024-02-14T15:00:48.268982+0000","last_active":"2024-02-14T15:02:45.621377+0000","last_peered":"2024-02-14T15:02:45.621377+0000","last_clean":"2024-02-14T15:02:45.621377+0000","last_became_active":"2024-02-14T15:00:48.267137+0000","last_became_peered":"2024-02-14T15:00:48.267137+0000","last_unstale":"2024-02-14T15:02:45.621377+0000","last_undegraded":"2024-02-14T15:02:45.621377+0000","last_fullsized":"2024-02-14T15:02:45.621377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:44.351561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616587+0000","last_change":"2024-02-14T15:00:49.267493+0000","last_active":"2024-02-14T15:02:45.616587+0000","last_peered":"2024-02-14T15:02:45.616587+0000","last_clean":"2024-02-14T15:02:45.616587+0000","last_became_active":"2024-02-14T15:00:49.267161+0000","last_became_peered":"2024-02-14T15:00:49.267161+0000","last_unstale":"2024-02-14T15:02:45.616587+0000","last_undegraded":"2024-02-14T15:02:45.616587+0000","last_fullsized":"2024-02-14T15:02:45.616587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:35:22.997999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.768993+0000","last_change":"2024-02-14T15:02:45.768993+0000","last_active":"2024-02-14T15:02:45.768993+0000","last_peered":"2024-02-14T15:02:45.768993+0000","last_clean":"2024-02-14T15:02:45.768993+0000","last_became_active":"2024-02-14T15:02:45.768413+0000","last_became_peered":"2024-02-14T15:02:45.768413+0000","last_unstale":"2024-02-14T15:02:45.768993+0000","last_undegraded":"2024-02-14T15:02:45.768993+0000","last_fullsized":"2024-02-14T15:02:45.768993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:45:54.193127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618346+0000","last_change":"2024-02-14T15:00:48.282085+0000","last_active":"2024-02-14T15:02:45.618346+0000","last_peered":"2024-02-14T15:02:45.618346+0000","last_clean":"2024-02-14T15:02:45.618346+0000","last_became_active":"2024-02-14T15:00:48.281836+0000","last_became_peered":"2024-02-14T15:00:48.281836+0000","last_unstale":"2024-02-14T15:02:45.618346+0000","last_undegraded":"2024-02-14T15:02:45.618346+0000","last_fullsized":"2024-02-14T15:02:45.618346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:07:28.145074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767345+0000","last_change":"2024-02-14T15:00:49.280013+0000","last_active":"2024-02-14T15:02:45.767345+0000","last_peered":"2024-02-14T15:02:45.767345+0000","last_clean":"2024-02-14T15:02:45.767345+0000","last_became_active":"2024-02-14T15:00:49.279667+0000","last_became_peered":"2024-02-14T15:00:49.279667+0000","last_unstale":"2024-02-14T15:02:45.767345+0000","last_undegraded":"2024-02-14T15:02:45.767345+0000","last_fullsized":"2024-02-14T15:02:45.767345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:03:16.559295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766612+0000","last_change":"2024-02-14T15:00:53.580269+0000","last_active":"2024-02-14T15:02:45.766612+0000","last_peered":"2024-02-14T15:02:45.766612+0000","last_clean":"2024-02-14T15:02:45.766612+0000","last_became_active":"2024-02-14T15:00:53.579863+0000","last_became_peered":"2024-02-14T15:00:53.579863+0000","last_unstale":"2024-02-14T15:02:45.766612+0000","last_undegraded":"2024-02-14T15:02:45.766612+0000","last_fullsized":"2024-02-14T15:02:45.766612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:37:19.275654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.744250+0000","last_change":"2024-02-14T15:00:48.267310+0000","last_active":"2024-02-14T15:03:09.744250+0000","last_peered":"2024-02-14T15:03:09.744250+0000","last_clean":"2024-02-14T15:03:09.744250+0000","last_became_active":"2024-02-14T15:00:48.266864+0000","last_became_peered":"2024-02-14T15:00:48.266864+0000","last_unstale":"2024-02-14T15:03:09.744250+0000","last_undegraded":"2024-02-14T15:03:09.744250+0000","last_fullsized":"2024-02-14T15:03:09.744250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:07.703549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:49.288568+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:49.288222+0000","last_became_peered":"2024-02-14T15:00:49.288222+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:38:38.909769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616994+0000","last_change":"2024-02-14T15:00:53.585673+0000","last_active":"2024-02-14T15:02:45.616994+0000","last_peered":"2024-02-14T15:02:45.616994+0000","last_clean":"2024-02-14T15:02:45.616994+0000","last_became_active":"2024-02-14T15:00:53.585238+0000","last_became_peered":"2024-02-14T15:00:53.585238+0000","last_unstale":"2024-02-14T15:02:45.616994+0000","last_undegraded":"2024-02-14T15:02:45.616994+0000","last_fullsized":"2024-02-14T15:02:45.616994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:48:11.845921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:48.275833+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:48.275499+0000","last_became_peered":"2024-02-14T15:00:48.275499+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:35:10.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618303+0000","last_change":"2024-02-14T15:01:44.605394+0000","last_active":"2024-02-14T15:02:45.618303+0000","last_peered":"2024-02-14T15:02:45.618303+0000","last_clean":"2024-02-14T15:02:45.618303+0000","last_became_active":"2024-02-14T15:01:44.604874+0000","last_became_peered":"2024-02-14T15:01:44.604874+0000","last_unstale":"2024-02-14T15:02:45.618303+0000","last_undegraded":"2024-02-14T15:02:45.618303+0000","last_fullsized":"2024-02-14T15:02:45.618303+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:41:47.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618214+0000","last_change":"2024-02-14T15:00:53.586406+0000","last_active":"2024-02-14T15:02:45.618214+0000","last_peered":"2024-02-14T15:02:45.618214+0000","last_clean":"2024-02-14T15:02:45.618214+0000","last_became_active":"2024-02-14T15:00:53.586034+0000","last_became_peered":"2024-02-14T15:00:53.586034+0000","last_unstale":"2024-02-14T15:02:45.618214+0000","last_undegraded":"2024-02-14T15:02:45.618214+0000","last_fullsized":"2024-02-14T15:02:45.618214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:46.886154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767472+0000","last_change":"2024-02-14T15:00:48.274305+0000","last_active":"2024-02-14T15:02:45.767472+0000","last_peered":"2024-02-14T15:02:45.767472+0000","last_clean":"2024-02-14T15:02:45.767472+0000","last_became_active":"2024-02-14T15:00:48.273966+0000","last_became_peered":"2024-02-14T15:00:48.273966+0000","last_unstale":"2024-02-14T15:02:45.767472+0000","last_undegraded":"2024-02-14T15:02:45.767472+0000","last_fullsized":"2024-02-14T15:02:45.767472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:49.198099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620493+0000","last_change":"2024-02-14T15:00:49.278473+0000","last_active":"2024-02-14T15:02:45.620493+0000","last_peered":"2024-02-14T15:02:45.620493+0000","last_clean":"2024-02-14T15:02:45.620493+0000","last_became_active":"2024-02-14T15:00:49.278145+0000","last_became_peered":"2024-02-14T15:00:49.278145+0000","last_unstale":"2024-02-14T15:02:45.620493+0000","last_undegraded":"2024-02-14T15:02:45.620493+0000","last_fullsized":"2024-02-14T15:02:45.620493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:49:11.718035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767006+0000","last_change":"2024-02-14T15:00:53.581164+0000","last_active":"2024-02-14T15:02:45.767006+0000","last_peered":"2024-02-14T15:02:45.767006+0000","last_clean":"2024-02-14T15:02:45.767006+0000","last_became_active":"2024-02-14T15:00:53.580796+0000","last_became_peered":"2024-02-14T15:00:53.580796+0000","last_unstale":"2024-02-14T15:02:45.767006+0000","last_undegraded":"2024-02-14T15:02:45.767006+0000","last_fullsized":"2024-02-14T15:02:45.767006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:37.245035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766486+0000","last_change":"2024-02-14T15:00:48.271427+0000","last_active":"2024-02-14T15:02:45.766486+0000","last_peered":"2024-02-14T15:02:45.766486+0000","last_clean":"2024-02-14T15:02:45.766486+0000","last_became_active":"2024-02-14T15:00:48.271115+0000","last_became_peered":"2024-02-14T15:00:48.271115+0000","last_unstale":"2024-02-14T15:02:45.766486+0000","last_undegraded":"2024-02-14T15:02:45.766486+0000","last_fullsized":"2024-02-14T15:02:45.766486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:56:01.008058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706430+0000","last_change":"2024-02-14T15:00:49.269964+0000","last_active":"2024-02-14T15:02:45.706430+0000","last_peered":"2024-02-14T15:02:45.706430+0000","last_clean":"2024-02-14T15:02:45.706430+0000","last_became_active":"2024-02-14T15:00:49.269628+0000","last_became_peered":"2024-02-14T15:00:49.269628+0000","last_unstale":"2024-02-14T15:02:45.706430+0000","last_undegraded":"2024-02-14T15:02:45.706430+0000","last_fullsized":"2024-02-14T15:02:45.706430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:04:26.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616880+0000","last_change":"2024-02-14T15:00:53.579843+0000","last_active":"2024-02-14T15:02:45.616880+0000","last_peered":"2024-02-14T15:02:45.616880+0000","last_clean":"2024-02-14T15:02:45.616880+0000","last_became_active":"2024-02-14T15:00:53.579457+0000","last_became_peered":"2024-02-14T15:00:53.579457+0000","last_unstale":"2024-02-14T15:02:45.616880+0000","last_undegraded":"2024-02-14T15:02:45.616880+0000","last_fullsized":"2024-02-14T15:02:45.616880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:34:49.438317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767597+0000","last_change":"2024-02-14T15:00:48.275016+0000","last_active":"2024-02-14T15:02:45.767597+0000","last_peered":"2024-02-14T15:02:45.767597+0000","last_clean":"2024-02-14T15:02:45.767597+0000","last_became_active":"2024-02-14T15:00:48.274636+0000","last_became_peered":"2024-02-14T15:00:48.274636+0000","last_unstale":"2024-02-14T15:02:45.767597+0000","last_undegraded":"2024-02-14T15:02:45.767597+0000","last_fullsized":"2024-02-14T15:02:45.767597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:12:12.961276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706105+0000","last_change":"2024-02-14T15:01:44.602904+0000","last_active":"2024-02-14T15:02:45.706105+0000","last_peered":"2024-02-14T15:02:45.706105+0000","last_clean":"2024-02-14T15:02:45.706105+0000","last_became_active":"2024-02-14T15:01:44.602480+0000","last_became_peered":"2024-02-14T15:01:44.602480+0000","last_unstale":"2024-02-14T15:02:45.706105+0000","last_undegraded":"2024-02-14T15:02:45.706105+0000","last_fullsized":"2024-02-14T15:02:45.706105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:27:33.451581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616988+0000","last_change":"2024-02-14T15:00:53.584068+0000","last_active":"2024-02-14T15:02:45.616988+0000","last_peered":"2024-02-14T15:02:45.616988+0000","last_clean":"2024-02-14T15:02:45.616988+0000","last_became_active":"2024-02-14T15:00:53.583263+0000","last_became_peered":"2024-02-14T15:00:53.583263+0000","last_unstale":"2024-02-14T15:02:45.616988+0000","last_undegraded":"2024-02-14T15:02:45.616988+0000","last_fullsized":"2024-02-14T15:02:45.616988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:15.726442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617918+0000","last_change":"2024-02-14T15:00:49.271265+0000","last_active":"2024-02-14T15:02:45.617918+0000","last_peered":"2024-02-14T15:02:45.617918+0000","last_clean":"2024-02-14T15:02:45.617918+0000","last_became_active":"2024-02-14T15:00:49.270931+0000","last_became_peered":"2024-02-14T15:00:49.270931+0000","last_unstale":"2024-02-14T15:02:45.617918+0000","last_undegraded":"2024-02-14T15:02:45.617918+0000","last_fullsized":"2024-02-14T15:02:45.617918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:15:40.570730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.743769+0000","last_change":"2024-02-14T15:00:48.286669+0000","last_active":"2024-02-14T15:03:09.743769+0000","last_peered":"2024-02-14T15:03:09.743769+0000","last_clean":"2024-02-14T15:03:09.743769+0000","last_became_active":"2024-02-14T15:00:48.286252+0000","last_became_peered":"2024-02-14T15:00:48.286252+0000","last_unstale":"2024-02-14T15:03:09.743769+0000","last_undegraded":"2024-02-14T15:03:09.743769+0000","last_fullsized":"2024-02-14T15:03:09.743769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:40.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767025+0000","last_change":"2024-02-14T15:00:53.583353+0000","last_active":"2024-02-14T15:02:45.767025+0000","last_peered":"2024-02-14T15:02:45.767025+0000","last_clean":"2024-02-14T15:02:45.767025+0000","last_became_active":"2024-02-14T15:00:53.582449+0000","last_became_peered":"2024-02-14T15:00:53.582449+0000","last_unstale":"2024-02-14T15:02:45.767025+0000","last_undegraded":"2024-02-14T15:02:45.767025+0000","last_fullsized":"2024-02-14T15:02:45.767025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:09:01.514603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713191+0000","last_change":"2024-02-14T15:00:49.285579+0000","last_active":"2024-02-14T15:02:45.713191+0000","last_peered":"2024-02-14T15:02:45.713191+0000","last_clean":"2024-02-14T15:02:45.713191+0000","last_became_active":"2024-02-14T15:00:49.283741+0000","last_became_peered":"2024-02-14T15:00:49.283741+0000","last_unstale":"2024-02-14T15:02:45.713191+0000","last_undegraded":"2024-02-14T15:02:45.713191+0000","last_fullsized":"2024-02-14T15:02:45.713191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:05.813218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617501+0000","last_change":"2024-02-14T15:00:48.264434+0000","last_active":"2024-02-14T15:02:45.617501+0000","last_peered":"2024-02-14T15:02:45.617501+0000","last_clean":"2024-02-14T15:02:45.617501+0000","last_became_active":"2024-02-14T15:00:48.264048+0000","last_became_peered":"2024-02-14T15:00:48.264048+0000","last_unstale":"2024-02-14T15:02:45.617501+0000","last_undegraded":"2024-02-14T15:02:45.617501+0000","last_fullsized":"2024-02-14T15:02:45.617501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:52:32.498128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766621+0000","last_change":"2024-02-14T14:59:36.539524+0000","last_active":"2024-02-14T15:02:45.766621+0000","last_peered":"2024-02-14T15:02:45.766621+0000","last_clean":"2024-02-14T15:02:45.766621+0000","last_became_active":"2024-02-14T14:59:36.538922+0000","last_became_peered":"2024-02-14T14:59:36.538922+0000","last_unstale":"2024-02-14T15:02:45.766621+0000","last_undegraded":"2024-02-14T15:02:45.766621+0000","last_fullsized":"2024-02-14T15:02:45.766621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_clean_scrub_stamp":"2024-02-14T14:56:09.567049+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:58.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.707765+0000","last_change":"2024-02-14T15:02:45.707287+0000","last_active":"2024-02-14T15:02:45.707765+0000","last_peered":"2024-02-14T15:02:45.707765+0000","last_clean":"2024-02-14T15:02:45.707765+0000","last_became_active":"2024-02-14T15:02:45.706798+0000","last_became_peered":"2024-02-14T15:02:45.706798+0000","last_unstale":"2024-02-14T15:02:45.707765+0000","last_undegraded":"2024-02-14T15:02:45.707765+0000","last_fullsized":"2024-02-14T15:02:45.707765+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:12.237714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616413+0000","last_change":"2024-02-14T15:00:49.272692+0000","last_active":"2024-02-14T15:02:45.616413+0000","last_peered":"2024-02-14T15:02:45.616413+0000","last_clean":"2024-02-14T15:02:45.616413+0000","last_became_active":"2024-02-14T15:00:49.272353+0000","last_became_peered":"2024-02-14T15:00:49.272353+0000","last_unstale":"2024-02-14T15:02:45.616413+0000","last_undegraded":"2024-02-14T15:02:45.616413+0000","last_fullsized":"2024-02-14T15:02:45.616413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:47.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766756+0000","last_change":"2024-02-14T15:00:48.272470+0000","last_active":"2024-02-14T15:02:45.766756+0000","last_peered":"2024-02-14T15:02:45.766756+0000","last_clean":"2024-02-14T15:02:45.766756+0000","last_became_active":"2024-02-14T15:00:48.272126+0000","last_became_peered":"2024-02-14T15:00:48.272126+0000","last_unstale":"2024-02-14T15:02:45.766756+0000","last_undegraded":"2024-02-14T15:02:45.766756+0000","last_fullsized":"2024-02-14T15:02:45.766756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:21:02.601411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620193+0000","last_change":"2024-02-14T15:00:53.585950+0000","last_active":"2024-02-14T15:02:45.620193+0000","last_peered":"2024-02-14T15:02:45.620193+0000","last_clean":"2024-02-14T15:02:45.620193+0000","last_became_active":"2024-02-14T15:00:53.585547+0000","last_became_peered":"2024-02-14T15:00:53.585547+0000","last_unstale":"2024-02-14T15:02:45.620193+0000","last_undegraded":"2024-02-14T15:02:45.620193+0000","last_fullsized":"2024-02-14T15:02:45.620193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:22.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767371+0000","last_change":"2024-02-14T15:00:48.274650+0000","last_active":"2024-02-14T15:02:45.767371+0000","last_peered":"2024-02-14T15:02:45.767371+0000","last_clean":"2024-02-14T15:02:45.767371+0000","last_became_active":"2024-02-14T15:00:48.274085+0000","last_became_peered":"2024-02-14T15:00:48.274085+0000","last_unstale":"2024-02-14T15:02:45.767371+0000","last_undegraded":"2024-02-14T15:02:45.767371+0000","last_fullsized":"2024-02-14T15:02:45.767371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:53.307596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617022+0000","last_change":"2024-02-14T15:00:49.268419+0000","last_active":"2024-02-14T15:02:45.617022+0000","last_peered":"2024-02-14T15:02:45.617022+0000","last_clean":"2024-02-14T15:02:45.617022+0000","last_became_active":"2024-02-14T15:00:49.268120+0000","last_became_peered":"2024-02-14T15:00:49.268120+0000","last_unstale":"2024-02-14T15:02:45.617022+0000","last_undegraded":"2024-02-14T15:02:45.617022+0000","last_fullsized":"2024-02-14T15:02:45.617022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:10:36.319337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713131+0000","last_change":"2024-02-14T15:00:53.586151+0000","last_active":"2024-02-14T15:02:45.713131+0000","last_peered":"2024-02-14T15:02:45.713131+0000","last_clean":"2024-02-14T15:02:45.713131+0000","last_became_active":"2024-02-14T15:00:53.585727+0000","last_became_peered":"2024-02-14T15:00:53.585727+0000","last_unstale":"2024-02-14T15:02:45.713131+0000","last_undegraded":"2024-02-14T15:02:45.713131+0000","last_fullsized":"2024-02-14T15:02:45.713131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:40:03.741063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":53,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766847+0000","last_change":"2024-02-14T15:01:44.615028+0000","last_active":"2024-02-14T15:02:45.766847+0000","last_peered":"2024-02-14T15:02:45.766847+0000","last_clean":"2024-02-14T15:02:45.766847+0000","last_became_active":"2024-02-14T15:01:44.606281+0000","last_became_peered":"2024-02-14T15:01:44.606281+0000","last_unstale":"2024-02-14T15:02:45.766847+0000","last_undegraded":"2024-02-14T15:02:45.766847+0000","last_fullsized":"2024-02-14T15:02:45.766847+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:37.706659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713701+0000","last_change":"2024-02-14T15:00:49.285726+0000","last_active":"2024-02-14T15:02:45.713701+0000","last_peered":"2024-02-14T15:02:45.713701+0000","last_clean":"2024-02-14T15:02:45.713701+0000","last_became_active":"2024-02-14T15:00:49.283728+0000","last_became_peered":"2024-02-14T15:00:49.283728+0000","last_unstale":"2024-02-14T15:02:45.713701+0000","last_undegraded":"2024-02-14T15:02:45.713701+0000","last_fullsized":"2024-02-14T15:02:45.713701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.764048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620850+0000","last_change":"2024-02-14T15:00:53.583644+0000","last_active":"2024-02-14T15:02:45.620850+0000","last_peered":"2024-02-14T15:02:45.620850+0000","last_clean":"2024-02-14T15:02:45.620850+0000","last_became_active":"2024-02-14T15:00:53.582801+0000","last_became_peered":"2024-02-14T15:00:53.582801+0000","last_unstale":"2024-02-14T15:02:45.620850+0000","last_undegraded":"2024-02-14T15:02:45.620850+0000","last_fullsized":"2024-02-14T15:02:45.620850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:41:40.622091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617759+0000","last_change":"2024-02-14T15:00:48.270231+0000","last_active":"2024-02-14T15:02:45.617759+0000","last_peered":"2024-02-14T15:02:45.617759+0000","last_clean":"2024-02-14T15:02:45.617759+0000","last_became_active":"2024-02-14T15:00:48.269880+0000","last_became_peered":"2024-02-14T15:00:48.269880+0000","last_unstale":"2024-02-14T15:02:45.617759+0000","last_undegraded":"2024-02-14T15:02:45.617759+0000","last_fullsized":"2024-02-14T15:02:45.617759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:08:47.376417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512291+0000","last_change":"2024-02-14T15:00:49.277227+0000","last_active":"2024-02-14T15:02:46.512291+0000","last_peered":"2024-02-14T15:02:46.512291+0000","last_clean":"2024-02-14T15:02:46.512291+0000","last_became_active":"2024-02-14T15:00:49.276904+0000","last_became_peered":"2024-02-14T15:00:49.276904+0000","last_unstale":"2024-02-14T15:02:46.512291+0000","last_undegraded":"2024-02-14T15:02:46.512291+0000","last_fullsized":"2024-02-14T15:02:46.512291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:01.091327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617891+0000","last_change":"2024-02-14T15:00:53.587855+0000","last_active":"2024-02-14T15:02:45.617891+0000","last_peered":"2024-02-14T15:02:45.617891+0000","last_clean":"2024-02-14T15:02:45.617891+0000","last_became_active":"2024-02-14T15:00:53.586313+0000","last_became_peered":"2024-02-14T15:00:53.586313+0000","last_unstale":"2024-02-14T15:02:45.617891+0000","last_undegraded":"2024-02-14T15:02:45.617891+0000","last_fullsized":"2024-02-14T15:02:45.617891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:36:41.279582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766114+0000","last_change":"2024-02-14T15:00:48.267385+0000","last_active":"2024-02-14T15:02:45.766114+0000","last_peered":"2024-02-14T15:02:45.766114+0000","last_clean":"2024-02-14T15:02:45.766114+0000","last_became_active":"2024-02-14T15:00:48.267036+0000","last_became_peered":"2024-02-14T15:00:48.267036+0000","last_unstale":"2024-02-14T15:02:45.766114+0000","last_undegraded":"2024-02-14T15:02:45.766114+0000","last_fullsized":"2024-02-14T15:02:45.766114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:29:24.840030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766956+0000","last_change":"2024-02-14T15:00:49.272233+0000","last_active":"2024-02-14T15:02:45.766956+0000","last_peered":"2024-02-14T15:02:45.766956+0000","last_clean":"2024-02-14T15:02:45.766956+0000","last_became_active":"2024-02-14T15:00:49.271813+0000","last_became_peered":"2024-02-14T15:00:49.271813+0000","last_unstale":"2024-02-14T15:02:45.766956+0000","last_undegraded":"2024-02-14T15:02:45.766956+0000","last_fullsized":"2024-02-14T15:02:45.766956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:30:32.521443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766727+0000","last_change":"2024-02-14T15:00:53.583385+0000","last_active":"2024-02-14T15:02:45.766727+0000","last_peered":"2024-02-14T15:02:45.766727+0000","last_clean":"2024-02-14T15:02:45.766727+0000","last_became_active":"2024-02-14T15:00:53.583023+0000","last_became_peered":"2024-02-14T15:00:53.583023+0000","last_unstale":"2024-02-14T15:02:45.766727+0000","last_undegraded":"2024-02-14T15:02:45.766727+0000","last_fullsized":"2024-02-14T15:02:45.766727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:40:53.500939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713137+0000","last_change":"2024-02-14T15:00:48.261376+0000","last_active":"2024-02-14T15:02:45.713137+0000","last_peered":"2024-02-14T15:02:45.713137+0000","last_clean":"2024-02-14T15:02:45.713137+0000","last_became_active":"2024-02-14T15:00:48.260461+0000","last_became_peered":"2024-02-14T15:00:48.260461+0000","last_unstale":"2024-02-14T15:02:45.713137+0000","last_undegraded":"2024-02-14T15:02:45.713137+0000","last_fullsized":"2024-02-14T15:02:45.713137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:35.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.619887+0000","last_change":"2024-02-14T15:00:49.273117+0000","last_active":"2024-02-14T15:02:45.619887+0000","last_peered":"2024-02-14T15:02:45.619887+0000","last_clean":"2024-02-14T15:02:45.619887+0000","last_became_active":"2024-02-14T15:00:49.272810+0000","last_became_peered":"2024-02-14T15:00:49.272810+0000","last_unstale":"2024-02-14T15:02:45.619887+0000","last_undegraded":"2024-02-14T15:02:45.619887+0000","last_fullsized":"2024-02-14T15:02:45.619887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:37:00.992212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617747+0000","last_change":"2024-02-14T15:00:53.583972+0000","last_active":"2024-02-14T15:02:45.617747+0000","last_peered":"2024-02-14T15:02:45.617747+0000","last_clean":"2024-02-14T15:02:45.617747+0000","last_became_active":"2024-02-14T15:00:53.583563+0000","last_became_peered":"2024-02-14T15:00:53.583563+0000","last_unstale":"2024-02-14T15:02:45.617747+0000","last_undegraded":"2024-02-14T15:02:45.617747+0000","last_fullsized":"2024-02-14T15:02:45.617747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:08:55.811082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621077+0000","last_change":"2024-02-14T15:00:48.270363+0000","last_active":"2024-02-14T15:02:45.621077+0000","last_peered":"2024-02-14T15:02:45.621077+0000","last_clean":"2024-02-14T15:02:45.621077+0000","last_became_active":"2024-02-14T15:00:48.270030+0000","last_became_peered":"2024-02-14T15:00:48.270030+0000","last_unstale":"2024-02-14T15:02:45.621077+0000","last_undegraded":"2024-02-14T15:02:45.621077+0000","last_fullsized":"2024-02-14T15:02:45.621077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:28.885451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713465+0000","last_change":"2024-02-14T15:00:49.285647+0000","last_active":"2024-02-14T15:02:45.713465+0000","last_peered":"2024-02-14T15:02:45.713465+0000","last_clean":"2024-02-14T15:02:45.713465+0000","last_became_active":"2024-02-14T15:00:49.285267+0000","last_became_peered":"2024-02-14T15:00:49.285267+0000","last_unstale":"2024-02-14T15:02:45.713465+0000","last_undegraded":"2024-02-14T15:02:45.713465+0000","last_fullsized":"2024-02-14T15:02:45.713465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:03.143177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617467+0000","last_change":"2024-02-14T15:00:53.591029+0000","last_active":"2024-02-14T15:02:45.617467+0000","last_peered":"2024-02-14T15:02:45.617467+0000","last_clean":"2024-02-14T15:02:45.617467+0000","last_became_active":"2024-02-14T15:00:53.590570+0000","last_became_peered":"2024-02-14T15:00:53.590570+0000","last_unstale":"2024-02-14T15:02:45.617467+0000","last_undegraded":"2024-02-14T15:02:45.617467+0000","last_fullsized":"2024-02-14T15:02:45.617467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:41.809227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620092+0000","last_change":"2024-02-14T15:01:44.611339+0000","last_active":"2024-02-14T15:02:45.620092+0000","last_peered":"2024-02-14T15:02:45.620092+0000","last_clean":"2024-02-14T15:02:45.620092+0000","last_became_active":"2024-02-14T15:01:44.605631+0000","last_became_peered":"2024-02-14T15:01:44.605631+0000","last_unstale":"2024-02-14T15:02:45.620092+0000","last_undegraded":"2024-02-14T15:02:45.620092+0000","last_fullsized":"2024-02-14T15:02:45.620092+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:24:19.853345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767748+0000","last_change":"2024-02-14T15:00:49.276346+0000","last_active":"2024-02-14T15:02:45.767748+0000","last_peered":"2024-02-14T15:02:45.767748+0000","last_clean":"2024-02-14T15:02:45.767748+0000","last_became_active":"2024-02-14T15:00:49.276000+0000","last_became_peered":"2024-02-14T15:00:49.276000+0000","last_unstale":"2024-02-14T15:02:45.767748+0000","last_undegraded":"2024-02-14T15:02:45.767748+0000","last_fullsized":"2024-02-14T15:02:45.767748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:14:03.811981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766244+0000","last_change":"2024-02-14T15:00:53.587021+0000","last_active":"2024-02-14T15:02:45.766244+0000","last_peered":"2024-02-14T15:02:45.766244+0000","last_clean":"2024-02-14T15:02:45.766244+0000","last_became_active":"2024-02-14T15:00:53.586492+0000","last_became_peered":"2024-02-14T15:00:53.586492+0000","last_unstale":"2024-02-14T15:02:45.766244+0000","last_undegraded":"2024-02-14T15:02:45.766244+0000","last_fullsized":"2024-02-14T15:02:45.766244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:51:02.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766373+0000","last_change":"2024-02-14T15:01:44.611916+0000","last_active":"2024-02-14T15:02:45.766373+0000","last_peered":"2024-02-14T15:02:45.766373+0000","last_clean":"2024-02-14T15:02:45.766373+0000","last_became_active":"2024-02-14T15:01:44.605791+0000","last_became_peered":"2024-02-14T15:01:44.605791+0000","last_unstale":"2024-02-14T15:02:45.766373+0000","last_undegraded":"2024-02-14T15:02:45.766373+0000","last_fullsized":"2024-02-14T15:02:45.766373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:50:54.066205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767679+0000","last_change":"2024-02-14T15:00:49.276110+0000","last_active":"2024-02-14T15:02:45.767679+0000","last_peered":"2024-02-14T15:02:45.767679+0000","last_clean":"2024-02-14T15:02:45.767679+0000","last_became_active":"2024-02-14T15:00:49.275729+0000","last_became_peered":"2024-02-14T15:00:49.275729+0000","last_unstale":"2024-02-14T15:02:45.767679+0000","last_undegraded":"2024-02-14T15:02:45.767679+0000","last_fullsized":"2024-02-14T15:02:45.767679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:07:29.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511830+0000","last_change":"2024-02-14T15:00:53.582061+0000","last_active":"2024-02-14T15:02:46.511830+0000","last_peered":"2024-02-14T15:02:46.511830+0000","last_clean":"2024-02-14T15:02:46.511830+0000","last_became_active":"2024-02-14T15:00:53.581659+0000","last_became_peered":"2024-02-14T15:00:53.581659+0000","last_unstale":"2024-02-14T15:02:46.511830+0000","last_undegraded":"2024-02-14T15:02:46.511830+0000","last_fullsized":"2024-02-14T15:02:46.511830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:13.814496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618666+0000","last_change":"2024-02-14T15:00:48.283300+0000","last_active":"2024-02-14T15:02:45.618666+0000","last_peered":"2024-02-14T15:02:45.618666+0000","last_clean":"2024-02-14T15:02:45.618666+0000","last_became_active":"2024-02-14T15:00:48.282939+0000","last_became_peered":"2024-02-14T15:00:48.282939+0000","last_unstale":"2024-02-14T15:02:45.618666+0000","last_undegraded":"2024-02-14T15:02:45.618666+0000","last_fullsized":"2024-02-14T15:02:45.618666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:56:52.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767479+0000","last_change":"2024-02-14T15:00:49.273435+0000","last_active":"2024-02-14T15:02:45.767479+0000","last_peered":"2024-02-14T15:02:45.767479+0000","last_clean":"2024-02-14T15:02:45.767479+0000","last_became_active":"2024-02-14T15:00:49.272980+0000","last_became_peered":"2024-02-14T15:00:49.272980+0000","last_unstale":"2024-02-14T15:02:45.767479+0000","last_undegraded":"2024-02-14T15:02:45.767479+0000","last_fullsized":"2024-02-14T15:02:45.767479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:19:05.294943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617563+0000","last_change":"2024-02-14T15:00:53.583509+0000","last_active":"2024-02-14T15:02:45.617563+0000","last_peered":"2024-02-14T15:02:45.617563+0000","last_clean":"2024-02-14T15:02:45.617563+0000","last_became_active":"2024-02-14T15:00:53.583148+0000","last_became_peered":"2024-02-14T15:00:53.583148+0000","last_unstale":"2024-02-14T15:02:45.617563+0000","last_undegraded":"2024-02-14T15:02:45.617563+0000","last_fullsized":"2024-02-14T15:02:45.617563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:06:47.059643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617044+0000","last_change":"2024-02-14T15:00:48.286707+0000","last_active":"2024-02-14T15:02:45.617044+0000","last_peered":"2024-02-14T15:02:45.617044+0000","last_clean":"2024-02-14T15:02:45.617044+0000","last_became_active":"2024-02-14T15:00:48.286355+0000","last_became_peered":"2024-02-14T15:00:48.286355+0000","last_unstale":"2024-02-14T15:02:45.617044+0000","last_undegraded":"2024-02-14T15:02:45.617044+0000","last_fullsized":"2024-02-14T15:02:45.617044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:48:31.024464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767643+0000","last_change":"2024-02-14T15:00:49.271598+0000","last_active":"2024-02-14T15:02:45.767643+0000","last_peered":"2024-02-14T15:02:45.767643+0000","last_clean":"2024-02-14T15:02:45.767643+0000","last_became_active":"2024-02-14T15:00:49.270572+0000","last_became_peered":"2024-02-14T15:00:49.270572+0000","last_unstale":"2024-02-14T15:02:45.767643+0000","last_undegraded":"2024-02-14T15:02:45.767643+0000","last_fullsized":"2024-02-14T15:02:45.767643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:12.666557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.619943+0000","last_change":"2024-02-14T15:00:49.276697+0000","last_active":"2024-02-14T15:02:45.619943+0000","last_peered":"2024-02-14T15:02:45.619943+0000","last_clean":"2024-02-14T15:02:45.619943+0000","last_became_active":"2024-02-14T15:00:49.276279+0000","last_became_peered":"2024-02-14T15:00:49.276279+0000","last_unstale":"2024-02-14T15:02:45.619943+0000","last_undegraded":"2024-02-14T15:02:45.619943+0000","last_fullsized":"2024-02-14T15:02:45.619943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:11:31.824974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621107+0000","last_change":"2024-02-14T15:01:44.607605+0000","last_active":"2024-02-14T15:02:45.621107+0000","last_peered":"2024-02-14T15:02:45.621107+0000","last_clean":"2024-02-14T15:02:45.621107+0000","last_became_active":"2024-02-14T15:01:44.607171+0000","last_became_peered":"2024-02-14T15:01:44.607171+0000","last_unstale":"2024-02-14T15:02:45.621107+0000","last_undegraded":"2024-02-14T15:02:45.621107+0000","last_fullsized":"2024-02-14T15:02:45.621107+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:30:20.503814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621004+0000","last_change":"2024-02-14T15:00:49.274219+0000","last_active":"2024-02-14T15:02:45.621004+0000","last_peered":"2024-02-14T15:02:45.621004+0000","last_clean":"2024-02-14T15:02:45.621004+0000","last_became_active":"2024-02-14T15:00:49.273833+0000","last_became_peered":"2024-02-14T15:00:49.273833+0000","last_unstale":"2024-02-14T15:02:45.621004+0000","last_undegraded":"2024-02-14T15:02:45.621004+0000","last_fullsized":"2024-02-14T15:02:45.621004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:06:02.720273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512067+0000","last_change":"2024-02-14T15:00:49.273576+0000","last_active":"2024-02-14T15:02:46.512067+0000","last_peered":"2024-02-14T15:02:46.512067+0000","last_clean":"2024-02-14T15:02:46.512067+0000","last_became_active":"2024-02-14T15:00:49.272920+0000","last_became_peered":"2024-02-14T15:00:49.272920+0000","last_unstale":"2024-02-14T15:02:46.512067+0000","last_undegraded":"2024-02-14T15:02:46.512067+0000","last_fullsized":"2024-02-14T15:02:46.512067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:22:03.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617100+0000","last_change":"2024-02-14T15:00:49.274549+0000","last_active":"2024-02-14T15:02:45.617100+0000","last_peered":"2024-02-14T15:02:45.617100+0000","last_clean":"2024-02-14T15:02:45.617100+0000","last_became_active":"2024-02-14T15:00:49.274097+0000","last_became_peered":"2024-02-14T15:00:49.274097+0000","last_unstale":"2024-02-14T15:02:45.617100+0000","last_undegraded":"2024-02-14T15:02:45.617100+0000","last_fullsized":"2024-02-14T15:02:45.617100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:47:44.017607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620430+0000","last_change":"2024-02-14T15:00:49.275699+0000","last_active":"2024-02-14T15:02:45.620430+0000","last_peered":"2024-02-14T15:02:45.620430+0000","last_clean":"2024-02-14T15:02:45.620430+0000","last_became_active":"2024-02-14T15:00:49.275361+0000","last_became_peered":"2024-02-14T15:00:49.275361+0000","last_unstale":"2024-02-14T15:02:45.620430+0000","last_undegraded":"2024-02-14T15:02:45.620430+0000","last_fullsized":"2024-02-14T15:02:45.620430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:34:02.638758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621841+0000","last_change":"2024-02-14T15:00:49.275024+0000","last_active":"2024-02-14T15:02:45.621841+0000","last_peered":"2024-02-14T15:02:45.621841+0000","last_clean":"2024-02-14T15:02:45.621841+0000","last_became_active":"2024-02-14T15:00:49.274710+0000","last_became_peered":"2024-02-14T15:00:49.274710+0000","last_unstale":"2024-02-14T15:02:45.621841+0000","last_undegraded":"2024-02-14T15:02:45.621841+0000","last_fullsized":"2024-02-14T15:02:45.621841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:42.471112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713794+0000","last_change":"2024-02-14T15:00:49.282638+0000","last_active":"2024-02-14T15:02:45.713794+0000","last_peered":"2024-02-14T15:02:45.713794+0000","last_clean":"2024-02-14T15:02:45.713794+0000","last_became_active":"2024-02-14T15:00:49.280223+0000","last_became_peered":"2024-02-14T15:00:49.280223+0000","last_unstale":"2024-02-14T15:02:45.713794+0000","last_undegraded":"2024-02-14T15:02:45.713794+0000","last_fullsized":"2024-02-14T15:02:45.713794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:40.250242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617680+0000","last_change":"2024-02-14T15:00:49.273025+0000","last_active":"2024-02-14T15:02:45.617680+0000","last_peered":"2024-02-14T15:02:45.617680+0000","last_clean":"2024-02-14T15:02:45.617680+0000","last_became_active":"2024-02-14T15:00:49.272700+0000","last_became_peered":"2024-02-14T15:00:49.272700+0000","last_unstale":"2024-02-14T15:02:45.617680+0000","last_undegraded":"2024-02-14T15:02:45.617680+0000","last_fullsized":"2024-02-14T15:02:45.617680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:08.677428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766013+0000","last_change":"2024-02-14T15:01:44.605957+0000","last_active":"2024-02-14T15:02:45.766013+0000","last_peered":"2024-02-14T15:02:45.766013+0000","last_clean":"2024-02-14T15:02:45.766013+0000","last_became_active":"2024-02-14T15:01:44.605398+0000","last_became_peered":"2024-02-14T15:01:44.605398+0000","last_unstale":"2024-02-14T15:02:45.766013+0000","last_undegraded":"2024-02-14T15:02:45.766013+0000","last_fullsized":"2024-02-14T15:02:45.766013+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:50:43.667496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766620+0000","last_change":"2024-02-14T15:00:49.282178+0000","last_active":"2024-02-14T15:02:45.766620+0000","last_peered":"2024-02-14T15:02:45.766620+0000","last_clean":"2024-02-14T15:02:45.766620+0000","last_became_active":"2024-02-14T15:00:49.281804+0000","last_became_peered":"2024-02-14T15:00:49.281804+0000","last_unstale":"2024-02-14T15:02:45.766620+0000","last_undegraded":"2024-02-14T15:02:45.766620+0000","last_fullsized":"2024-02-14T15:02:45.766620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:58.672808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617022+0000","last_change":"2024-02-14T15:00:49.269882+0000","last_active":"2024-02-14T15:02:45.617022+0000","last_peered":"2024-02-14T15:02:45.617022+0000","last_clean":"2024-02-14T15:02:45.617022+0000","last_became_active":"2024-02-14T15:00:49.269489+0000","last_became_peered":"2024-02-14T15:00:49.269489+0000","last_unstale":"2024-02-14T15:02:45.617022+0000","last_undegraded":"2024-02-14T15:02:45.617022+0000","last_fullsized":"2024-02-14T15:02:45.617022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:57:40.465044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620120+0000","last_change":"2024-02-14T15:00:49.277688+0000","last_active":"2024-02-14T15:02:45.620120+0000","last_peered":"2024-02-14T15:02:45.620120+0000","last_clean":"2024-02-14T15:02:45.620120+0000","last_became_active":"2024-02-14T15:00:49.277348+0000","last_became_peered":"2024-02-14T15:00:49.277348+0000","last_unstale":"2024-02-14T15:02:45.620120+0000","last_undegraded":"2024-02-14T15:02:45.620120+0000","last_fullsized":"2024-02-14T15:02:45.620120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:33.946319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68055,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":710,"num_read_kb":426,"num_write":170,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":63938,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":518659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.623},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.61299999999999999,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.52600000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.46899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.66700000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81200000000000006,"5min":1.155,"15min":1.155},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.42699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59099999999999997,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85099999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83799999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85699999999999998,"5min":1.121,"15min":1.121},"last":0.85699999999999998}]},{"osd":8,"last update":"Wed Feb 14 15:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75700000000000001,"5min":0.875,"15min":0.875},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.89200000000000002,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.875}]},{"osd":9,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92000000000000004,"5min":1.002,"15min":1.002},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98499999999999999,"5min":1.012,"15min":1.012},"last":0.76200000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.871,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.67600000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.95799999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.92100000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1744896,"data_stored":1170064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81399999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.625}]},{"osd":4,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65600000000000003,"15min":0.67300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1.054,"15min":1.054},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60399999999999998,"15min":0.64400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":1.145,"15min":1.145},"last":0.35299999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97399999999999998,"5min":1.032,"15min":1.032},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98199999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":0.82999999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.071,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.59399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0609999999999999,"5min":1.129,"15min":1.129},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81200000000000006,"15min":0.84999999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.872,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54000000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.04,"5min":1.169,"15min":1.169},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.83099999999999996,"15min":0.876},"min":{"1min":0.57199999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96199999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.66400000000000003}]},{"osd":11,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.873,"15min":0.876},"min":{"1min":0.71799999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0409999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.68200000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.083,"5min":1.141,"15min":1.141},"last":0.85099999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":522991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44900000000000001,"15min":0.46200000000000002},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":1.22,"15min":1.22},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.745},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.54200000000000004,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0629999999999999,"5min":1.141,"15min":1.141},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.69999999999999996,"15min":0.73699999999999999},"min":{"1min":0.45600000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97799999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.90600000000000003,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4370000000000001,"5min":19.152000000000001,"15min":19.152000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.31},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.94199999999999995,"15min":0.86299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":18.734999999999999,"15min":18.734999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.93999999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5069999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.94799999999999995,"15min":0.85999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.165,"5min":19.135999999999999,"15min":19.135999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0049999999999999,"5min":1.23,"15min":1.23},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed Feb 14 15:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.76400000000000001,"15min":0.79400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.79800000000000004,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1100000000000001,"5min":1.264,"15min":1.264},"last":0.79600000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.58199999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0609999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96199999999999997,"5min":1.778,"15min":1.778},"last":0.78200000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":518815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80200000000000005,"5min":13.997,"15min":13.997},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79600000000000004}]},{"osd":2,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44,"15min":0.41799999999999998},"min":{"1min":0.36899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64000000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64300000000000002,"15min":0.627},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.53200000000000003},"min":{"1min":0.373,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81200000000000006}]},{"osd":6,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63300000000000001,"15min":0.61599999999999999},"min":{"1min":0.47899999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.93700000000000006,"15min":1.175},"min":{"1min":0.56999999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93200000000000005,"5min":13.618,"15min":13.618},"last":0.86699999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.71699999999999997,"15min":0.68700000000000006},"min":{"1min":0.61499999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.69899999999999995,"15min":0.67100000000000004},"min":{"1min":0.627,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.59599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.68400000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.91100000000000003}]},{"osd":10,"last update":"Wed Feb 14 15:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71299999999999997,"15min":0.67300000000000004},"min":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.94899999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.73599999999999999,"15min":0.69999999999999996},"min":{"1min":0.67600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85599999999999998}]},{"osd":11,"last update":"Wed Feb 14 15:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.66300000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.98599999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.627,"5min":0.501,"15min":0.501},"max":{"1min":0.95099999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.96499999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1115955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71599999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":0.877,"15min":0.877},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.68899999999999995,"15min":0.65000000000000002},"min":{"1min":0.52900000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83199999999999996,"5min":0.872,"15min":0.872},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.58199999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91000000000000003,"5min":0.997,"15min":0.997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85499999999999998,"5min":1.095,"15min":1.095},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.78400000000000003,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:02:47 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.036,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.79300000000000004},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":1.139,"15min":1.139},"last":0.73599999999999999}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.052,"15min":1.052},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.67300000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.0339999999999998,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.075,"5min":1.179,"15min":1.179},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.82699999999999996},"min":{"1min":0.626,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.014,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Feb 14 15:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90400000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.94399999999999995,"15min":0.94799999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.155,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84999999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":514071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.377},"min":{"1min":0.34000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.434,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63800000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.55400000000000005},"min":{"1min":0.47899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80600000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78300000000000003,"15min":0.622},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1259999999999999,"5min":19.181000000000001,"15min":19.181000000000001},"last":0.626}]},{"osd":2,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.626,"15min":0.58499999999999996},"min":{"1min":0.53000000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85099999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.44600000000000001},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.48499999999999999},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.79000000000000004,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":19.167000000000002,"15min":19.167000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.65800000000000003,"15min":0.623},"min":{"1min":0.52900000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.56499999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.99199999999999999,"5min":1.107,"15min":1.107},"last":0.66800000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0449999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.266,"15min":1.266},"last":0.76700000000000002}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68899999999999995,"15min":0.72499999999999998},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.92300000000000004,"5min":0.996,"15min":0.996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78600000000000003,"15min":0.77800000000000002},"min":{"1min":0.64100000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.748},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79700000000000004},"min":{"1min":0.57099999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.024,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.79500000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.66700000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":578476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.64900000000000002,"15min":0.53800000000000003},"min":{"1min":0.38300000000000001,"5min":0.33900000000000002,"15min":0.32800000000000001},"max":{"1min":6.5899999999999999,"5min":6.5899999999999999,"15min":6.5899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.41599999999999998,"5min":0.33100000000000002,"15min":0.309},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.53600000000000003},"min":{"1min":0.47099999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.754},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.80800000000000005,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":18.332000000000001,"15min":18.332000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:02:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.315},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.38200000000000001},"max":{"1min":1.0349999999999999,"5min":1.046,"15min":1.046},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.68799999999999994,"15min":0.59399999999999997},"min":{"1min":0.51300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70699999999999996,"15min":0.621},"min":{"1min":0.629,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":1.016,"15min":1.016},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61299999999999999,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89400000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.73199999999999998,"15min":0.65800000000000003},"min":{"1min":0.53000000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65500000000000003,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0209999999999999,"5min":1.077,"15min":1.077},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55700000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68500000000000005,"15min":0.66400000000000003},"min":{"1min":0.56299999999999994,"5min":0.376,"15min":0.376},"max":{"1min":0.996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.80000000000000004,"15min":0.748},"min":{"1min":0.69199999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0820000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67800000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.0069999999999999,"5min":1.044,"15min":1.044},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.86399999999999999,"15min":0.84799999999999998},"min":{"1min":0.67800000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.65600000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0629999999999999,"5min":1.107,"15min":1.107},"last":0.81499999999999995}]},{"osd":11,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.93200000000000005,"15min":0.92100000000000004},"min":{"1min":0.84699999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.115,"5min":1.133,"15min":1.133},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.83199999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.84999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":515459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.31,"15min":0.253},"max":{"1min":0.76300000000000001,"5min":1.002,"15min":1.002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57499999999999996,"15min":0.52100000000000002},"min":{"1min":0.45200000000000001,"5min":0.32300000000000001,"15min":0.28199999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.53000000000000003},"min":{"1min":0.48999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72199999999999998,"5min":1.002,"15min":1.002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.58699999999999997,"15min":0.52500000000000002},"min":{"1min":0.47399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.5},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.87,"5min":1.694,"15min":1.694},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51300000000000001,"15min":0.53100000000000003},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76900000000000002,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85599999999999998,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.36399999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84399999999999997,"5min":1.139,"15min":1.139},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.1459999999999999,"5min":0.76000000000000001,"15min":0.69599999999999995},"min":{"1min":0.53500000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.8949999999999996,"5min":9.8949999999999996,"15min":9.8949999999999996},"last":0.53900000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84599999999999997,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.498,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.69299999999999995},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.109,"5min":1.153,"15min":1.153},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0649999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58699999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.60499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.63900000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.67000000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.751,"15min":0.748},"min":{"1min":0.61199999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.802,"15min":1.802},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75900000000000001,"15min":0.73299999999999998},"min":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.97399999999999998,"5min":1.242,"15min":1.242},"last":0.75900000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":514749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77400000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.501}]},{"osd":1,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58099999999999996,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.55800000000000005},"min":{"1min":0.45400000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.874}]},{"osd":5,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.46500000000000002,"15min":0.436},"min":{"1min":0.36099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.372,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93799999999999994,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.625},{"interface":"front","average":{"1min":1.284,"5min":0.748,"15min":0.65600000000000003},"min":{"1min":0.54000000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":13.955,"5min":13.955,"15min":13.955},"last":0.66400000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93400000000000005,"5min":1.046,"15min":1.046},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94999999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70999999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.498,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":1.01,"15min":1.01},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.749}]},{"osd":10,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.621,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0409999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.753,"15min":0.71799999999999997},"min":{"1min":0.54900000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.89200000000000002}]},{"osd":11,"last update":"Wed Feb 14 15:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.64700000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76900000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":514265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.375,"15min":0.32700000000000001},"min":{"1min":0.29599999999999999,"5min":0.20399999999999999,"15min":0.191},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.66900000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.76500000000000001,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.33600000000000002},"max":{"1min":0.91000000000000003,"5min":19.896999999999998,"15min":19.896999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.77400000000000002,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.99399999999999999,"5min":19.911999999999999,"15min":19.911999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59499999999999997,"15min":0.53700000000000003},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":0.995,"15min":0.995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60899999999999999,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.751,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.57999999999999996},"min":{"1min":0.46400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.879,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.498}]},{"osd":5,"last update":"Wed Feb 14 15:02:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.80500000000000005,"15min":1.3919999999999999},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":19.879999999999999,"15min":19.879999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.441},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.501,"15min":0.48499999999999999},"min":{"1min":0.33500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.39400000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.56999999999999995}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70499999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64300000000000002,"15min":0.67200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.64500000000000002,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92700000000000005,"5min":1.391,"15min":1.391},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.93600000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.628}]},{"osd":11,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86199999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67800000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":583166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.42199999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72299999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.379}]},{"osd":2,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.56100000000000005,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.91900000000000004,"5min":0.997,"15min":0.997},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89500000000000002,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89100000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65400000000000003,"15min":0.58499999999999996},"min":{"1min":0.52500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.67900000000000005,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.92400000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":0.79000000000000004}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.012,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52700000000000002}]},{"osd":8,"last update":"Wed Feb 14 15:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74399999999999999,"15min":0.71699999999999997},"min":{"1min":0.626,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0629999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.1299999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.84099999999999997}]},{"osd":9,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":1.04,"15min":1.04},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0609999999999999,"5min":1.105,"15min":1.105},"last":0.90200000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.56999999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.012,"5min":1.032,"15min":1.032},"last":0.749}]},{"osd":11,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.876},{"interface":"front","average":{"1min":0.877,"5min":0.872,"15min":0.871},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71699999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1671168,"data_stored":1110229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53600000000000003,"15min":0.49199999999999999},"min":{"1min":0.33700000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.48099999999999998},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79700000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56599999999999995,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.84899999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.52700000000000002},"min":{"1min":0.46500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78300000000000003,"5min":0.876,"15min":0.876},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40600000000000003,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.61199999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41599999999999998,"15min":0.39300000000000002},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.505},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83999999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":0.878,"15min":0.878},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81899999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63200000000000001,"15min":0.58399999999999996},"min":{"1min":0.55100000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82699999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89200000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70999999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60199999999999998,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.56399999999999995},"min":{"1min":0.45200000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95799999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.038,"15min":1.038},"last":0.90200000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.59299999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91900000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75,"15min":0.73099999999999998},"min":{"1min":0.65600000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93000000000000005,"5min":1.032,"15min":1.032},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T15:03:13.956 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 -- 172.21.15.17:0/3722403766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570100160 msgr2=0x7fa570100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 --2- 172.21.15.17:0/3722403766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570100160 0x7fa570100580 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa560009900 tx=0x7fa56002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 -- 172.21.15.17:0/3722403766 shutdown_connections 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 --2- 172.21.15.17:0/3722403766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa570101d10 0x7fa5701085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 --2- 172.21.15.17:0/3722403766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570101350 0x7fa5701017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 --2- 172.21.15.17:0/3722403766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570100160 0x7fa570100580 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 -- 172.21.15.17:0/3722403766 >> 172.21.15.17:0/3722403766 conn(0x7fa5700fb6d0 msgr2=0x7fa5700fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 -- 172.21.15.17:0/3722403766 shutdown_connections 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.090+0000 7fa574dc7640 1 -- 172.21.15.17:0/3722403766 wait complete. 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 Processor -- start 2024-02-14T15:03:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 -- start start 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570100160 0x7fa570069040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa570101350 0x7fa57019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570101d10 0x7fa57019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57019edd0 con 0x7fa570101d10 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57019ef40 con 0x7fa570101350 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.091+0000 7fa574dc7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57019f100 con 0x7fa570100160 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570101d10 0x7fa57019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570101d10 0x7fa57019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59040/0 (socket says 172.21.15.17:59040) 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 -- 172.21.15.17:0/1319317842 learned_addr learned my addr 172.21.15.17:0/1319317842 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56e59c640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570100160 0x7fa570069040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa565d9b640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa570101350 0x7fa57019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 -- 172.21.15.17:0/1319317842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570100160 msgr2=0x7fa570069040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570100160 0x7fa570069040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 -- 172.21.15.17:0/1319317842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa570101350 msgr2=0x7fa57019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa570101350 0x7fa57019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 -- 172.21.15.17:0/1319317842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa560009560 con 0x7fa570101d10 2024-02-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa56ed9d640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570101d10 0x7fa57019e640 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa56800eb10 tx=0x7fa56800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa567fff640 1 -- 172.21.15.17:0/1319317842 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568004b20 con 0x7fa570101d10 2024-02-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa567fff640 1 -- 172.21.15.17:0/1319317842 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa568004c80 con 0x7fa570101d10 2024-02-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57019f3e0 con 0x7fa570101d10 2024-02-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa567fff640 1 -- 172.21.15.17:0/1319317842 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568010650 con 0x7fa570101d10 2024-02-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.092+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57019f8a0 con 0x7fa570101d10 2024-02-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.093+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5701074a0 con 0x7fa570101d10 2024-02-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.094+0000 7fa567fff640 1 -- 172.21.15.17:0/1319317842 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa568005020 con 0x7fa570101d10 2024-02-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.095+0000 7fa567fff640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa55c0a2150 0x7fa55c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.095+0000 7fa567fff640 1 -- 172.21.15.17:0/1319317842 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fa568014070 con 0x7fa570101d10 2024-02-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.095+0000 7fa56e59c640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa55c0a2150 0x7fa55c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.096+0000 7fa56e59c640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa55c0a2150 0x7fa55c0a4610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa560005ab0 tx=0x7fa560005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.100+0000 7fa567fff640 1 -- 172.21.15.17:0/1319317842 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa568097270 con 0x7fa570101d10 2024-02-14T15:03:14.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:13 smithi050 ceph-mon[36512]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-14T15:03:14.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:13 smithi050 ceph-mon[36512]: mgrmap e19: x(active, since 9m), standbys: y, z 2024-02-14T15:03:14.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2441812074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:14.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/4087931561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:14.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:13 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/76462526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:14.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:13 smithi076 ceph-mon[37057]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-14T15:03:14.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:13 smithi076 ceph-mon[37057]: mgrmap e19: x(active, since 9m), standbys: y, z 2024-02-14T15:03:14.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2441812074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:14.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/4087931561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:14.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:13 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/76462526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:14.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.279+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa57002cef0 con 0x7fa55c0a2150 2024-02-14T15:03:14.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.291+0000 7fa567fff640 1 -- 172.21.15.17:0/1319317842 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428031 (secure 0 0 0) 0x7fa57002cef0 con 0x7fa55c0a2150 2024-02-14T15:03:14.297 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:14.299 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-02-14T15:03:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa55c0a2150 msgr2=0x7fa55c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa55c0a2150 0x7fa55c0a4610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa560005ab0 tx=0x7fa560005fd0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570101d10 msgr2=0x7fa57019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570101d10 0x7fa57019e640 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa56800eb10 tx=0x7fa56800c2d0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 shutdown_connections 2024-02-14T15:03:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa55c0a2150 0x7fa55c0a4610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa570101d10 0x7fa57019e640 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa570101350 0x7fa57019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 --2- 172.21.15.17:0/1319317842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570100160 0x7fa570069040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.295+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 >> 172.21.15.17:0/1319317842 conn(0x7fa5700fb6d0 msgr2=0x7fa570104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.296+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 shutdown_connections 2024-02-14T15:03:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.296+0000 7fa574dc7640 1 -- 172.21.15.17:0/1319317842 wait complete. 2024-02-14T15:03:14.307 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-02-14T15:03:12.696051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658657,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":846,"num_read_kb":545,"num_write":353,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1244674,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388604,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553220,"statfs":{"total":1151940427776,"available":1151542497280,"internally_reserved":0,"allocated":14778368,"data_stored":8176899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008748"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620302+0000","last_change":"2024-02-14T15:00:49.281057+0000","last_active":"2024-02-14T15:02:45.620302+0000","last_peered":"2024-02-14T15:02:45.620302+0000","last_clean":"2024-02-14T15:02:45.620302+0000","last_became_active":"2024-02-14T15:00:49.280642+0000","last_became_peered":"2024-02-14T15:00:49.280642+0000","last_unstale":"2024-02-14T15:02:45.620302+0000","last_undegraded":"2024-02-14T15:02:45.620302+0000","last_fullsized":"2024-02-14T15:02:45.620302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:03.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512248+0000","last_change":"2024-02-14T15:00:49.282794+0000","last_active":"2024-02-14T15:02:46.512248+0000","last_peered":"2024-02-14T15:02:46.512248+0000","last_clean":"2024-02-14T15:02:46.512248+0000","last_became_active":"2024-02-14T15:00:49.281441+0000","last_became_peered":"2024-02-14T15:00:49.281441+0000","last_unstale":"2024-02-14T15:02:46.512248+0000","last_undegraded":"2024-02-14T15:02:46.512248+0000","last_fullsized":"2024-02-14T15:02:46.512248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:57:36.577281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512181+0000","last_change":"2024-02-14T15:00:49.286253+0000","last_active":"2024-02-14T15:02:46.512181+0000","last_peered":"2024-02-14T15:02:46.512181+0000","last_clean":"2024-02-14T15:02:46.512181+0000","last_became_active":"2024-02-14T15:00:49.285950+0000","last_became_peered":"2024-02-14T15:00:49.285950+0000","last_unstale":"2024-02-14T15:02:46.512181+0000","last_undegraded":"2024-02-14T15:02:46.512181+0000","last_fullsized":"2024-02-14T15:02:46.512181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:53.395339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617486+0000","last_change":"2024-02-14T15:00:49.272764+0000","last_active":"2024-02-14T15:02:45.617486+0000","last_peered":"2024-02-14T15:02:45.617486+0000","last_clean":"2024-02-14T15:02:45.617486+0000","last_became_active":"2024-02-14T15:00:49.272430+0000","last_became_peered":"2024-02-14T15:00:49.272430+0000","last_unstale":"2024-02-14T15:02:45.617486+0000","last_undegraded":"2024-02-14T15:02:45.617486+0000","last_fullsized":"2024-02-14T15:02:45.617486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:36.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766939+0000","last_change":"2024-02-14T15:00:49.275028+0000","last_active":"2024-02-14T15:02:45.766939+0000","last_peered":"2024-02-14T15:02:45.766939+0000","last_clean":"2024-02-14T15:02:45.766939+0000","last_became_active":"2024-02-14T15:00:49.274686+0000","last_became_peered":"2024-02-14T15:00:49.274686+0000","last_unstale":"2024-02-14T15:02:45.766939+0000","last_undegraded":"2024-02-14T15:02:45.766939+0000","last_fullsized":"2024-02-14T15:02:45.766939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:21:23.465321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.705981+0000","last_change":"2024-02-14T15:00:49.287144+0000","last_active":"2024-02-14T15:02:45.705981+0000","last_peered":"2024-02-14T15:02:45.705981+0000","last_clean":"2024-02-14T15:02:45.705981+0000","last_became_active":"2024-02-14T15:00:49.286767+0000","last_became_peered":"2024-02-14T15:00:49.286767+0000","last_unstale":"2024-02-14T15:02:45.705981+0000","last_undegraded":"2024-02-14T15:02:45.705981+0000","last_fullsized":"2024-02-14T15:02:45.705981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:45:09.391595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767289+0000","last_change":"2024-02-14T15:00:49.274442+0000","last_active":"2024-02-14T15:02:45.767289+0000","last_peered":"2024-02-14T15:02:45.767289+0000","last_clean":"2024-02-14T15:02:45.767289+0000","last_became_active":"2024-02-14T15:00:49.274122+0000","last_became_peered":"2024-02-14T15:00:49.274122+0000","last_unstale":"2024-02-14T15:02:45.767289+0000","last_undegraded":"2024-02-14T15:02:45.767289+0000","last_fullsized":"2024-02-14T15:02:45.767289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:55.974889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.712957+0000","last_change":"2024-02-14T15:00:49.285432+0000","last_active":"2024-02-14T15:02:45.712957+0000","last_peered":"2024-02-14T15:02:45.712957+0000","last_clean":"2024-02-14T15:02:45.712957+0000","last_became_active":"2024-02-14T15:00:49.283703+0000","last_became_peered":"2024-02-14T15:00:49.283703+0000","last_unstale":"2024-02-14T15:02:45.712957+0000","last_undegraded":"2024-02-14T15:02:45.712957+0000","last_fullsized":"2024-02-14T15:02:45.712957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:14.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767120+0000","last_change":"2024-02-14T15:00:49.283908+0000","last_active":"2024-02-14T15:02:45.767120+0000","last_peered":"2024-02-14T15:02:45.767120+0000","last_clean":"2024-02-14T15:02:45.767120+0000","last_became_active":"2024-02-14T15:00:49.283131+0000","last_became_peered":"2024-02-14T15:00:49.283131+0000","last_unstale":"2024-02-14T15:02:45.767120+0000","last_undegraded":"2024-02-14T15:02:45.767120+0000","last_fullsized":"2024-02-14T15:02:45.767120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:42.247624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706471+0000","last_change":"2024-02-14T15:00:49.287229+0000","last_active":"2024-02-14T15:02:45.706471+0000","last_peered":"2024-02-14T15:02:45.706471+0000","last_clean":"2024-02-14T15:02:45.706471+0000","last_became_active":"2024-02-14T15:00:49.286791+0000","last_became_peered":"2024-02-14T15:00:49.286791+0000","last_unstale":"2024-02-14T15:02:45.706471+0000","last_undegraded":"2024-02-14T15:02:45.706471+0000","last_fullsized":"2024-02-14T15:02:45.706471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:33:01.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767467+0000","last_change":"2024-02-14T15:00:49.281669+0000","last_active":"2024-02-14T15:02:45.767467+0000","last_peered":"2024-02-14T15:02:45.767467+0000","last_clean":"2024-02-14T15:02:45.767467+0000","last_became_active":"2024-02-14T15:00:49.281298+0000","last_became_peered":"2024-02-14T15:00:49.281298+0000","last_unstale":"2024-02-14T15:02:45.767467+0000","last_undegraded":"2024-02-14T15:02:45.767467+0000","last_fullsized":"2024-02-14T15:02:45.767467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:54:40.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617437+0000","last_change":"2024-02-14T15:00:49.289679+0000","last_active":"2024-02-14T15:02:45.617437+0000","last_peered":"2024-02-14T15:02:45.617437+0000","last_clean":"2024-02-14T15:02:45.617437+0000","last_became_active":"2024-02-14T15:00:49.288767+0000","last_became_peered":"2024-02-14T15:00:49.288767+0000","last_unstale":"2024-02-14T15:02:45.617437+0000","last_undegraded":"2024-02-14T15:02:45.617437+0000","last_fullsized":"2024-02-14T15:02:45.617437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:16:20.181517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616867+0000","last_change":"2024-02-14T15:00:49.257471+0000","last_active":"2024-02-14T15:02:45.616867+0000","last_peered":"2024-02-14T15:02:45.616867+0000","last_clean":"2024-02-14T15:02:45.616867+0000","last_became_active":"2024-02-14T15:00:49.256766+0000","last_became_peered":"2024-02-14T15:00:49.256766+0000","last_unstale":"2024-02-14T15:02:45.616867+0000","last_undegraded":"2024-02-14T15:02:45.616867+0000","last_fullsized":"2024-02-14T15:02:45.616867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:14:00.525621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511859+0000","last_change":"2024-02-14T15:00:49.278462+0000","last_active":"2024-02-14T15:02:46.511859+0000","last_peered":"2024-02-14T15:02:46.511859+0000","last_clean":"2024-02-14T15:02:46.511859+0000","last_became_active":"2024-02-14T15:00:49.278114+0000","last_became_peered":"2024-02-14T15:00:49.278114+0000","last_unstale":"2024-02-14T15:02:46.511859+0000","last_undegraded":"2024-02-14T15:02:46.511859+0000","last_fullsized":"2024-02-14T15:02:46.511859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:02:16.601359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511865+0000","last_change":"2024-02-14T15:00:49.270653+0000","last_active":"2024-02-14T15:02:46.511865+0000","last_peered":"2024-02-14T15:02:46.511865+0000","last_clean":"2024-02-14T15:02:46.511865+0000","last_became_active":"2024-02-14T15:00:49.270334+0000","last_became_peered":"2024-02-14T15:00:49.270334+0000","last_unstale":"2024-02-14T15:02:46.511865+0000","last_undegraded":"2024-02-14T15:02:46.511865+0000","last_fullsized":"2024-02-14T15:02:46.511865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:13:53.840909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766103+0000","last_change":"2024-02-14T15:00:49.280597+0000","last_active":"2024-02-14T15:02:45.766103+0000","last_peered":"2024-02-14T15:02:45.766103+0000","last_clean":"2024-02-14T15:02:45.766103+0000","last_became_active":"2024-02-14T15:00:49.279612+0000","last_became_peered":"2024-02-14T15:00:49.279612+0000","last_unstale":"2024-02-14T15:02:45.766103+0000","last_undegraded":"2024-02-14T15:02:45.766103+0000","last_fullsized":"2024-02-14T15:02:45.766103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:11.676608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620327+0000","last_change":"2024-02-14T15:00:49.281422+0000","last_active":"2024-02-14T15:02:45.620327+0000","last_peered":"2024-02-14T15:02:45.620327+0000","last_clean":"2024-02-14T15:02:45.620327+0000","last_became_active":"2024-02-14T15:00:49.281048+0000","last_became_peered":"2024-02-14T15:00:49.281048+0000","last_unstale":"2024-02-14T15:02:45.620327+0000","last_undegraded":"2024-02-14T15:02:45.620327+0000","last_fullsized":"2024-02-14T15:02:45.620327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:04.994390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766686+0000","last_change":"2024-02-14T15:00:49.288245+0000","last_active":"2024-02-14T15:02:45.766686+0000","last_peered":"2024-02-14T15:02:45.766686+0000","last_clean":"2024-02-14T15:02:45.766686+0000","last_became_active":"2024-02-14T15:00:49.287908+0000","last_became_peered":"2024-02-14T15:00:49.287908+0000","last_unstale":"2024-02-14T15:02:45.766686+0000","last_undegraded":"2024-02-14T15:02:45.766686+0000","last_fullsized":"2024-02-14T15:02:45.766686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:52.397826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621696+0000","last_change":"2024-02-14T15:00:49.275428+0000","last_active":"2024-02-14T15:02:45.621696+0000","last_peered":"2024-02-14T15:02:45.621696+0000","last_clean":"2024-02-14T15:02:45.621696+0000","last_became_active":"2024-02-14T15:00:49.273464+0000","last_became_peered":"2024-02-14T15:00:49.273464+0000","last_unstale":"2024-02-14T15:02:45.621696+0000","last_undegraded":"2024-02-14T15:02:45.621696+0000","last_fullsized":"2024-02-14T15:02:45.621696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:34.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621145+0000","last_change":"2024-02-14T15:00:53.587219+0000","last_active":"2024-02-14T15:02:45.621145+0000","last_peered":"2024-02-14T15:02:45.621145+0000","last_clean":"2024-02-14T15:02:45.621145+0000","last_became_active":"2024-02-14T15:00:53.586917+0000","last_became_peered":"2024-02-14T15:00:53.586917+0000","last_unstale":"2024-02-14T15:02:45.621145+0000","last_undegraded":"2024-02-14T15:02:45.621145+0000","last_fullsized":"2024-02-14T15:02:45.621145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:39:38.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766753+0000","last_change":"2024-02-14T15:00:48.269243+0000","last_active":"2024-02-14T15:02:45.766753+0000","last_peered":"2024-02-14T15:02:45.766753+0000","last_clean":"2024-02-14T15:02:45.766753+0000","last_became_active":"2024-02-14T15:00:48.268883+0000","last_became_peered":"2024-02-14T15:00:48.268883+0000","last_unstale":"2024-02-14T15:02:45.766753+0000","last_undegraded":"2024-02-14T15:02:45.766753+0000","last_fullsized":"2024-02-14T15:02:45.766753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:08.530092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713654+0000","last_change":"2024-02-14T15:00:49.282458+0000","last_active":"2024-02-14T15:02:45.713654+0000","last_peered":"2024-02-14T15:02:45.713654+0000","last_clean":"2024-02-14T15:02:45.713654+0000","last_became_active":"2024-02-14T15:00:49.281982+0000","last_became_peered":"2024-02-14T15:00:49.281982+0000","last_unstale":"2024-02-14T15:02:45.713654+0000","last_undegraded":"2024-02-14T15:02:45.713654+0000","last_fullsized":"2024-02-14T15:02:45.713654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:51:33.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617171+0000","last_change":"2024-02-14T15:00:53.586578+0000","last_active":"2024-02-14T15:02:45.617171+0000","last_peered":"2024-02-14T15:02:45.617171+0000","last_clean":"2024-02-14T15:02:45.617171+0000","last_became_active":"2024-02-14T15:00:53.586109+0000","last_became_peered":"2024-02-14T15:00:53.586109+0000","last_unstale":"2024-02-14T15:02:45.617171+0000","last_undegraded":"2024-02-14T15:02:45.617171+0000","last_fullsized":"2024-02-14T15:02:45.617171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:07.332435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713729+0000","last_change":"2024-02-14T15:00:48.281953+0000","last_active":"2024-02-14T15:02:45.713729+0000","last_peered":"2024-02-14T15:02:45.713729+0000","last_clean":"2024-02-14T15:02:45.713729+0000","last_became_active":"2024-02-14T15:00:48.281590+0000","last_became_peered":"2024-02-14T15:00:48.281590+0000","last_unstale":"2024-02-14T15:02:45.713729+0000","last_undegraded":"2024-02-14T15:02:45.713729+0000","last_fullsized":"2024-02-14T15:02:45.713729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:22.991037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512389+0000","last_change":"2024-02-14T15:00:49.284729+0000","last_active":"2024-02-14T15:02:46.512389+0000","last_peered":"2024-02-14T15:02:46.512389+0000","last_clean":"2024-02-14T15:02:46.512389+0000","last_became_active":"2024-02-14T15:00:49.284013+0000","last_became_peered":"2024-02-14T15:00:49.284013+0000","last_unstale":"2024-02-14T15:02:46.512389+0000","last_undegraded":"2024-02-14T15:02:46.512389+0000","last_fullsized":"2024-02-14T15:02:46.512389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:03:31.313032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617441+0000","last_change":"2024-02-14T15:00:53.589825+0000","last_active":"2024-02-14T15:02:45.617441+0000","last_peered":"2024-02-14T15:02:45.617441+0000","last_clean":"2024-02-14T15:02:45.617441+0000","last_became_active":"2024-02-14T15:00:53.589424+0000","last_became_peered":"2024-02-14T15:00:53.589424+0000","last_unstale":"2024-02-14T15:02:45.617441+0000","last_undegraded":"2024-02-14T15:02:45.617441+0000","last_fullsized":"2024-02-14T15:02:45.617441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:24.522152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767437+0000","last_change":"2024-02-14T15:00:48.276884+0000","last_active":"2024-02-14T15:02:45.767437+0000","last_peered":"2024-02-14T15:02:45.767437+0000","last_clean":"2024-02-14T15:02:45.767437+0000","last_became_active":"2024-02-14T15:00:48.276576+0000","last_became_peered":"2024-02-14T15:00:48.276576+0000","last_unstale":"2024-02-14T15:02:45.767437+0000","last_undegraded":"2024-02-14T15:02:45.767437+0000","last_fullsized":"2024-02-14T15:02:45.767437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:25.793368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766543+0000","last_change":"2024-02-14T15:00:49.277602+0000","last_active":"2024-02-14T15:02:45.766543+0000","last_peered":"2024-02-14T15:02:45.766543+0000","last_clean":"2024-02-14T15:02:45.766543+0000","last_became_active":"2024-02-14T15:00:49.277249+0000","last_became_peered":"2024-02-14T15:00:49.277249+0000","last_unstale":"2024-02-14T15:02:45.766543+0000","last_undegraded":"2024-02-14T15:02:45.766543+0000","last_fullsized":"2024-02-14T15:02:45.766543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.347962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617245+0000","last_change":"2024-02-14T15:00:53.587838+0000","last_active":"2024-02-14T15:02:45.617245+0000","last_peered":"2024-02-14T15:02:45.617245+0000","last_clean":"2024-02-14T15:02:45.617245+0000","last_became_active":"2024-02-14T15:00:53.587012+0000","last_became_peered":"2024-02-14T15:00:53.587012+0000","last_unstale":"2024-02-14T15:02:45.617245+0000","last_undegraded":"2024-02-14T15:02:45.617245+0000","last_fullsized":"2024-02-14T15:02:45.617245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:32:12.276578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767139+0000","last_change":"2024-02-14T15:00:48.273529+0000","last_active":"2024-02-14T15:02:45.767139+0000","last_peered":"2024-02-14T15:02:45.767139+0000","last_clean":"2024-02-14T15:02:45.767139+0000","last_became_active":"2024-02-14T15:00:48.272555+0000","last_became_peered":"2024-02-14T15:00:48.272555+0000","last_unstale":"2024-02-14T15:02:45.767139+0000","last_undegraded":"2024-02-14T15:02:45.767139+0000","last_fullsized":"2024-02-14T15:02:45.767139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:19.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618504+0000","last_change":"2024-02-14T15:00:49.278834+0000","last_active":"2024-02-14T15:02:45.618504+0000","last_peered":"2024-02-14T15:02:45.618504+0000","last_clean":"2024-02-14T15:02:45.618504+0000","last_became_active":"2024-02-14T15:00:49.277095+0000","last_became_peered":"2024-02-14T15:00:49.277095+0000","last_unstale":"2024-02-14T15:02:45.618504+0000","last_undegraded":"2024-02-14T15:02:45.618504+0000","last_fullsized":"2024-02-14T15:02:45.618504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:27.291974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.768818+0000","last_change":"2024-02-14T15:02:45.768818+0000","last_active":"2024-02-14T15:02:45.768818+0000","last_peered":"2024-02-14T15:02:45.768818+0000","last_clean":"2024-02-14T15:02:45.768818+0000","last_became_active":"2024-02-14T15:02:45.768283+0000","last_became_peered":"2024-02-14T15:02:45.768283+0000","last_unstale":"2024-02-14T15:02:45.768818+0000","last_undegraded":"2024-02-14T15:02:45.768818+0000","last_fullsized":"2024-02-14T15:02:45.768818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:59:14.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.743886+0000","last_change":"2024-02-14T15:00:48.276146+0000","last_active":"2024-02-14T15:03:09.743886+0000","last_peered":"2024-02-14T15:03:09.743886+0000","last_clean":"2024-02-14T15:03:09.743886+0000","last_became_active":"2024-02-14T15:00:48.275833+0000","last_became_peered":"2024-02-14T15:00:48.275833+0000","last_unstale":"2024-02-14T15:03:09.743886+0000","last_undegraded":"2024-02-14T15:03:09.743886+0000","last_fullsized":"2024-02-14T15:03:09.743886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:46.649486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766980+0000","last_change":"2024-02-14T15:00:49.267285+0000","last_active":"2024-02-14T15:02:45.766980+0000","last_peered":"2024-02-14T15:02:45.766980+0000","last_clean":"2024-02-14T15:02:45.766980+0000","last_became_active":"2024-02-14T15:00:49.265592+0000","last_became_peered":"2024-02-14T15:00:49.265592+0000","last_unstale":"2024-02-14T15:02:45.766980+0000","last_undegraded":"2024-02-14T15:02:45.766980+0000","last_fullsized":"2024-02-14T15:02:45.766980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:18.469835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621465+0000","last_change":"2024-02-14T15:00:53.582085+0000","last_active":"2024-02-14T15:02:45.621465+0000","last_peered":"2024-02-14T15:02:45.621465+0000","last_clean":"2024-02-14T15:02:45.621465+0000","last_became_active":"2024-02-14T15:00:53.581706+0000","last_became_peered":"2024-02-14T15:00:53.581706+0000","last_unstale":"2024-02-14T15:02:45.621465+0000","last_undegraded":"2024-02-14T15:02:45.621465+0000","last_fullsized":"2024-02-14T15:02:45.621465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:32.481142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.705924+0000","last_change":"2024-02-14T15:00:48.268975+0000","last_active":"2024-02-14T15:02:45.705924+0000","last_peered":"2024-02-14T15:02:45.705924+0000","last_clean":"2024-02-14T15:02:45.705924+0000","last_became_active":"2024-02-14T15:00:48.268547+0000","last_became_peered":"2024-02-14T15:00:48.268547+0000","last_unstale":"2024-02-14T15:02:45.705924+0000","last_undegraded":"2024-02-14T15:02:45.705924+0000","last_fullsized":"2024-02-14T15:02:45.705924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:13.937917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618357+0000","last_change":"2024-02-14T15:00:49.276909+0000","last_active":"2024-02-14T15:02:45.618357+0000","last_peered":"2024-02-14T15:02:45.618357+0000","last_clean":"2024-02-14T15:02:45.618357+0000","last_became_active":"2024-02-14T15:00:49.276533+0000","last_became_peered":"2024-02-14T15:00:49.276533+0000","last_unstale":"2024-02-14T15:02:45.618357+0000","last_undegraded":"2024-02-14T15:02:45.618357+0000","last_fullsized":"2024-02-14T15:02:45.618357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:35:31.321455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620174+0000","last_change":"2024-02-14T15:00:53.592223+0000","last_active":"2024-02-14T15:02:45.620174+0000","last_peered":"2024-02-14T15:02:45.620174+0000","last_clean":"2024-02-14T15:02:45.620174+0000","last_became_active":"2024-02-14T15:00:53.591883+0000","last_became_peered":"2024-02-14T15:00:53.591883+0000","last_unstale":"2024-02-14T15:02:45.620174+0000","last_undegraded":"2024-02-14T15:02:45.620174+0000","last_fullsized":"2024-02-14T15:02:45.620174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:10.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616486+0000","last_change":"2024-02-14T15:00:48.274485+0000","last_active":"2024-02-14T15:02:45.616486+0000","last_peered":"2024-02-14T15:02:45.616486+0000","last_clean":"2024-02-14T15:02:45.616486+0000","last_became_active":"2024-02-14T15:00:48.274160+0000","last_became_peered":"2024-02-14T15:00:48.274160+0000","last_unstale":"2024-02-14T15:02:45.616486+0000","last_undegraded":"2024-02-14T15:02:45.616486+0000","last_fullsized":"2024-02-14T15:02:45.616486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:00:47.105069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713108+0000","last_change":"2024-02-14T15:00:49.272577+0000","last_active":"2024-02-14T15:02:45.713108+0000","last_peered":"2024-02-14T15:02:45.713108+0000","last_clean":"2024-02-14T15:02:45.713108+0000","last_became_active":"2024-02-14T15:00:49.271670+0000","last_became_peered":"2024-02-14T15:00:49.271670+0000","last_unstale":"2024-02-14T15:02:45.713108+0000","last_undegraded":"2024-02-14T15:02:45.713108+0000","last_fullsized":"2024-02-14T15:02:45.713108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:19.968389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512199+0000","last_change":"2024-02-14T15:00:53.581212+0000","last_active":"2024-02-14T15:02:46.512199+0000","last_peered":"2024-02-14T15:02:46.512199+0000","last_clean":"2024-02-14T15:02:46.512199+0000","last_became_active":"2024-02-14T15:00:53.580782+0000","last_became_peered":"2024-02-14T15:00:53.580782+0000","last_unstale":"2024-02-14T15:02:46.512199+0000","last_undegraded":"2024-02-14T15:02:46.512199+0000","last_fullsized":"2024-02-14T15:02:46.512199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:14.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617597+0000","last_change":"2024-02-14T15:00:48.273572+0000","last_active":"2024-02-14T15:02:45.617597+0000","last_peered":"2024-02-14T15:02:45.617597+0000","last_clean":"2024-02-14T15:02:45.617597+0000","last_became_active":"2024-02-14T15:00:48.273138+0000","last_became_peered":"2024-02-14T15:00:48.273138+0000","last_unstale":"2024-02-14T15:02:45.617597+0000","last_undegraded":"2024-02-14T15:02:45.617597+0000","last_fullsized":"2024-02-14T15:02:45.617597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:04.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766140+0000","last_change":"2024-02-14T15:00:49.288226+0000","last_active":"2024-02-14T15:02:45.766140+0000","last_peered":"2024-02-14T15:02:45.766140+0000","last_clean":"2024-02-14T15:02:45.766140+0000","last_became_active":"2024-02-14T15:00:49.287874+0000","last_became_peered":"2024-02-14T15:00:49.287874+0000","last_unstale":"2024-02-14T15:02:45.766140+0000","last_undegraded":"2024-02-14T15:02:45.766140+0000","last_fullsized":"2024-02-14T15:02:45.766140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:24:02.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713194+0000","last_change":"2024-02-14T15:00:53.590543+0000","last_active":"2024-02-14T15:02:45.713194+0000","last_peered":"2024-02-14T15:02:45.713194+0000","last_clean":"2024-02-14T15:02:45.713194+0000","last_became_active":"2024-02-14T15:00:53.589072+0000","last_became_peered":"2024-02-14T15:00:53.589072+0000","last_unstale":"2024-02-14T15:02:45.713194+0000","last_undegraded":"2024-02-14T15:02:45.713194+0000","last_fullsized":"2024-02-14T15:02:45.713194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:13.685655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":146,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:06.739070+0000","last_change":"2024-02-14T15:00:48.272027+0000","last_active":"2024-02-14T15:03:06.739070+0000","last_peered":"2024-02-14T15:03:06.739070+0000","last_clean":"2024-02-14T15:03:06.739070+0000","last_became_active":"2024-02-14T15:00:48.271587+0000","last_became_peered":"2024-02-14T15:00:48.271587+0000","last_unstale":"2024-02-14T15:03:06.739070+0000","last_undegraded":"2024-02-14T15:03:06.739070+0000","last_fullsized":"2024-02-14T15:03:06.739070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:33:03.566189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766999+0000","last_change":"2024-02-14T15:00:49.278936+0000","last_active":"2024-02-14T15:02:45.766999+0000","last_peered":"2024-02-14T15:02:45.766999+0000","last_clean":"2024-02-14T15:02:45.766999+0000","last_became_active":"2024-02-14T15:00:49.278318+0000","last_became_peered":"2024-02-14T15:00:49.278318+0000","last_unstale":"2024-02-14T15:02:45.766999+0000","last_undegraded":"2024-02-14T15:02:45.766999+0000","last_fullsized":"2024-02-14T15:02:45.766999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:01.327303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766017+0000","last_change":"2024-02-14T15:00:53.580340+0000","last_active":"2024-02-14T15:02:45.766017+0000","last_peered":"2024-02-14T15:02:45.766017+0000","last_clean":"2024-02-14T15:02:45.766017+0000","last_became_active":"2024-02-14T15:00:53.579742+0000","last_became_peered":"2024-02-14T15:00:53.579742+0000","last_unstale":"2024-02-14T15:02:45.766017+0000","last_undegraded":"2024-02-14T15:02:45.766017+0000","last_fullsized":"2024-02-14T15:02:45.766017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:38:33.182328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767303+0000","last_change":"2024-02-14T15:00:48.271891+0000","last_active":"2024-02-14T15:02:45.767303+0000","last_peered":"2024-02-14T15:02:45.767303+0000","last_clean":"2024-02-14T15:02:45.767303+0000","last_became_active":"2024-02-14T15:00:48.271419+0000","last_became_peered":"2024-02-14T15:00:48.271419+0000","last_unstale":"2024-02-14T15:02:45.767303+0000","last_undegraded":"2024-02-14T15:02:45.767303+0000","last_fullsized":"2024-02-14T15:02:45.767303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:03:04.725647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512325+0000","last_change":"2024-02-14T15:00:49.277025+0000","last_active":"2024-02-14T15:02:46.512325+0000","last_peered":"2024-02-14T15:02:46.512325+0000","last_clean":"2024-02-14T15:02:46.512325+0000","last_became_active":"2024-02-14T15:00:49.276694+0000","last_became_peered":"2024-02-14T15:00:49.276694+0000","last_unstale":"2024-02-14T15:02:46.512325+0000","last_undegraded":"2024-02-14T15:02:46.512325+0000","last_fullsized":"2024-02-14T15:02:46.512325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:43:07.249407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617568+0000","last_change":"2024-02-14T15:01:44.607053+0000","last_active":"2024-02-14T15:02:45.617568+0000","last_peered":"2024-02-14T15:02:45.617568+0000","last_clean":"2024-02-14T15:02:45.617568+0000","last_became_active":"2024-02-14T15:01:44.606596+0000","last_became_peered":"2024-02-14T15:01:44.606596+0000","last_unstale":"2024-02-14T15:02:45.617568+0000","last_undegraded":"2024-02-14T15:02:45.617568+0000","last_fullsized":"2024-02-14T15:02:45.617568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:35.949953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618389+0000","last_change":"2024-02-14T15:00:48.272866+0000","last_active":"2024-02-14T15:02:45.618389+0000","last_peered":"2024-02-14T15:02:45.618389+0000","last_clean":"2024-02-14T15:02:45.618389+0000","last_became_active":"2024-02-14T15:00:48.272520+0000","last_became_peered":"2024-02-14T15:00:48.272520+0000","last_unstale":"2024-02-14T15:02:45.618389+0000","last_undegraded":"2024-02-14T15:02:45.618389+0000","last_fullsized":"2024-02-14T15:02:45.618389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:39.434577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:49.280994+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:49.280205+0000","last_became_peered":"2024-02-14T15:00:49.280205+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:53:48.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766150+0000","last_change":"2024-02-14T15:00:53.582599+0000","last_active":"2024-02-14T15:02:45.766150+0000","last_peered":"2024-02-14T15:02:45.766150+0000","last_clean":"2024-02-14T15:02:45.766150+0000","last_became_active":"2024-02-14T15:00:53.582231+0000","last_became_peered":"2024-02-14T15:00:53.582231+0000","last_unstale":"2024-02-14T15:02:45.766150+0000","last_undegraded":"2024-02-14T15:02:45.766150+0000","last_fullsized":"2024-02-14T15:02:45.766150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:19.126264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":319,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:07.740623+0000","last_change":"2024-02-14T15:01:44.615393+0000","last_active":"2024-02-14T15:03:07.740623+0000","last_peered":"2024-02-14T15:03:07.740623+0000","last_clean":"2024-02-14T15:03:07.740623+0000","last_became_active":"2024-02-14T15:01:44.604294+0000","last_became_peered":"2024-02-14T15:01:44.604294+0000","last_unstale":"2024-02-14T15:03:07.740623+0000","last_undegraded":"2024-02-14T15:03:07.740623+0000","last_fullsized":"2024-02-14T15:03:07.740623+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:00.459860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":180,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767208+0000","last_change":"2024-02-14T15:00:49.270144+0000","last_active":"2024-02-14T15:02:45.767208+0000","last_peered":"2024-02-14T15:02:45.767208+0000","last_clean":"2024-02-14T15:02:45.767208+0000","last_became_active":"2024-02-14T15:00:49.269806+0000","last_became_peered":"2024-02-14T15:00:49.269806+0000","last_unstale":"2024-02-14T15:02:45.767208+0000","last_undegraded":"2024-02-14T15:02:45.767208+0000","last_fullsized":"2024-02-14T15:02:45.767208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:42:18.559226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618862+0000","last_change":"2024-02-14T15:00:53.582755+0000","last_active":"2024-02-14T15:02:45.618862+0000","last_peered":"2024-02-14T15:02:45.618862+0000","last_clean":"2024-02-14T15:02:45.618862+0000","last_became_active":"2024-02-14T15:00:53.582375+0000","last_became_peered":"2024-02-14T15:00:53.582375+0000","last_unstale":"2024-02-14T15:02:45.618862+0000","last_undegraded":"2024-02-14T15:02:45.618862+0000","last_fullsized":"2024-02-14T15:02:45.618862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:11:49.115136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621377+0000","last_change":"2024-02-14T15:00:48.268982+0000","last_active":"2024-02-14T15:02:45.621377+0000","last_peered":"2024-02-14T15:02:45.621377+0000","last_clean":"2024-02-14T15:02:45.621377+0000","last_became_active":"2024-02-14T15:00:48.267137+0000","last_became_peered":"2024-02-14T15:00:48.267137+0000","last_unstale":"2024-02-14T15:02:45.621377+0000","last_undegraded":"2024-02-14T15:02:45.621377+0000","last_fullsized":"2024-02-14T15:02:45.621377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:44.351561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616587+0000","last_change":"2024-02-14T15:00:49.267493+0000","last_active":"2024-02-14T15:02:45.616587+0000","last_peered":"2024-02-14T15:02:45.616587+0000","last_clean":"2024-02-14T15:02:45.616587+0000","last_became_active":"2024-02-14T15:00:49.267161+0000","last_became_peered":"2024-02-14T15:00:49.267161+0000","last_unstale":"2024-02-14T15:02:45.616587+0000","last_undegraded":"2024-02-14T15:02:45.616587+0000","last_fullsized":"2024-02-14T15:02:45.616587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:35:22.997999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.768993+0000","last_change":"2024-02-14T15:02:45.768993+0000","last_active":"2024-02-14T15:02:45.768993+0000","last_peered":"2024-02-14T15:02:45.768993+0000","last_clean":"2024-02-14T15:02:45.768993+0000","last_became_active":"2024-02-14T15:02:45.768413+0000","last_became_peered":"2024-02-14T15:02:45.768413+0000","last_unstale":"2024-02-14T15:02:45.768993+0000","last_undegraded":"2024-02-14T15:02:45.768993+0000","last_fullsized":"2024-02-14T15:02:45.768993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:45:54.193127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618346+0000","last_change":"2024-02-14T15:00:48.282085+0000","last_active":"2024-02-14T15:02:45.618346+0000","last_peered":"2024-02-14T15:02:45.618346+0000","last_clean":"2024-02-14T15:02:45.618346+0000","last_became_active":"2024-02-14T15:00:48.281836+0000","last_became_peered":"2024-02-14T15:00:48.281836+0000","last_unstale":"2024-02-14T15:02:45.618346+0000","last_undegraded":"2024-02-14T15:02:45.618346+0000","last_fullsized":"2024-02-14T15:02:45.618346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:07:28.145074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767345+0000","last_change":"2024-02-14T15:00:49.280013+0000","last_active":"2024-02-14T15:02:45.767345+0000","last_peered":"2024-02-14T15:02:45.767345+0000","last_clean":"2024-02-14T15:02:45.767345+0000","last_became_active":"2024-02-14T15:00:49.279667+0000","last_became_peered":"2024-02-14T15:00:49.279667+0000","last_unstale":"2024-02-14T15:02:45.767345+0000","last_undegraded":"2024-02-14T15:02:45.767345+0000","last_fullsized":"2024-02-14T15:02:45.767345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:03:16.559295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766612+0000","last_change":"2024-02-14T15:00:53.580269+0000","last_active":"2024-02-14T15:02:45.766612+0000","last_peered":"2024-02-14T15:02:45.766612+0000","last_clean":"2024-02-14T15:02:45.766612+0000","last_became_active":"2024-02-14T15:00:53.579863+0000","last_became_peered":"2024-02-14T15:00:53.579863+0000","last_unstale":"2024-02-14T15:02:45.766612+0000","last_undegraded":"2024-02-14T15:02:45.766612+0000","last_fullsized":"2024-02-14T15:02:45.766612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:37:19.275654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.744250+0000","last_change":"2024-02-14T15:00:48.267310+0000","last_active":"2024-02-14T15:03:09.744250+0000","last_peered":"2024-02-14T15:03:09.744250+0000","last_clean":"2024-02-14T15:03:09.744250+0000","last_became_active":"2024-02-14T15:00:48.266864+0000","last_became_peered":"2024-02-14T15:00:48.266864+0000","last_unstale":"2024-02-14T15:03:09.744250+0000","last_undegraded":"2024-02-14T15:03:09.744250+0000","last_fullsized":"2024-02-14T15:03:09.744250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:07.703549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:49.288568+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:49.288222+0000","last_became_peered":"2024-02-14T15:00:49.288222+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:38:38.909769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616994+0000","last_change":"2024-02-14T15:00:53.585673+0000","last_active":"2024-02-14T15:02:45.616994+0000","last_peered":"2024-02-14T15:02:45.616994+0000","last_clean":"2024-02-14T15:02:45.616994+0000","last_became_active":"2024-02-14T15:00:53.585238+0000","last_became_peered":"2024-02-14T15:00:53.585238+0000","last_unstale":"2024-02-14T15:02:45.616994+0000","last_undegraded":"2024-02-14T15:02:45.616994+0000","last_fullsized":"2024-02-14T15:02:45.616994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:48:11.845921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:48.275833+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:48.275499+0000","last_became_peered":"2024-02-14T15:00:48.275499+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:35:10.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618303+0000","last_change":"2024-02-14T15:01:44.605394+0000","last_active":"2024-02-14T15:02:45.618303+0000","last_peered":"2024-02-14T15:02:45.618303+0000","last_clean":"2024-02-14T15:02:45.618303+0000","last_became_active":"2024-02-14T15:01:44.604874+0000","last_became_peered":"2024-02-14T15:01:44.604874+0000","last_unstale":"2024-02-14T15:02:45.618303+0000","last_undegraded":"2024-02-14T15:02:45.618303+0000","last_fullsized":"2024-02-14T15:02:45.618303+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:41:47.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618214+0000","last_change":"2024-02-14T15:00:53.586406+0000","last_active":"2024-02-14T15:02:45.618214+0000","last_peered":"2024-02-14T15:02:45.618214+0000","last_clean":"2024-02-14T15:02:45.618214+0000","last_became_active":"2024-02-14T15:00:53.586034+0000","last_became_peered":"2024-02-14T15:00:53.586034+0000","last_unstale":"2024-02-14T15:02:45.618214+0000","last_undegraded":"2024-02-14T15:02:45.618214+0000","last_fullsized":"2024-02-14T15:02:45.618214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:46.886154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767472+0000","last_change":"2024-02-14T15:00:48.274305+0000","last_active":"2024-02-14T15:02:45.767472+0000","last_peered":"2024-02-14T15:02:45.767472+0000","last_clean":"2024-02-14T15:02:45.767472+0000","last_became_active":"2024-02-14T15:00:48.273966+0000","last_became_peered":"2024-02-14T15:00:48.273966+0000","last_unstale":"2024-02-14T15:02:45.767472+0000","last_undegraded":"2024-02-14T15:02:45.767472+0000","last_fullsized":"2024-02-14T15:02:45.767472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:49.198099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620493+0000","last_change":"2024-02-14T15:00:49.278473+0000","last_active":"2024-02-14T15:02:45.620493+0000","last_peered":"2024-02-14T15:02:45.620493+0000","last_clean":"2024-02-14T15:02:45.620493+0000","last_became_active":"2024-02-14T15:00:49.278145+0000","last_became_peered":"2024-02-14T15:00:49.278145+0000","last_unstale":"2024-02-14T15:02:45.620493+0000","last_undegraded":"2024-02-14T15:02:45.620493+0000","last_fullsized":"2024-02-14T15:02:45.620493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:49:11.718035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767006+0000","last_change":"2024-02-14T15:00:53.581164+0000","last_active":"2024-02-14T15:02:45.767006+0000","last_peered":"2024-02-14T15:02:45.767006+0000","last_clean":"2024-02-14T15:02:45.767006+0000","last_became_active":"2024-02-14T15:00:53.580796+0000","last_became_peered":"2024-02-14T15:00:53.580796+0000","last_unstale":"2024-02-14T15:02:45.767006+0000","last_undegraded":"2024-02-14T15:02:45.767006+0000","last_fullsized":"2024-02-14T15:02:45.767006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:37.245035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766486+0000","last_change":"2024-02-14T15:00:48.271427+0000","last_active":"2024-02-14T15:02:45.766486+0000","last_peered":"2024-02-14T15:02:45.766486+0000","last_clean":"2024-02-14T15:02:45.766486+0000","last_became_active":"2024-02-14T15:00:48.271115+0000","last_became_peered":"2024-02-14T15:00:48.271115+0000","last_unstale":"2024-02-14T15:02:45.766486+0000","last_undegraded":"2024-02-14T15:02:45.766486+0000","last_fullsized":"2024-02-14T15:02:45.766486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:56:01.008058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706430+0000","last_change":"2024-02-14T15:00:49.269964+0000","last_active":"2024-02-14T15:02:45.706430+0000","last_peered":"2024-02-14T15:02:45.706430+0000","last_clean":"2024-02-14T15:02:45.706430+0000","last_became_active":"2024-02-14T15:00:49.269628+0000","last_became_peered":"2024-02-14T15:00:49.269628+0000","last_unstale":"2024-02-14T15:02:45.706430+0000","last_undegraded":"2024-02-14T15:02:45.706430+0000","last_fullsized":"2024-02-14T15:02:45.706430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:04:26.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616880+0000","last_change":"2024-02-14T15:00:53.579843+0000","last_active":"2024-02-14T15:02:45.616880+0000","last_peered":"2024-02-14T15:02:45.616880+0000","last_clean":"2024-02-14T15:02:45.616880+0000","last_became_active":"2024-02-14T15:00:53.579457+0000","last_became_peered":"2024-02-14T15:00:53.579457+0000","last_unstale":"2024-02-14T15:02:45.616880+0000","last_undegraded":"2024-02-14T15:02:45.616880+0000","last_fullsized":"2024-02-14T15:02:45.616880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:34:49.438317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767597+0000","last_change":"2024-02-14T15:00:48.275016+0000","last_active":"2024-02-14T15:02:45.767597+0000","last_peered":"2024-02-14T15:02:45.767597+0000","last_clean":"2024-02-14T15:02:45.767597+0000","last_became_active":"2024-02-14T15:00:48.274636+0000","last_became_peered":"2024-02-14T15:00:48.274636+0000","last_unstale":"2024-02-14T15:02:45.767597+0000","last_undegraded":"2024-02-14T15:02:45.767597+0000","last_fullsized":"2024-02-14T15:02:45.767597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:12:12.961276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706105+0000","last_change":"2024-02-14T15:01:44.602904+0000","last_active":"2024-02-14T15:02:45.706105+0000","last_peered":"2024-02-14T15:02:45.706105+0000","last_clean":"2024-02-14T15:02:45.706105+0000","last_became_active":"2024-02-14T15:01:44.602480+0000","last_became_peered":"2024-02-14T15:01:44.602480+0000","last_unstale":"2024-02-14T15:02:45.706105+0000","last_undegraded":"2024-02-14T15:02:45.706105+0000","last_fullsized":"2024-02-14T15:02:45.706105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:27:33.451581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616988+0000","last_change":"2024-02-14T15:00:53.584068+0000","last_active":"2024-02-14T15:02:45.616988+0000","last_peered":"2024-02-14T15:02:45.616988+0000","last_clean":"2024-02-14T15:02:45.616988+0000","last_became_active":"2024-02-14T15:00:53.583263+0000","last_became_peered":"2024-02-14T15:00:53.583263+0000","last_unstale":"2024-02-14T15:02:45.616988+0000","last_undegraded":"2024-02-14T15:02:45.616988+0000","last_fullsized":"2024-02-14T15:02:45.616988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:15.726442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617918+0000","last_change":"2024-02-14T15:00:49.271265+0000","last_active":"2024-02-14T15:02:45.617918+0000","last_peered":"2024-02-14T15:02:45.617918+0000","last_clean":"2024-02-14T15:02:45.617918+0000","last_became_active":"2024-02-14T15:00:49.270931+0000","last_became_peered":"2024-02-14T15:00:49.270931+0000","last_unstale":"2024-02-14T15:02:45.617918+0000","last_undegraded":"2024-02-14T15:02:45.617918+0000","last_fullsized":"2024-02-14T15:02:45.617918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:15:40.570730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.743769+0000","last_change":"2024-02-14T15:00:48.286669+0000","last_active":"2024-02-14T15:03:09.743769+0000","last_peered":"2024-02-14T15:03:09.743769+0000","last_clean":"2024-02-14T15:03:09.743769+0000","last_became_active":"2024-02-14T15:00:48.286252+0000","last_became_peered":"2024-02-14T15:00:48.286252+0000","last_unstale":"2024-02-14T15:03:09.743769+0000","last_undegraded":"2024-02-14T15:03:09.743769+0000","last_fullsized":"2024-02-14T15:03:09.743769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:40.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767025+0000","last_change":"2024-02-14T15:00:53.583353+0000","last_active":"2024-02-14T15:02:45.767025+0000","last_peered":"2024-02-14T15:02:45.767025+0000","last_clean":"2024-02-14T15:02:45.767025+0000","last_became_active":"2024-02-14T15:00:53.582449+0000","last_became_peered":"2024-02-14T15:00:53.582449+0000","last_unstale":"2024-02-14T15:02:45.767025+0000","last_undegraded":"2024-02-14T15:02:45.767025+0000","last_fullsized":"2024-02-14T15:02:45.767025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:09:01.514603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713191+0000","last_change":"2024-02-14T15:00:49.285579+0000","last_active":"2024-02-14T15:02:45.713191+0000","last_peered":"2024-02-14T15:02:45.713191+0000","last_clean":"2024-02-14T15:02:45.713191+0000","last_became_active":"2024-02-14T15:00:49.283741+0000","last_became_peered":"2024-02-14T15:00:49.283741+0000","last_unstale":"2024-02-14T15:02:45.713191+0000","last_undegraded":"2024-02-14T15:02:45.713191+0000","last_fullsized":"2024-02-14T15:02:45.713191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:05.813218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617501+0000","last_change":"2024-02-14T15:00:48.264434+0000","last_active":"2024-02-14T15:02:45.617501+0000","last_peered":"2024-02-14T15:02:45.617501+0000","last_clean":"2024-02-14T15:02:45.617501+0000","last_became_active":"2024-02-14T15:00:48.264048+0000","last_became_peered":"2024-02-14T15:00:48.264048+0000","last_unstale":"2024-02-14T15:02:45.617501+0000","last_undegraded":"2024-02-14T15:02:45.617501+0000","last_fullsized":"2024-02-14T15:02:45.617501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:52:32.498128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766621+0000","last_change":"2024-02-14T14:59:36.539524+0000","last_active":"2024-02-14T15:02:45.766621+0000","last_peered":"2024-02-14T15:02:45.766621+0000","last_clean":"2024-02-14T15:02:45.766621+0000","last_became_active":"2024-02-14T14:59:36.538922+0000","last_became_peered":"2024-02-14T14:59:36.538922+0000","last_unstale":"2024-02-14T15:02:45.766621+0000","last_undegraded":"2024-02-14T15:02:45.766621+0000","last_fullsized":"2024-02-14T15:02:45.766621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_clean_scrub_stamp":"2024-02-14T14:56:09.567049+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:58.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.707765+0000","last_change":"2024-02-14T15:02:45.707287+0000","last_active":"2024-02-14T15:02:45.707765+0000","last_peered":"2024-02-14T15:02:45.707765+0000","last_clean":"2024-02-14T15:02:45.707765+0000","last_became_active":"2024-02-14T15:02:45.706798+0000","last_became_peered":"2024-02-14T15:02:45.706798+0000","last_unstale":"2024-02-14T15:02:45.707765+0000","last_undegraded":"2024-02-14T15:02:45.707765+0000","last_fullsized":"2024-02-14T15:02:45.707765+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:12.237714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616413+0000","last_change":"2024-02-14T15:00:49.272692+0000","last_active":"2024-02-14T15:02:45.616413+0000","last_peered":"2024-02-14T15:02:45.616413+0000","last_clean":"2024-02-14T15:02:45.616413+0000","last_became_active":"2024-02-14T15:00:49.272353+0000","last_became_peered":"2024-02-14T15:00:49.272353+0000","last_unstale":"2024-02-14T15:02:45.616413+0000","last_undegraded":"2024-02-14T15:02:45.616413+0000","last_fullsized":"2024-02-14T15:02:45.616413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:47.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766756+0000","last_change":"2024-02-14T15:00:48.272470+0000","last_active":"2024-02-14T15:02:45.766756+0000","last_peered":"2024-02-14T15:02:45.766756+0000","last_clean":"2024-02-14T15:02:45.766756+0000","last_became_active":"2024-02-14T15:00:48.272126+0000","last_became_peered":"2024-02-14T15:00:48.272126+0000","last_unstale":"2024-02-14T15:02:45.766756+0000","last_undegraded":"2024-02-14T15:02:45.766756+0000","last_fullsized":"2024-02-14T15:02:45.766756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:21:02.601411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620193+0000","last_change":"2024-02-14T15:00:53.585950+0000","last_active":"2024-02-14T15:02:45.620193+0000","last_peered":"2024-02-14T15:02:45.620193+0000","last_clean":"2024-02-14T15:02:45.620193+0000","last_became_active":"2024-02-14T15:00:53.585547+0000","last_became_peered":"2024-02-14T15:00:53.585547+0000","last_unstale":"2024-02-14T15:02:45.620193+0000","last_undegraded":"2024-02-14T15:02:45.620193+0000","last_fullsized":"2024-02-14T15:02:45.620193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:22.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767371+0000","last_change":"2024-02-14T15:00:48.274650+0000","last_active":"2024-02-14T15:02:45.767371+0000","last_peered":"2024-02-14T15:02:45.767371+0000","last_clean":"2024-02-14T15:02:45.767371+0000","last_became_active":"2024-02-14T15:00:48.274085+0000","last_became_peered":"2024-02-14T15:00:48.274085+0000","last_unstale":"2024-02-14T15:02:45.767371+0000","last_undegraded":"2024-02-14T15:02:45.767371+0000","last_fullsized":"2024-02-14T15:02:45.767371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:53.307596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617022+0000","last_change":"2024-02-14T15:00:49.268419+0000","last_active":"2024-02-14T15:02:45.617022+0000","last_peered":"2024-02-14T15:02:45.617022+0000","last_clean":"2024-02-14T15:02:45.617022+0000","last_became_active":"2024-02-14T15:00:49.268120+0000","last_became_peered":"2024-02-14T15:00:49.268120+0000","last_unstale":"2024-02-14T15:02:45.617022+0000","last_undegraded":"2024-02-14T15:02:45.617022+0000","last_fullsized":"2024-02-14T15:02:45.617022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:10:36.319337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713131+0000","last_change":"2024-02-14T15:00:53.586151+0000","last_active":"2024-02-14T15:02:45.713131+0000","last_peered":"2024-02-14T15:02:45.713131+0000","last_clean":"2024-02-14T15:02:45.713131+0000","last_became_active":"2024-02-14T15:00:53.585727+0000","last_became_peered":"2024-02-14T15:00:53.585727+0000","last_unstale":"2024-02-14T15:02:45.713131+0000","last_undegraded":"2024-02-14T15:02:45.713131+0000","last_fullsized":"2024-02-14T15:02:45.713131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:40:03.741063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":53,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766847+0000","last_change":"2024-02-14T15:01:44.615028+0000","last_active":"2024-02-14T15:02:45.766847+0000","last_peered":"2024-02-14T15:02:45.766847+0000","last_clean":"2024-02-14T15:02:45.766847+0000","last_became_active":"2024-02-14T15:01:44.606281+0000","last_became_peered":"2024-02-14T15:01:44.606281+0000","last_unstale":"2024-02-14T15:02:45.766847+0000","last_undegraded":"2024-02-14T15:02:45.766847+0000","last_fullsized":"2024-02-14T15:02:45.766847+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:37.706659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713701+0000","last_change":"2024-02-14T15:00:49.285726+0000","last_active":"2024-02-14T15:02:45.713701+0000","last_peered":"2024-02-14T15:02:45.713701+0000","last_clean":"2024-02-14T15:02:45.713701+0000","last_became_active":"2024-02-14T15:00:49.283728+0000","last_became_peered":"2024-02-14T15:00:49.283728+0000","last_unstale":"2024-02-14T15:02:45.713701+0000","last_undegraded":"2024-02-14T15:02:45.713701+0000","last_fullsized":"2024-02-14T15:02:45.713701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.764048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620850+0000","last_change":"2024-02-14T15:00:53.583644+0000","last_active":"2024-02-14T15:02:45.620850+0000","last_peered":"2024-02-14T15:02:45.620850+0000","last_clean":"2024-02-14T15:02:45.620850+0000","last_became_active":"2024-02-14T15:00:53.582801+0000","last_became_peered":"2024-02-14T15:00:53.582801+0000","last_unstale":"2024-02-14T15:02:45.620850+0000","last_undegraded":"2024-02-14T15:02:45.620850+0000","last_fullsized":"2024-02-14T15:02:45.620850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:41:40.622091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617759+0000","last_change":"2024-02-14T15:00:48.270231+0000","last_active":"2024-02-14T15:02:45.617759+0000","last_peered":"2024-02-14T15:02:45.617759+0000","last_clean":"2024-02-14T15:02:45.617759+0000","last_became_active":"2024-02-14T15:00:48.269880+0000","last_became_peered":"2024-02-14T15:00:48.269880+0000","last_unstale":"2024-02-14T15:02:45.617759+0000","last_undegraded":"2024-02-14T15:02:45.617759+0000","last_fullsized":"2024-02-14T15:02:45.617759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:08:47.376417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512291+0000","last_change":"2024-02-14T15:00:49.277227+0000","last_active":"2024-02-14T15:02:46.512291+0000","last_peered":"2024-02-14T15:02:46.512291+0000","last_clean":"2024-02-14T15:02:46.512291+0000","last_became_active":"2024-02-14T15:00:49.276904+0000","last_became_peered":"2024-02-14T15:00:49.276904+0000","last_unstale":"2024-02-14T15:02:46.512291+0000","last_undegraded":"2024-02-14T15:02:46.512291+0000","last_fullsized":"2024-02-14T15:02:46.512291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:01.091327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617891+0000","last_change":"2024-02-14T15:00:53.587855+0000","last_active":"2024-02-14T15:02:45.617891+0000","last_peered":"2024-02-14T15:02:45.617891+0000","last_clean":"2024-02-14T15:02:45.617891+0000","last_became_active":"2024-02-14T15:00:53.586313+0000","last_became_peered":"2024-02-14T15:00:53.586313+0000","last_unstale":"2024-02-14T15:02:45.617891+0000","last_undegraded":"2024-02-14T15:02:45.617891+0000","last_fullsized":"2024-02-14T15:02:45.617891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:36:41.279582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766114+0000","last_change":"2024-02-14T15:00:48.267385+0000","last_active":"2024-02-14T15:02:45.766114+0000","last_peered":"2024-02-14T15:02:45.766114+0000","last_clean":"2024-02-14T15:02:45.766114+0000","last_became_active":"2024-02-14T15:00:48.267036+0000","last_became_peered":"2024-02-14T15:00:48.267036+0000","last_unstale":"2024-02-14T15:02:45.766114+0000","last_undegraded":"2024-02-14T15:02:45.766114+0000","last_fullsized":"2024-02-14T15:02:45.766114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:29:24.840030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766956+0000","last_change":"2024-02-14T15:00:49.272233+0000","last_active":"2024-02-14T15:02:45.766956+0000","last_peered":"2024-02-14T15:02:45.766956+0000","last_clean":"2024-02-14T15:02:45.766956+0000","last_became_active":"2024-02-14T15:00:49.271813+0000","last_became_peered":"2024-02-14T15:00:49.271813+0000","last_unstale":"2024-02-14T15:02:45.766956+0000","last_undegraded":"2024-02-14T15:02:45.766956+0000","last_fullsized":"2024-02-14T15:02:45.766956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:30:32.521443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766727+0000","last_change":"2024-02-14T15:00:53.583385+0000","last_active":"2024-02-14T15:02:45.766727+0000","last_peered":"2024-02-14T15:02:45.766727+0000","last_clean":"2024-02-14T15:02:45.766727+0000","last_became_active":"2024-02-14T15:00:53.583023+0000","last_became_peered":"2024-02-14T15:00:53.583023+0000","last_unstale":"2024-02-14T15:02:45.766727+0000","last_undegraded":"2024-02-14T15:02:45.766727+0000","last_fullsized":"2024-02-14T15:02:45.766727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:40:53.500939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713137+0000","last_change":"2024-02-14T15:00:48.261376+0000","last_active":"2024-02-14T15:02:45.713137+0000","last_peered":"2024-02-14T15:02:45.713137+0000","last_clean":"2024-02-14T15:02:45.713137+0000","last_became_active":"2024-02-14T15:00:48.260461+0000","last_became_peered":"2024-02-14T15:00:48.260461+0000","last_unstale":"2024-02-14T15:02:45.713137+0000","last_undegraded":"2024-02-14T15:02:45.713137+0000","last_fullsized":"2024-02-14T15:02:45.713137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:35.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.619887+0000","last_change":"2024-02-14T15:00:49.273117+0000","last_active":"2024-02-14T15:02:45.619887+0000","last_peered":"2024-02-14T15:02:45.619887+0000","last_clean":"2024-02-14T15:02:45.619887+0000","last_became_active":"2024-02-14T15:00:49.272810+0000","last_became_peered":"2024-02-14T15:00:49.272810+0000","last_unstale":"2024-02-14T15:02:45.619887+0000","last_undegraded":"2024-02-14T15:02:45.619887+0000","last_fullsized":"2024-02-14T15:02:45.619887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:37:00.992212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617747+0000","last_change":"2024-02-14T15:00:53.583972+0000","last_active":"2024-02-14T15:02:45.617747+0000","last_peered":"2024-02-14T15:02:45.617747+0000","last_clean":"2024-02-14T15:02:45.617747+0000","last_became_active":"2024-02-14T15:00:53.583563+0000","last_became_peered":"2024-02-14T15:00:53.583563+0000","last_unstale":"2024-02-14T15:02:45.617747+0000","last_undegraded":"2024-02-14T15:02:45.617747+0000","last_fullsized":"2024-02-14T15:02:45.617747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:08:55.811082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621077+0000","last_change":"2024-02-14T15:00:48.270363+0000","last_active":"2024-02-14T15:02:45.621077+0000","last_peered":"2024-02-14T15:02:45.621077+0000","last_clean":"2024-02-14T15:02:45.621077+0000","last_became_active":"2024-02-14T15:00:48.270030+0000","last_became_peered":"2024-02-14T15:00:48.270030+0000","last_unstale":"2024-02-14T15:02:45.621077+0000","last_undegraded":"2024-02-14T15:02:45.621077+0000","last_fullsized":"2024-02-14T15:02:45.621077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:28.885451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713465+0000","last_change":"2024-02-14T15:00:49.285647+0000","last_active":"2024-02-14T15:02:45.713465+0000","last_peered":"2024-02-14T15:02:45.713465+0000","last_clean":"2024-02-14T15:02:45.713465+0000","last_became_active":"2024-02-14T15:00:49.285267+0000","last_became_peered":"2024-02-14T15:00:49.285267+0000","last_unstale":"2024-02-14T15:02:45.713465+0000","last_undegraded":"2024-02-14T15:02:45.713465+0000","last_fullsized":"2024-02-14T15:02:45.713465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:03.143177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617467+0000","last_change":"2024-02-14T15:00:53.591029+0000","last_active":"2024-02-14T15:02:45.617467+0000","last_peered":"2024-02-14T15:02:45.617467+0000","last_clean":"2024-02-14T15:02:45.617467+0000","last_became_active":"2024-02-14T15:00:53.590570+0000","last_became_peered":"2024-02-14T15:00:53.590570+0000","last_unstale":"2024-02-14T15:02:45.617467+0000","last_undegraded":"2024-02-14T15:02:45.617467+0000","last_fullsized":"2024-02-14T15:02:45.617467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:41.809227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620092+0000","last_change":"2024-02-14T15:01:44.611339+0000","last_active":"2024-02-14T15:02:45.620092+0000","last_peered":"2024-02-14T15:02:45.620092+0000","last_clean":"2024-02-14T15:02:45.620092+0000","last_became_active":"2024-02-14T15:01:44.605631+0000","last_became_peered":"2024-02-14T15:01:44.605631+0000","last_unstale":"2024-02-14T15:02:45.620092+0000","last_undegraded":"2024-02-14T15:02:45.620092+0000","last_fullsized":"2024-02-14T15:02:45.620092+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:24:19.853345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767748+0000","last_change":"2024-02-14T15:00:49.276346+0000","last_active":"2024-02-14T15:02:45.767748+0000","last_peered":"2024-02-14T15:02:45.767748+0000","last_clean":"2024-02-14T15:02:45.767748+0000","last_became_active":"2024-02-14T15:00:49.276000+0000","last_became_peered":"2024-02-14T15:00:49.276000+0000","last_unstale":"2024-02-14T15:02:45.767748+0000","last_undegraded":"2024-02-14T15:02:45.767748+0000","last_fullsized":"2024-02-14T15:02:45.767748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:14:03.811981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766244+0000","last_change":"2024-02-14T15:00:53.587021+0000","last_active":"2024-02-14T15:02:45.766244+0000","last_peered":"2024-02-14T15:02:45.766244+0000","last_clean":"2024-02-14T15:02:45.766244+0000","last_became_active":"2024-02-14T15:00:53.586492+0000","last_became_peered":"2024-02-14T15:00:53.586492+0000","last_unstale":"2024-02-14T15:02:45.766244+0000","last_undegraded":"2024-02-14T15:02:45.766244+0000","last_fullsized":"2024-02-14T15:02:45.766244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:51:02.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766373+0000","last_change":"2024-02-14T15:01:44.611916+0000","last_active":"2024-02-14T15:02:45.766373+0000","last_peered":"2024-02-14T15:02:45.766373+0000","last_clean":"2024-02-14T15:02:45.766373+0000","last_became_active":"2024-02-14T15:01:44.605791+0000","last_became_peered":"2024-02-14T15:01:44.605791+0000","last_unstale":"2024-02-14T15:02:45.766373+0000","last_undegraded":"2024-02-14T15:02:45.766373+0000","last_fullsized":"2024-02-14T15:02:45.766373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:50:54.066205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767679+0000","last_change":"2024-02-14T15:00:49.276110+0000","last_active":"2024-02-14T15:02:45.767679+0000","last_peered":"2024-02-14T15:02:45.767679+0000","last_clean":"2024-02-14T15:02:45.767679+0000","last_became_active":"2024-02-14T15:00:49.275729+0000","last_became_peered":"2024-02-14T15:00:49.275729+0000","last_unstale":"2024-02-14T15:02:45.767679+0000","last_undegraded":"2024-02-14T15:02:45.767679+0000","last_fullsized":"2024-02-14T15:02:45.767679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:07:29.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511830+0000","last_change":"2024-02-14T15:00:53.582061+0000","last_active":"2024-02-14T15:02:46.511830+0000","last_peered":"2024-02-14T15:02:46.511830+0000","last_clean":"2024-02-14T15:02:46.511830+0000","last_became_active":"2024-02-14T15:00:53.581659+0000","last_became_peered":"2024-02-14T15:00:53.581659+0000","last_unstale":"2024-02-14T15:02:46.511830+0000","last_undegraded":"2024-02-14T15:02:46.511830+0000","last_fullsized":"2024-02-14T15:02:46.511830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:13.814496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618666+0000","last_change":"2024-02-14T15:00:48.283300+0000","last_active":"2024-02-14T15:02:45.618666+0000","last_peered":"2024-02-14T15:02:45.618666+0000","last_clean":"2024-02-14T15:02:45.618666+0000","last_became_active":"2024-02-14T15:00:48.282939+0000","last_became_peered":"2024-02-14T15:00:48.282939+0000","last_unstale":"2024-02-14T15:02:45.618666+0000","last_undegraded":"2024-02-14T15:02:45.618666+0000","last_fullsized":"2024-02-14T15:02:45.618666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:56:52.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767479+0000","last_change":"2024-02-14T15:00:49.273435+0000","last_active":"2024-02-14T15:02:45.767479+0000","last_peered":"2024-02-14T15:02:45.767479+0000","last_clean":"2024-02-14T15:02:45.767479+0000","last_became_active":"2024-02-14T15:00:49.272980+0000","last_became_peered":"2024-02-14T15:00:49.272980+0000","last_unstale":"2024-02-14T15:02:45.767479+0000","last_undegraded":"2024-02-14T15:02:45.767479+0000","last_fullsized":"2024-02-14T15:02:45.767479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:19:05.294943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617563+0000","last_change":"2024-02-14T15:00:53.583509+0000","last_active":"2024-02-14T15:02:45.617563+0000","last_peered":"2024-02-14T15:02:45.617563+0000","last_clean":"2024-02-14T15:02:45.617563+0000","last_became_active":"2024-02-14T15:00:53.583148+0000","last_became_peered":"2024-02-14T15:00:53.583148+0000","last_unstale":"2024-02-14T15:02:45.617563+0000","last_undegraded":"2024-02-14T15:02:45.617563+0000","last_fullsized":"2024-02-14T15:02:45.617563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:06:47.059643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617044+0000","last_change":"2024-02-14T15:00:48.286707+0000","last_active":"2024-02-14T15:02:45.617044+0000","last_peered":"2024-02-14T15:02:45.617044+0000","last_clean":"2024-02-14T15:02:45.617044+0000","last_became_active":"2024-02-14T15:00:48.286355+0000","last_became_peered":"2024-02-14T15:00:48.286355+0000","last_unstale":"2024-02-14T15:02:45.617044+0000","last_undegraded":"2024-02-14T15:02:45.617044+0000","last_fullsized":"2024-02-14T15:02:45.617044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:48:31.024464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767643+0000","last_change":"2024-02-14T15:00:49.271598+0000","last_active":"2024-02-14T15:02:45.767643+0000","last_peered":"2024-02-14T15:02:45.767643+0000","last_clean":"2024-02-14T15:02:45.767643+0000","last_became_active":"2024-02-14T15:00:49.270572+0000","last_became_peered":"2024-02-14T15:00:49.270572+0000","last_unstale":"2024-02-14T15:02:45.767643+0000","last_undegraded":"2024-02-14T15:02:45.767643+0000","last_fullsized":"2024-02-14T15:02:45.767643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:12.666557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.619943+0000","last_change":"2024-02-14T15:00:49.276697+0000","last_active":"2024-02-14T15:02:45.619943+0000","last_peered":"2024-02-14T15:02:45.619943+0000","last_clean":"2024-02-14T15:02:45.619943+0000","last_became_active":"2024-02-14T15:00:49.276279+0000","last_became_peered":"2024-02-14T15:00:49.276279+0000","last_unstale":"2024-02-14T15:02:45.619943+0000","last_undegraded":"2024-02-14T15:02:45.619943+0000","last_fullsized":"2024-02-14T15:02:45.619943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:11:31.824974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621107+0000","last_change":"2024-02-14T15:01:44.607605+0000","last_active":"2024-02-14T15:02:45.621107+0000","last_peered":"2024-02-14T15:02:45.621107+0000","last_clean":"2024-02-14T15:02:45.621107+0000","last_became_active":"2024-02-14T15:01:44.607171+0000","last_became_peered":"2024-02-14T15:01:44.607171+0000","last_unstale":"2024-02-14T15:02:45.621107+0000","last_undegraded":"2024-02-14T15:02:45.621107+0000","last_fullsized":"2024-02-14T15:02:45.621107+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:30:20.503814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621004+0000","last_change":"2024-02-14T15:00:49.274219+0000","last_active":"2024-02-14T15:02:45.621004+0000","last_peered":"2024-02-14T15:02:45.621004+0000","last_clean":"2024-02-14T15:02:45.621004+0000","last_became_active":"2024-02-14T15:00:49.273833+0000","last_became_peered":"2024-02-14T15:00:49.273833+0000","last_unstale":"2024-02-14T15:02:45.621004+0000","last_undegraded":"2024-02-14T15:02:45.621004+0000","last_fullsized":"2024-02-14T15:02:45.621004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:06:02.720273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512067+0000","last_change":"2024-02-14T15:00:49.273576+0000","last_active":"2024-02-14T15:02:46.512067+0000","last_peered":"2024-02-14T15:02:46.512067+0000","last_clean":"2024-02-14T15:02:46.512067+0000","last_became_active":"2024-02-14T15:00:49.272920+0000","last_became_peered":"2024-02-14T15:00:49.272920+0000","last_unstale":"2024-02-14T15:02:46.512067+0000","last_undegraded":"2024-02-14T15:02:46.512067+0000","last_fullsized":"2024-02-14T15:02:46.512067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:22:03.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617100+0000","last_change":"2024-02-14T15:00:49.274549+0000","last_active":"2024-02-14T15:02:45.617100+0000","last_peered":"2024-02-14T15:02:45.617100+0000","last_clean":"2024-02-14T15:02:45.617100+0000","last_became_active":"2024-02-14T15:00:49.274097+0000","last_became_peered":"2024-02-14T15:00:49.274097+0000","last_unstale":"2024-02-14T15:02:45.617100+0000","last_undegraded":"2024-02-14T15:02:45.617100+0000","last_fullsized":"2024-02-14T15:02:45.617100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:47:44.017607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620430+0000","last_change":"2024-02-14T15:00:49.275699+0000","last_active":"2024-02-14T15:02:45.620430+0000","last_peered":"2024-02-14T15:02:45.620430+0000","last_clean":"2024-02-14T15:02:45.620430+0000","last_became_active":"2024-02-14T15:00:49.275361+0000","last_became_peered":"2024-02-14T15:00:49.275361+0000","last_unstale":"2024-02-14T15:02:45.620430+0000","last_undegraded":"2024-02-14T15:02:45.620430+0000","last_fullsized":"2024-02-14T15:02:45.620430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:34:02.638758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621841+0000","last_change":"2024-02-14T15:00:49.275024+0000","last_active":"2024-02-14T15:02:45.621841+0000","last_peered":"2024-02-14T15:02:45.621841+0000","last_clean":"2024-02-14T15:02:45.621841+0000","last_became_active":"2024-02-14T15:00:49.274710+0000","last_became_peered":"2024-02-14T15:00:49.274710+0000","last_unstale":"2024-02-14T15:02:45.621841+0000","last_undegraded":"2024-02-14T15:02:45.621841+0000","last_fullsized":"2024-02-14T15:02:45.621841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:42.471112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713794+0000","last_change":"2024-02-14T15:00:49.282638+0000","last_active":"2024-02-14T15:02:45.713794+0000","last_peered":"2024-02-14T15:02:45.713794+0000","last_clean":"2024-02-14T15:02:45.713794+0000","last_became_active":"2024-02-14T15:00:49.280223+0000","last_became_peered":"2024-02-14T15:00:49.280223+0000","last_unstale":"2024-02-14T15:02:45.713794+0000","last_undegraded":"2024-02-14T15:02:45.713794+0000","last_fullsized":"2024-02-14T15:02:45.713794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:40.250242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617680+0000","last_change":"2024-02-14T15:00:49.273025+0000","last_active":"2024-02-14T15:02:45.617680+0000","last_peered":"2024-02-14T15:02:45.617680+0000","last_clean":"2024-02-14T15:02:45.617680+0000","last_became_active":"2024-02-14T15:00:49.272700+0000","last_became_peered":"2024-02-14T15:00:49.272700+0000","last_unstale":"2024-02-14T15:02:45.617680+0000","last_undegraded":"2024-02-14T15:02:45.617680+0000","last_fullsized":"2024-02-14T15:02:45.617680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:08.677428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766013+0000","last_change":"2024-02-14T15:01:44.605957+0000","last_active":"2024-02-14T15:02:45.766013+0000","last_peered":"2024-02-14T15:02:45.766013+0000","last_clean":"2024-02-14T15:02:45.766013+0000","last_became_active":"2024-02-14T15:01:44.605398+0000","last_became_peered":"2024-02-14T15:01:44.605398+0000","last_unstale":"2024-02-14T15:02:45.766013+0000","last_undegraded":"2024-02-14T15:02:45.766013+0000","last_fullsized":"2024-02-14T15:02:45.766013+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:50:43.667496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766620+0000","last_change":"2024-02-14T15:00:49.282178+0000","last_active":"2024-02-14T15:02:45.766620+0000","last_peered":"2024-02-14T15:02:45.766620+0000","last_clean":"2024-02-14T15:02:45.766620+0000","last_became_active":"2024-02-14T15:00:49.281804+0000","last_became_peered":"2024-02-14T15:00:49.281804+0000","last_unstale":"2024-02-14T15:02:45.766620+0000","last_undegraded":"2024-02-14T15:02:45.766620+0000","last_fullsized":"2024-02-14T15:02:45.766620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:58.672808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617022+0000","last_change":"2024-02-14T15:00:49.269882+0000","last_active":"2024-02-14T15:02:45.617022+0000","last_peered":"2024-02-14T15:02:45.617022+0000","last_clean":"2024-02-14T15:02:45.617022+0000","last_became_active":"2024-02-14T15:00:49.269489+0000","last_became_peered":"2024-02-14T15:00:49.269489+0000","last_unstale":"2024-02-14T15:02:45.617022+0000","last_undegraded":"2024-02-14T15:02:45.617022+0000","last_fullsized":"2024-02-14T15:02:45.617022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:57:40.465044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620120+0000","last_change":"2024-02-14T15:00:49.277688+0000","last_active":"2024-02-14T15:02:45.620120+0000","last_peered":"2024-02-14T15:02:45.620120+0000","last_clean":"2024-02-14T15:02:45.620120+0000","last_became_active":"2024-02-14T15:00:49.277348+0000","last_became_peered":"2024-02-14T15:00:49.277348+0000","last_unstale":"2024-02-14T15:02:45.620120+0000","last_undegraded":"2024-02-14T15:02:45.620120+0000","last_fullsized":"2024-02-14T15:02:45.620120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:33.946319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68055,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":710,"num_read_kb":426,"num_write":170,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":63938,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":518659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.623},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.61299999999999999,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.52600000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.46899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.66700000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81200000000000006,"5min":1.155,"15min":1.155},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.42699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59099999999999997,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85099999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83799999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85699999999999998,"5min":1.121,"15min":1.121},"last":0.85699999999999998}]},{"osd":8,"last update":"Wed Feb 14 15:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75700000000000001,"5min":0.875,"15min":0.875},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.89200000000000002,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.875}]},{"osd":9,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92000000000000004,"5min":1.002,"15min":1.002},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98499999999999999,"5min":1.012,"15min":1.012},"last":0.76200000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.871,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.67600000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.95799999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.92100000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1744896,"data_stored":1170064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81399999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.625}]},{"osd":4,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65600000000000003,"15min":0.67300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1.054,"15min":1.054},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60399999999999998,"15min":0.64400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":1.145,"15min":1.145},"last":0.35299999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97399999999999998,"5min":1.032,"15min":1.032},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98199999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":0.82999999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.071,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.59399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0609999999999999,"5min":1.129,"15min":1.129},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81200000000000006,"15min":0.84999999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.872,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54000000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.04,"5min":1.169,"15min":1.169},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.83099999999999996,"15min":0.876},"min":{"1min":0.57199999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96199999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.66400000000000003}]},{"osd":11,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.873,"15min":0.876},"min":{"1min":0.71799999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0409999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.68200000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.083,"5min":1.141,"15min":1.141},"last":0.85099999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":522991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44900000000000001,"15min":0.46200000000000002},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":1.22,"15min":1.22},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.745},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.54200000000000004,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0629999999999999,"5min":1.141,"15min":1.141},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.69999999999999996,"15min":0.73699999999999999},"min":{"1min":0.45600000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97799999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.90600000000000003,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4370000000000001,"5min":19.152000000000001,"15min":19.152000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.31},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.94199999999999995,"15min":0.86299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":18.734999999999999,"15min":18.734999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.93999999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5069999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.94799999999999995,"15min":0.85999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.165,"5min":19.135999999999999,"15min":19.135999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0049999999999999,"5min":1.23,"15min":1.23},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed Feb 14 15:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.76400000000000001,"15min":0.79400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.79800000000000004,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1100000000000001,"5min":1.264,"15min":1.264},"last":0.79600000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.58199999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0609999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96199999999999997,"5min":1.778,"15min":1.778},"last":0.78200000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":518815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80200000000000005,"5min":13.997,"15min":13.997},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79600000000000004}]},{"osd":2,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44,"15min":0.41799999999999998},"min":{"1min":0.36899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64000000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64300000000000002,"15min":0.627},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.53200000000000003},"min":{"1min":0.373,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81200000000000006}]},{"osd":6,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63300000000000001,"15min":0.61599999999999999},"min":{"1min":0.47899999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.93700000000000006,"15min":1.175},"min":{"1min":0.56999999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93200000000000005,"5min":13.618,"15min":13.618},"last":0.86699999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.71699999999999997,"15min":0.68700000000000006},"min":{"1min":0.61499999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.69899999999999995,"15min":0.67100000000000004},"min":{"1min":0.627,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.59599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.68400000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.91100000000000003}]},{"osd":10,"last update":"Wed Feb 14 15:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71299999999999997,"15min":0.67300000000000004},"min":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.94899999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.73599999999999999,"15min":0.69999999999999996},"min":{"1min":0.67600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85599999999999998}]},{"osd":11,"last update":"Wed Feb 14 15:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.66300000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.98599999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.627,"5min":0.501,"15min":0.501},"max":{"1min":0.95099999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.96499999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1115955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71599999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":0.877,"15min":0.877},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.68899999999999995,"15min":0.65000000000000002},"min":{"1min":0.52900000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83199999999999996,"5min":0.872,"15min":0.872},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.58199999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91000000000000003,"5min":0.997,"15min":0.997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85499999999999998,"5min":1.095,"15min":1.095},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.78400000000000003,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:02:47 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.036,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.79300000000000004},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":1.139,"15min":1.139},"last":0.73599999999999999}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.052,"15min":1.052},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.67300000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.0339999999999998,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.075,"5min":1.179,"15min":1.179},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.82699999999999996},"min":{"1min":0.626,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.014,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Feb 14 15:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90400000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.94399999999999995,"15min":0.94799999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.155,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84999999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":514071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.377},"min":{"1min":0.34000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.434,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63800000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.55400000000000005},"min":{"1min":0.47899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80600000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78300000000000003,"15min":0.622},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1259999999999999,"5min":19.181000000000001,"15min":19.181000000000001},"last":0.626}]},{"osd":2,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.626,"15min":0.58499999999999996},"min":{"1min":0.53000000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85099999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.44600000000000001},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.48499999999999999},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.79000000000000004,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":19.167000000000002,"15min":19.167000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.65800000000000003,"15min":0.623},"min":{"1min":0.52900000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.56499999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.99199999999999999,"5min":1.107,"15min":1.107},"last":0.66800000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0449999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.266,"15min":1.266},"last":0.76700000000000002}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68899999999999995,"15min":0.72499999999999998},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.92300000000000004,"5min":0.996,"15min":0.996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78600000000000003,"15min":0.77800000000000002},"min":{"1min":0.64100000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.748},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79700000000000004},"min":{"1min":0.57099999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.024,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.79500000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.66700000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":578476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.64900000000000002,"15min":0.53800000000000003},"min":{"1min":0.38300000000000001,"5min":0.33900000000000002,"15min":0.32800000000000001},"max":{"1min":6.5899999999999999,"5min":6.5899999999999999,"15min":6.5899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.41599999999999998,"5min":0.33100000000000002,"15min":0.309},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.53600000000000003},"min":{"1min":0.47099999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.754},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.80800000000000005,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":18.332000000000001,"15min":18.332000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:02:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.315},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.38200000000000001},"max":{"1min":1.0349999999999999,"5min":1.046,"15min":1.046},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.68799999999999994,"15min":0.59399999999999997},"min":{"1min":0.51300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70699999999999996,"15min":0.621},"min":{"1min":0.629,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":1.016,"15min":1.016},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61299999999999999,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89400000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.73199999999999998,"15min":0.65800000000000003},"min":{"1min":0.53000000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65500000000000003,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0209999999999999,"5min":1.077,"15min":1.077},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55700000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68500000000000005,"15min":0.66400000000000003},"min":{"1min":0.56299999999999994,"5min":0.376,"15min":0.376},"max":{"1min":0.996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.80000000000000004,"15min":0.748},"min":{"1min":0.69199999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0820000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67800000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.0069999999999999,"5min":1.044,"15min":1.044},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.86399999999999999,"15min":0.84799999999999998},"min":{"1min":0.67800000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.65600000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0629999999999999,"5min":1.107,"15min":1.107},"last":0.81499999999999995}]},{"osd":11,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.93200000000000005,"15min":0.92100000000000004},"min":{"1min":0.84699999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.115,"5min":1.133,"15min":1.133},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.83199999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.84999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":515459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.31,"15min":0.253},"max":{"1min":0.76300000000000001,"5min":1.002,"15min":1.002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57499999999999996,"15min":0.52100000000000002},"min":{"1min":0.45200000000000001,"5min":0.32300000000000001,"15min":0.28199999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.53000000000000003},"min":{"1min":0.48999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72199999999999998,"5min":1.002,"15min":1.002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.58699999999999997,"15min":0.52500000000000002},"min":{"1min":0.47399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.5},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.87,"5min":1.694,"15min":1.694},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51300000000000001,"15min":0.53100000000000003},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76900000000000002,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85599999999999998,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.36399999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84399999999999997,"5min":1.139,"15min":1.139},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.1459999999999999,"5min":0.76000000000000001,"15min":0.69599999999999995},"min":{"1min":0.53500000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.8949999999999996,"5min":9.8949999999999996,"15min":9.8949999999999996},"last":0.53900000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84599999999999997,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.498,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.69299999999999995},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.109,"5min":1.153,"15min":1.153},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0649999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58699999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.60499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.63900000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.67000000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.751,"15min":0.748},"min":{"1min":0.61199999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.802,"15min":1.802},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75900000000000001,"15min":0.73299999999999998},"min":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.97399999999999998,"5min":1.242,"15min":1.242},"last":0.75900000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":514749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77400000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.501}]},{"osd":1,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58099999999999996,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.55800000000000005},"min":{"1min":0.45400000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.874}]},{"osd":5,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.46500000000000002,"15min":0.436},"min":{"1min":0.36099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.372,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93799999999999994,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.625},{"interface":"front","average":{"1min":1.284,"5min":0.748,"15min":0.65600000000000003},"min":{"1min":0.54000000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":13.955,"5min":13.955,"15min":13.955},"last":0.66400000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93400000000000005,"5min":1.046,"15min":1.046},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94999999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70999999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.498,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":1.01,"15min":1.01},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.749}]},{"osd":10,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.621,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0409999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.753,"15min":0.71799999999999997},"min":{"1min":0.54900000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.89200000000000002}]},{"osd":11,"last update":"Wed Feb 14 15:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.64700000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76900000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":514265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.375,"15min":0.32700000000000001},"min":{"1min":0.29599999999999999,"5min":0.20399999999999999,"15min":0.191},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.66900000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.76500000000000001,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.33600000000000002},"max":{"1min":0.91000000000000003,"5min":19.896999999999998,"15min":19.896999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.77400000000000002,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.99399999999999999,"5min":19.911999999999999,"15min":19.911999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59499999999999997,"15min":0.53700000000000003},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":0.995,"15min":0.995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60899999999999999,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.751,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.57999999999999996},"min":{"1min":0.46400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.879,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.498}]},{"osd":5,"last update":"Wed Feb 14 15:02:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.80500000000000005,"15min":1.3919999999999999},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":19.879999999999999,"15min":19.879999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.441},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.501,"15min":0.48499999999999999},"min":{"1min":0.33500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.39400000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.56999999999999995}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70499999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64300000000000002,"15min":0.67200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.64500000000000002,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92700000000000005,"5min":1.391,"15min":1.391},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.93600000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.628}]},{"osd":11,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86199999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67800000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":583166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.42199999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72299999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.379}]},{"osd":2,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.56100000000000005,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.91900000000000004,"5min":0.997,"15min":0.997},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89500000000000002,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89100000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65400000000000003,"15min":0.58499999999999996},"min":{"1min":0.52500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.67900000000000005,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.92400000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":0.79000000000000004}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.012,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52700000000000002}]},{"osd":8,"last update":"Wed Feb 14 15:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74399999999999999,"15min":0.71699999999999997},"min":{"1min":0.626,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0629999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.1299999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.84099999999999997}]},{"osd":9,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":1.04,"15min":1.04},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0609999999999999,"5min":1.105,"15min":1.105},"last":0.90200000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.56999999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.012,"5min":1.032,"15min":1.032},"last":0.749}]},{"osd":11,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.876},{"interface":"front","average":{"1min":0.877,"5min":0.872,"15min":0.871},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71699999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1671168,"data_stored":1110229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53600000000000003,"15min":0.49199999999999999},"min":{"1min":0.33700000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.48099999999999998},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79700000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56599999999999995,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.84899999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.52700000000000002},"min":{"1min":0.46500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78300000000000003,"5min":0.876,"15min":0.876},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40600000000000003,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.61199999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41599999999999998,"15min":0.39300000000000002},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.505},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83999999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":0.878,"15min":0.878},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81899999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63200000000000001,"15min":0.58399999999999996},"min":{"1min":0.55100000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82699999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89200000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70999999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60199999999999998,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.56399999999999995},"min":{"1min":0.45200000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95799999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.038,"15min":1.038},"last":0.90200000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.59299999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91900000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75,"15min":0.73099999999999998},"min":{"1min":0.65600000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93000000000000005,"5min":1.032,"15min":1.032},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T15:03:14.344 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-14T15:03:14.344 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-14T15:03:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 -- 172.21.15.17:0/3677820569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0101350 msgr2=0x7f24f01017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 --2- 172.21.15.17:0/3677820569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0101350 0x7f24f01017d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009900 tx=0x7f24e402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 -- 172.21.15.17:0/3677820569 shutdown_connections 2024-02-14T15:03:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 --2- 172.21.15.17:0/3677820569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0101350 0x7f24f01017d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 --2- 172.21.15.17:0/3677820569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f0101d10 0x7f24f01085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 --2- 172.21.15.17:0/3677820569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f0100160 0x7f24f0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 -- 172.21.15.17:0/3677820569 >> 172.21.15.17:0/3677820569 conn(0x7f24f00fb6f0 msgr2=0x7f24f00fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 -- 172.21.15.17:0/3677820569 shutdown_connections 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 -- 172.21.15.17:0/3677820569 wait complete. 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.449+0000 7f24f66b5640 1 Processor -- start 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f66b5640 1 -- start start 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f66b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0100160 0x7f24f0197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f66b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f0101350 0x7f24f0197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f66b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f0101d10 0x7f24f019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f66b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f019ed30 con 0x7f24f0100160 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f66b5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f019eea0 con 0x7f24f0101d10 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f66b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f019f060 con 0x7f24f0101350 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0100160 0x7f24f0197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0100160 0x7f24f0197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59056/0 (socket says 172.21.15.17:59056) 2024-02-14T15:03:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 -- 172.21.15.17:0/2082893484 learned_addr learned my addr 172.21.15.17:0/2082893484 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24f4c52640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f0101d10 0x7f24f019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24ef7fe640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f0101350 0x7f24f0197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 -- 172.21.15.17:0/2082893484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f0101350 msgr2=0x7f24f0197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f0101350 0x7f24f0197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 -- 172.21.15.17:0/2082893484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f0101d10 msgr2=0x7f24f019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f0101d10 0x7f24f019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 -- 172.21.15.17:0/2082893484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4009560 con 0x7f24f0100160 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24ef7fe640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f0101350 0x7f24f0197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.450+0000 7f24effff640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0100160 0x7f24f0197240 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c970 tx=0x7f24e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.451+0000 7f24ed7fa640 1 -- 172.21.15.17:0/2082893484 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0007c10 con 0x7f24f0100160 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.451+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f019f340 con 0x7f24f0100160 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.451+0000 7f24ed7fa640 1 -- 172.21.15.17:0/2082893484 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24e0007d70 con 0x7f24f0100160 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.451+0000 7f24ed7fa640 1 -- 172.21.15.17:0/2082893484 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000f8a0 con 0x7f24f0100160 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.451+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f019f850 con 0x7f24f0100160 2024-02-14T15:03:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.452+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f010bb70 con 0x7f24f0100160 2024-02-14T15:03:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.453+0000 7f24ed7fa640 1 -- 172.21.15.17:0/2082893484 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f24e0018af0 con 0x7f24f0100160 2024-02-14T15:03:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.453+0000 7f24ed7fa640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f24d80a2150 0x7f24d80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.454+0000 7f24ef7fe640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f24d80a2150 0x7f24d80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.454+0000 7f24ed7fa640 1 -- 172.21.15.17:0/2082893484 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f24e0013070 con 0x7f24f0100160 2024-02-14T15:03:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.454+0000 7f24ef7fe640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f24d80a2150 0x7f24d80a4610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24f0198750 tx=0x7f24e4005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:14.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.458+0000 7f24ed7fa640 1 -- 172.21.15.17:0/2082893484 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24e00970b0 con 0x7f24f0100160 2024-02-14T15:03:14.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.635+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24f006a020 con 0x7f24d80a2150 2024-02-14T15:03:14.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.648+0000 7f24ed7fa640 1 -- 172.21.15.17:0/2082893484 <== mgr.14150 v2:172.21.15.17:6800/3796717329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428031 (secure 0 0 0) 0x7f24f006a020 con 0x7f24d80a2150 2024-02-14T15:03:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:14.656 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f24d80a2150 msgr2=0x7f24d80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f24d80a2150 0x7f24d80a4610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24f0198750 tx=0x7f24e4005be0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0100160 msgr2=0x7f24f0197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0100160 0x7f24f0197240 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c970 tx=0x7f24e000ce30 comp rx=0 tx=0).stop 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 shutdown_connections 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f24d80a2150 0x7f24d80a4610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f0101d10 0x7f24f019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f0101350 0x7f24f0197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 --2- 172.21.15.17:0/2082893484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0100160 0x7f24f0197240 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 >> 172.21.15.17:0/2082893484 conn(0x7f24f00fb6f0 msgr2=0x7f24f0069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 shutdown_connections 2024-02-14T15:03:14.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.652+0000 7f24f66b5640 1 -- 172.21.15.17:0/2082893484 wait complete. 2024-02-14T15:03:14.664 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-02-14T15:03:12.696051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658657,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":846,"num_read_kb":545,"num_write":353,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1244674,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388604,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553220,"statfs":{"total":1151940427776,"available":1151542497280,"internally_reserved":0,"allocated":14778368,"data_stored":8176899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008748"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620302+0000","last_change":"2024-02-14T15:00:49.281057+0000","last_active":"2024-02-14T15:02:45.620302+0000","last_peered":"2024-02-14T15:02:45.620302+0000","last_clean":"2024-02-14T15:02:45.620302+0000","last_became_active":"2024-02-14T15:00:49.280642+0000","last_became_peered":"2024-02-14T15:00:49.280642+0000","last_unstale":"2024-02-14T15:02:45.620302+0000","last_undegraded":"2024-02-14T15:02:45.620302+0000","last_fullsized":"2024-02-14T15:02:45.620302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:03.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512248+0000","last_change":"2024-02-14T15:00:49.282794+0000","last_active":"2024-02-14T15:02:46.512248+0000","last_peered":"2024-02-14T15:02:46.512248+0000","last_clean":"2024-02-14T15:02:46.512248+0000","last_became_active":"2024-02-14T15:00:49.281441+0000","last_became_peered":"2024-02-14T15:00:49.281441+0000","last_unstale":"2024-02-14T15:02:46.512248+0000","last_undegraded":"2024-02-14T15:02:46.512248+0000","last_fullsized":"2024-02-14T15:02:46.512248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:57:36.577281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512181+0000","last_change":"2024-02-14T15:00:49.286253+0000","last_active":"2024-02-14T15:02:46.512181+0000","last_peered":"2024-02-14T15:02:46.512181+0000","last_clean":"2024-02-14T15:02:46.512181+0000","last_became_active":"2024-02-14T15:00:49.285950+0000","last_became_peered":"2024-02-14T15:00:49.285950+0000","last_unstale":"2024-02-14T15:02:46.512181+0000","last_undegraded":"2024-02-14T15:02:46.512181+0000","last_fullsized":"2024-02-14T15:02:46.512181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:02:53.395339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617486+0000","last_change":"2024-02-14T15:00:49.272764+0000","last_active":"2024-02-14T15:02:45.617486+0000","last_peered":"2024-02-14T15:02:45.617486+0000","last_clean":"2024-02-14T15:02:45.617486+0000","last_became_active":"2024-02-14T15:00:49.272430+0000","last_became_peered":"2024-02-14T15:00:49.272430+0000","last_unstale":"2024-02-14T15:02:45.617486+0000","last_undegraded":"2024-02-14T15:02:45.617486+0000","last_fullsized":"2024-02-14T15:02:45.617486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:36.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766939+0000","last_change":"2024-02-14T15:00:49.275028+0000","last_active":"2024-02-14T15:02:45.766939+0000","last_peered":"2024-02-14T15:02:45.766939+0000","last_clean":"2024-02-14T15:02:45.766939+0000","last_became_active":"2024-02-14T15:00:49.274686+0000","last_became_peered":"2024-02-14T15:00:49.274686+0000","last_unstale":"2024-02-14T15:02:45.766939+0000","last_undegraded":"2024-02-14T15:02:45.766939+0000","last_fullsized":"2024-02-14T15:02:45.766939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:21:23.465321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.705981+0000","last_change":"2024-02-14T15:00:49.287144+0000","last_active":"2024-02-14T15:02:45.705981+0000","last_peered":"2024-02-14T15:02:45.705981+0000","last_clean":"2024-02-14T15:02:45.705981+0000","last_became_active":"2024-02-14T15:00:49.286767+0000","last_became_peered":"2024-02-14T15:00:49.286767+0000","last_unstale":"2024-02-14T15:02:45.705981+0000","last_undegraded":"2024-02-14T15:02:45.705981+0000","last_fullsized":"2024-02-14T15:02:45.705981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:45:09.391595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767289+0000","last_change":"2024-02-14T15:00:49.274442+0000","last_active":"2024-02-14T15:02:45.767289+0000","last_peered":"2024-02-14T15:02:45.767289+0000","last_clean":"2024-02-14T15:02:45.767289+0000","last_became_active":"2024-02-14T15:00:49.274122+0000","last_became_peered":"2024-02-14T15:00:49.274122+0000","last_unstale":"2024-02-14T15:02:45.767289+0000","last_undegraded":"2024-02-14T15:02:45.767289+0000","last_fullsized":"2024-02-14T15:02:45.767289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:50:55.974889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.712957+0000","last_change":"2024-02-14T15:00:49.285432+0000","last_active":"2024-02-14T15:02:45.712957+0000","last_peered":"2024-02-14T15:02:45.712957+0000","last_clean":"2024-02-14T15:02:45.712957+0000","last_became_active":"2024-02-14T15:00:49.283703+0000","last_became_peered":"2024-02-14T15:00:49.283703+0000","last_unstale":"2024-02-14T15:02:45.712957+0000","last_undegraded":"2024-02-14T15:02:45.712957+0000","last_fullsized":"2024-02-14T15:02:45.712957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:14.429192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767120+0000","last_change":"2024-02-14T15:00:49.283908+0000","last_active":"2024-02-14T15:02:45.767120+0000","last_peered":"2024-02-14T15:02:45.767120+0000","last_clean":"2024-02-14T15:02:45.767120+0000","last_became_active":"2024-02-14T15:00:49.283131+0000","last_became_peered":"2024-02-14T15:00:49.283131+0000","last_unstale":"2024-02-14T15:02:45.767120+0000","last_undegraded":"2024-02-14T15:02:45.767120+0000","last_fullsized":"2024-02-14T15:02:45.767120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:48:42.247624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706471+0000","last_change":"2024-02-14T15:00:49.287229+0000","last_active":"2024-02-14T15:02:45.706471+0000","last_peered":"2024-02-14T15:02:45.706471+0000","last_clean":"2024-02-14T15:02:45.706471+0000","last_became_active":"2024-02-14T15:00:49.286791+0000","last_became_peered":"2024-02-14T15:00:49.286791+0000","last_unstale":"2024-02-14T15:02:45.706471+0000","last_undegraded":"2024-02-14T15:02:45.706471+0000","last_fullsized":"2024-02-14T15:02:45.706471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:33:01.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767467+0000","last_change":"2024-02-14T15:00:49.281669+0000","last_active":"2024-02-14T15:02:45.767467+0000","last_peered":"2024-02-14T15:02:45.767467+0000","last_clean":"2024-02-14T15:02:45.767467+0000","last_became_active":"2024-02-14T15:00:49.281298+0000","last_became_peered":"2024-02-14T15:00:49.281298+0000","last_unstale":"2024-02-14T15:02:45.767467+0000","last_undegraded":"2024-02-14T15:02:45.767467+0000","last_fullsized":"2024-02-14T15:02:45.767467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:54:40.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617437+0000","last_change":"2024-02-14T15:00:49.289679+0000","last_active":"2024-02-14T15:02:45.617437+0000","last_peered":"2024-02-14T15:02:45.617437+0000","last_clean":"2024-02-14T15:02:45.617437+0000","last_became_active":"2024-02-14T15:00:49.288767+0000","last_became_peered":"2024-02-14T15:00:49.288767+0000","last_unstale":"2024-02-14T15:02:45.617437+0000","last_undegraded":"2024-02-14T15:02:45.617437+0000","last_fullsized":"2024-02-14T15:02:45.617437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:16:20.181517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616867+0000","last_change":"2024-02-14T15:00:49.257471+0000","last_active":"2024-02-14T15:02:45.616867+0000","last_peered":"2024-02-14T15:02:45.616867+0000","last_clean":"2024-02-14T15:02:45.616867+0000","last_became_active":"2024-02-14T15:00:49.256766+0000","last_became_peered":"2024-02-14T15:00:49.256766+0000","last_unstale":"2024-02-14T15:02:45.616867+0000","last_undegraded":"2024-02-14T15:02:45.616867+0000","last_fullsized":"2024-02-14T15:02:45.616867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:14:00.525621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511859+0000","last_change":"2024-02-14T15:00:49.278462+0000","last_active":"2024-02-14T15:02:46.511859+0000","last_peered":"2024-02-14T15:02:46.511859+0000","last_clean":"2024-02-14T15:02:46.511859+0000","last_became_active":"2024-02-14T15:00:49.278114+0000","last_became_peered":"2024-02-14T15:00:49.278114+0000","last_unstale":"2024-02-14T15:02:46.511859+0000","last_undegraded":"2024-02-14T15:02:46.511859+0000","last_fullsized":"2024-02-14T15:02:46.511859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:02:16.601359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511865+0000","last_change":"2024-02-14T15:00:49.270653+0000","last_active":"2024-02-14T15:02:46.511865+0000","last_peered":"2024-02-14T15:02:46.511865+0000","last_clean":"2024-02-14T15:02:46.511865+0000","last_became_active":"2024-02-14T15:00:49.270334+0000","last_became_peered":"2024-02-14T15:00:49.270334+0000","last_unstale":"2024-02-14T15:02:46.511865+0000","last_undegraded":"2024-02-14T15:02:46.511865+0000","last_fullsized":"2024-02-14T15:02:46.511865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:13:53.840909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766103+0000","last_change":"2024-02-14T15:00:49.280597+0000","last_active":"2024-02-14T15:02:45.766103+0000","last_peered":"2024-02-14T15:02:45.766103+0000","last_clean":"2024-02-14T15:02:45.766103+0000","last_became_active":"2024-02-14T15:00:49.279612+0000","last_became_peered":"2024-02-14T15:00:49.279612+0000","last_unstale":"2024-02-14T15:02:45.766103+0000","last_undegraded":"2024-02-14T15:02:45.766103+0000","last_fullsized":"2024-02-14T15:02:45.766103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:11.676608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620327+0000","last_change":"2024-02-14T15:00:49.281422+0000","last_active":"2024-02-14T15:02:45.620327+0000","last_peered":"2024-02-14T15:02:45.620327+0000","last_clean":"2024-02-14T15:02:45.620327+0000","last_became_active":"2024-02-14T15:00:49.281048+0000","last_became_peered":"2024-02-14T15:00:49.281048+0000","last_unstale":"2024-02-14T15:02:45.620327+0000","last_undegraded":"2024-02-14T15:02:45.620327+0000","last_fullsized":"2024-02-14T15:02:45.620327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:04.994390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766686+0000","last_change":"2024-02-14T15:00:49.288245+0000","last_active":"2024-02-14T15:02:45.766686+0000","last_peered":"2024-02-14T15:02:45.766686+0000","last_clean":"2024-02-14T15:02:45.766686+0000","last_became_active":"2024-02-14T15:00:49.287908+0000","last_became_peered":"2024-02-14T15:00:49.287908+0000","last_unstale":"2024-02-14T15:02:45.766686+0000","last_undegraded":"2024-02-14T15:02:45.766686+0000","last_fullsized":"2024-02-14T15:02:45.766686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:52.397826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621696+0000","last_change":"2024-02-14T15:00:49.275428+0000","last_active":"2024-02-14T15:02:45.621696+0000","last_peered":"2024-02-14T15:02:45.621696+0000","last_clean":"2024-02-14T15:02:45.621696+0000","last_became_active":"2024-02-14T15:00:49.273464+0000","last_became_peered":"2024-02-14T15:00:49.273464+0000","last_unstale":"2024-02-14T15:02:45.621696+0000","last_undegraded":"2024-02-14T15:02:45.621696+0000","last_fullsized":"2024-02-14T15:02:45.621696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:34.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621145+0000","last_change":"2024-02-14T15:00:53.587219+0000","last_active":"2024-02-14T15:02:45.621145+0000","last_peered":"2024-02-14T15:02:45.621145+0000","last_clean":"2024-02-14T15:02:45.621145+0000","last_became_active":"2024-02-14T15:00:53.586917+0000","last_became_peered":"2024-02-14T15:00:53.586917+0000","last_unstale":"2024-02-14T15:02:45.621145+0000","last_undegraded":"2024-02-14T15:02:45.621145+0000","last_fullsized":"2024-02-14T15:02:45.621145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:39:38.729468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766753+0000","last_change":"2024-02-14T15:00:48.269243+0000","last_active":"2024-02-14T15:02:45.766753+0000","last_peered":"2024-02-14T15:02:45.766753+0000","last_clean":"2024-02-14T15:02:45.766753+0000","last_became_active":"2024-02-14T15:00:48.268883+0000","last_became_peered":"2024-02-14T15:00:48.268883+0000","last_unstale":"2024-02-14T15:02:45.766753+0000","last_undegraded":"2024-02-14T15:02:45.766753+0000","last_fullsized":"2024-02-14T15:02:45.766753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:08.530092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713654+0000","last_change":"2024-02-14T15:00:49.282458+0000","last_active":"2024-02-14T15:02:45.713654+0000","last_peered":"2024-02-14T15:02:45.713654+0000","last_clean":"2024-02-14T15:02:45.713654+0000","last_became_active":"2024-02-14T15:00:49.281982+0000","last_became_peered":"2024-02-14T15:00:49.281982+0000","last_unstale":"2024-02-14T15:02:45.713654+0000","last_undegraded":"2024-02-14T15:02:45.713654+0000","last_fullsized":"2024-02-14T15:02:45.713654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:51:33.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617171+0000","last_change":"2024-02-14T15:00:53.586578+0000","last_active":"2024-02-14T15:02:45.617171+0000","last_peered":"2024-02-14T15:02:45.617171+0000","last_clean":"2024-02-14T15:02:45.617171+0000","last_became_active":"2024-02-14T15:00:53.586109+0000","last_became_peered":"2024-02-14T15:00:53.586109+0000","last_unstale":"2024-02-14T15:02:45.617171+0000","last_undegraded":"2024-02-14T15:02:45.617171+0000","last_fullsized":"2024-02-14T15:02:45.617171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:16:07.332435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713729+0000","last_change":"2024-02-14T15:00:48.281953+0000","last_active":"2024-02-14T15:02:45.713729+0000","last_peered":"2024-02-14T15:02:45.713729+0000","last_clean":"2024-02-14T15:02:45.713729+0000","last_became_active":"2024-02-14T15:00:48.281590+0000","last_became_peered":"2024-02-14T15:00:48.281590+0000","last_unstale":"2024-02-14T15:02:45.713729+0000","last_undegraded":"2024-02-14T15:02:45.713729+0000","last_fullsized":"2024-02-14T15:02:45.713729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:22.991037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512389+0000","last_change":"2024-02-14T15:00:49.284729+0000","last_active":"2024-02-14T15:02:46.512389+0000","last_peered":"2024-02-14T15:02:46.512389+0000","last_clean":"2024-02-14T15:02:46.512389+0000","last_became_active":"2024-02-14T15:00:49.284013+0000","last_became_peered":"2024-02-14T15:00:49.284013+0000","last_unstale":"2024-02-14T15:02:46.512389+0000","last_undegraded":"2024-02-14T15:02:46.512389+0000","last_fullsized":"2024-02-14T15:02:46.512389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:03:31.313032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617441+0000","last_change":"2024-02-14T15:00:53.589825+0000","last_active":"2024-02-14T15:02:45.617441+0000","last_peered":"2024-02-14T15:02:45.617441+0000","last_clean":"2024-02-14T15:02:45.617441+0000","last_became_active":"2024-02-14T15:00:53.589424+0000","last_became_peered":"2024-02-14T15:00:53.589424+0000","last_unstale":"2024-02-14T15:02:45.617441+0000","last_undegraded":"2024-02-14T15:02:45.617441+0000","last_fullsized":"2024-02-14T15:02:45.617441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:24.522152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767437+0000","last_change":"2024-02-14T15:00:48.276884+0000","last_active":"2024-02-14T15:02:45.767437+0000","last_peered":"2024-02-14T15:02:45.767437+0000","last_clean":"2024-02-14T15:02:45.767437+0000","last_became_active":"2024-02-14T15:00:48.276576+0000","last_became_peered":"2024-02-14T15:00:48.276576+0000","last_unstale":"2024-02-14T15:02:45.767437+0000","last_undegraded":"2024-02-14T15:02:45.767437+0000","last_fullsized":"2024-02-14T15:02:45.767437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:25.793368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766543+0000","last_change":"2024-02-14T15:00:49.277602+0000","last_active":"2024-02-14T15:02:45.766543+0000","last_peered":"2024-02-14T15:02:45.766543+0000","last_clean":"2024-02-14T15:02:45.766543+0000","last_became_active":"2024-02-14T15:00:49.277249+0000","last_became_peered":"2024-02-14T15:00:49.277249+0000","last_unstale":"2024-02-14T15:02:45.766543+0000","last_undegraded":"2024-02-14T15:02:45.766543+0000","last_fullsized":"2024-02-14T15:02:45.766543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:05.347962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617245+0000","last_change":"2024-02-14T15:00:53.587838+0000","last_active":"2024-02-14T15:02:45.617245+0000","last_peered":"2024-02-14T15:02:45.617245+0000","last_clean":"2024-02-14T15:02:45.617245+0000","last_became_active":"2024-02-14T15:00:53.587012+0000","last_became_peered":"2024-02-14T15:00:53.587012+0000","last_unstale":"2024-02-14T15:02:45.617245+0000","last_undegraded":"2024-02-14T15:02:45.617245+0000","last_fullsized":"2024-02-14T15:02:45.617245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:32:12.276578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767139+0000","last_change":"2024-02-14T15:00:48.273529+0000","last_active":"2024-02-14T15:02:45.767139+0000","last_peered":"2024-02-14T15:02:45.767139+0000","last_clean":"2024-02-14T15:02:45.767139+0000","last_became_active":"2024-02-14T15:00:48.272555+0000","last_became_peered":"2024-02-14T15:00:48.272555+0000","last_unstale":"2024-02-14T15:02:45.767139+0000","last_undegraded":"2024-02-14T15:02:45.767139+0000","last_fullsized":"2024-02-14T15:02:45.767139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:19.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618504+0000","last_change":"2024-02-14T15:00:49.278834+0000","last_active":"2024-02-14T15:02:45.618504+0000","last_peered":"2024-02-14T15:02:45.618504+0000","last_clean":"2024-02-14T15:02:45.618504+0000","last_became_active":"2024-02-14T15:00:49.277095+0000","last_became_peered":"2024-02-14T15:00:49.277095+0000","last_unstale":"2024-02-14T15:02:45.618504+0000","last_undegraded":"2024-02-14T15:02:45.618504+0000","last_fullsized":"2024-02-14T15:02:45.618504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:27.291974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.768818+0000","last_change":"2024-02-14T15:02:45.768818+0000","last_active":"2024-02-14T15:02:45.768818+0000","last_peered":"2024-02-14T15:02:45.768818+0000","last_clean":"2024-02-14T15:02:45.768818+0000","last_became_active":"2024-02-14T15:02:45.768283+0000","last_became_peered":"2024-02-14T15:02:45.768283+0000","last_unstale":"2024-02-14T15:02:45.768818+0000","last_undegraded":"2024-02-14T15:02:45.768818+0000","last_fullsized":"2024-02-14T15:02:45.768818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:59:14.723619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,7,11],"acting":[2,10,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.743886+0000","last_change":"2024-02-14T15:00:48.276146+0000","last_active":"2024-02-14T15:03:09.743886+0000","last_peered":"2024-02-14T15:03:09.743886+0000","last_clean":"2024-02-14T15:03:09.743886+0000","last_became_active":"2024-02-14T15:00:48.275833+0000","last_became_peered":"2024-02-14T15:00:48.275833+0000","last_unstale":"2024-02-14T15:03:09.743886+0000","last_undegraded":"2024-02-14T15:03:09.743886+0000","last_fullsized":"2024-02-14T15:03:09.743886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:46.649486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766980+0000","last_change":"2024-02-14T15:00:49.267285+0000","last_active":"2024-02-14T15:02:45.766980+0000","last_peered":"2024-02-14T15:02:45.766980+0000","last_clean":"2024-02-14T15:02:45.766980+0000","last_became_active":"2024-02-14T15:00:49.265592+0000","last_became_peered":"2024-02-14T15:00:49.265592+0000","last_unstale":"2024-02-14T15:02:45.766980+0000","last_undegraded":"2024-02-14T15:02:45.766980+0000","last_fullsized":"2024-02-14T15:02:45.766980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:18.469835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621465+0000","last_change":"2024-02-14T15:00:53.582085+0000","last_active":"2024-02-14T15:02:45.621465+0000","last_peered":"2024-02-14T15:02:45.621465+0000","last_clean":"2024-02-14T15:02:45.621465+0000","last_became_active":"2024-02-14T15:00:53.581706+0000","last_became_peered":"2024-02-14T15:00:53.581706+0000","last_unstale":"2024-02-14T15:02:45.621465+0000","last_undegraded":"2024-02-14T15:02:45.621465+0000","last_fullsized":"2024-02-14T15:02:45.621465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:32.481142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.705924+0000","last_change":"2024-02-14T15:00:48.268975+0000","last_active":"2024-02-14T15:02:45.705924+0000","last_peered":"2024-02-14T15:02:45.705924+0000","last_clean":"2024-02-14T15:02:45.705924+0000","last_became_active":"2024-02-14T15:00:48.268547+0000","last_became_peered":"2024-02-14T15:00:48.268547+0000","last_unstale":"2024-02-14T15:02:45.705924+0000","last_undegraded":"2024-02-14T15:02:45.705924+0000","last_fullsized":"2024-02-14T15:02:45.705924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:13.937917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618357+0000","last_change":"2024-02-14T15:00:49.276909+0000","last_active":"2024-02-14T15:02:45.618357+0000","last_peered":"2024-02-14T15:02:45.618357+0000","last_clean":"2024-02-14T15:02:45.618357+0000","last_became_active":"2024-02-14T15:00:49.276533+0000","last_became_peered":"2024-02-14T15:00:49.276533+0000","last_unstale":"2024-02-14T15:02:45.618357+0000","last_undegraded":"2024-02-14T15:02:45.618357+0000","last_fullsized":"2024-02-14T15:02:45.618357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:35:31.321455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620174+0000","last_change":"2024-02-14T15:00:53.592223+0000","last_active":"2024-02-14T15:02:45.620174+0000","last_peered":"2024-02-14T15:02:45.620174+0000","last_clean":"2024-02-14T15:02:45.620174+0000","last_became_active":"2024-02-14T15:00:53.591883+0000","last_became_peered":"2024-02-14T15:00:53.591883+0000","last_unstale":"2024-02-14T15:02:45.620174+0000","last_undegraded":"2024-02-14T15:02:45.620174+0000","last_fullsized":"2024-02-14T15:02:45.620174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:10.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616486+0000","last_change":"2024-02-14T15:00:48.274485+0000","last_active":"2024-02-14T15:02:45.616486+0000","last_peered":"2024-02-14T15:02:45.616486+0000","last_clean":"2024-02-14T15:02:45.616486+0000","last_became_active":"2024-02-14T15:00:48.274160+0000","last_became_peered":"2024-02-14T15:00:48.274160+0000","last_unstale":"2024-02-14T15:02:45.616486+0000","last_undegraded":"2024-02-14T15:02:45.616486+0000","last_fullsized":"2024-02-14T15:02:45.616486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:00:47.105069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713108+0000","last_change":"2024-02-14T15:00:49.272577+0000","last_active":"2024-02-14T15:02:45.713108+0000","last_peered":"2024-02-14T15:02:45.713108+0000","last_clean":"2024-02-14T15:02:45.713108+0000","last_became_active":"2024-02-14T15:00:49.271670+0000","last_became_peered":"2024-02-14T15:00:49.271670+0000","last_unstale":"2024-02-14T15:02:45.713108+0000","last_undegraded":"2024-02-14T15:02:45.713108+0000","last_fullsized":"2024-02-14T15:02:45.713108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:19.968389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512199+0000","last_change":"2024-02-14T15:00:53.581212+0000","last_active":"2024-02-14T15:02:46.512199+0000","last_peered":"2024-02-14T15:02:46.512199+0000","last_clean":"2024-02-14T15:02:46.512199+0000","last_became_active":"2024-02-14T15:00:53.580782+0000","last_became_peered":"2024-02-14T15:00:53.580782+0000","last_unstale":"2024-02-14T15:02:46.512199+0000","last_undegraded":"2024-02-14T15:02:46.512199+0000","last_fullsized":"2024-02-14T15:02:46.512199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:14.743575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617597+0000","last_change":"2024-02-14T15:00:48.273572+0000","last_active":"2024-02-14T15:02:45.617597+0000","last_peered":"2024-02-14T15:02:45.617597+0000","last_clean":"2024-02-14T15:02:45.617597+0000","last_became_active":"2024-02-14T15:00:48.273138+0000","last_became_peered":"2024-02-14T15:00:48.273138+0000","last_unstale":"2024-02-14T15:02:45.617597+0000","last_undegraded":"2024-02-14T15:02:45.617597+0000","last_fullsized":"2024-02-14T15:02:45.617597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:04.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766140+0000","last_change":"2024-02-14T15:00:49.288226+0000","last_active":"2024-02-14T15:02:45.766140+0000","last_peered":"2024-02-14T15:02:45.766140+0000","last_clean":"2024-02-14T15:02:45.766140+0000","last_became_active":"2024-02-14T15:00:49.287874+0000","last_became_peered":"2024-02-14T15:00:49.287874+0000","last_unstale":"2024-02-14T15:02:45.766140+0000","last_undegraded":"2024-02-14T15:02:45.766140+0000","last_fullsized":"2024-02-14T15:02:45.766140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:24:02.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713194+0000","last_change":"2024-02-14T15:00:53.590543+0000","last_active":"2024-02-14T15:02:45.713194+0000","last_peered":"2024-02-14T15:02:45.713194+0000","last_clean":"2024-02-14T15:02:45.713194+0000","last_became_active":"2024-02-14T15:00:53.589072+0000","last_became_peered":"2024-02-14T15:00:53.589072+0000","last_unstale":"2024-02-14T15:02:45.713194+0000","last_undegraded":"2024-02-14T15:02:45.713194+0000","last_fullsized":"2024-02-14T15:02:45.713194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:25:13.685655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":146,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:06.739070+0000","last_change":"2024-02-14T15:00:48.272027+0000","last_active":"2024-02-14T15:03:06.739070+0000","last_peered":"2024-02-14T15:03:06.739070+0000","last_clean":"2024-02-14T15:03:06.739070+0000","last_became_active":"2024-02-14T15:00:48.271587+0000","last_became_peered":"2024-02-14T15:00:48.271587+0000","last_unstale":"2024-02-14T15:03:06.739070+0000","last_undegraded":"2024-02-14T15:03:06.739070+0000","last_fullsized":"2024-02-14T15:03:06.739070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:33:03.566189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766999+0000","last_change":"2024-02-14T15:00:49.278936+0000","last_active":"2024-02-14T15:02:45.766999+0000","last_peered":"2024-02-14T15:02:45.766999+0000","last_clean":"2024-02-14T15:02:45.766999+0000","last_became_active":"2024-02-14T15:00:49.278318+0000","last_became_peered":"2024-02-14T15:00:49.278318+0000","last_unstale":"2024-02-14T15:02:45.766999+0000","last_undegraded":"2024-02-14T15:02:45.766999+0000","last_fullsized":"2024-02-14T15:02:45.766999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:01.327303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766017+0000","last_change":"2024-02-14T15:00:53.580340+0000","last_active":"2024-02-14T15:02:45.766017+0000","last_peered":"2024-02-14T15:02:45.766017+0000","last_clean":"2024-02-14T15:02:45.766017+0000","last_became_active":"2024-02-14T15:00:53.579742+0000","last_became_peered":"2024-02-14T15:00:53.579742+0000","last_unstale":"2024-02-14T15:02:45.766017+0000","last_undegraded":"2024-02-14T15:02:45.766017+0000","last_fullsized":"2024-02-14T15:02:45.766017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:38:33.182328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767303+0000","last_change":"2024-02-14T15:00:48.271891+0000","last_active":"2024-02-14T15:02:45.767303+0000","last_peered":"2024-02-14T15:02:45.767303+0000","last_clean":"2024-02-14T15:02:45.767303+0000","last_became_active":"2024-02-14T15:00:48.271419+0000","last_became_peered":"2024-02-14T15:00:48.271419+0000","last_unstale":"2024-02-14T15:02:45.767303+0000","last_undegraded":"2024-02-14T15:02:45.767303+0000","last_fullsized":"2024-02-14T15:02:45.767303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:03:04.725647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512325+0000","last_change":"2024-02-14T15:00:49.277025+0000","last_active":"2024-02-14T15:02:46.512325+0000","last_peered":"2024-02-14T15:02:46.512325+0000","last_clean":"2024-02-14T15:02:46.512325+0000","last_became_active":"2024-02-14T15:00:49.276694+0000","last_became_peered":"2024-02-14T15:00:49.276694+0000","last_unstale":"2024-02-14T15:02:46.512325+0000","last_undegraded":"2024-02-14T15:02:46.512325+0000","last_fullsized":"2024-02-14T15:02:46.512325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:43:07.249407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617568+0000","last_change":"2024-02-14T15:01:44.607053+0000","last_active":"2024-02-14T15:02:45.617568+0000","last_peered":"2024-02-14T15:02:45.617568+0000","last_clean":"2024-02-14T15:02:45.617568+0000","last_became_active":"2024-02-14T15:01:44.606596+0000","last_became_peered":"2024-02-14T15:01:44.606596+0000","last_unstale":"2024-02-14T15:02:45.617568+0000","last_undegraded":"2024-02-14T15:02:45.617568+0000","last_fullsized":"2024-02-14T15:02:45.617568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:35.949953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618389+0000","last_change":"2024-02-14T15:00:48.272866+0000","last_active":"2024-02-14T15:02:45.618389+0000","last_peered":"2024-02-14T15:02:45.618389+0000","last_clean":"2024-02-14T15:02:45.618389+0000","last_became_active":"2024-02-14T15:00:48.272520+0000","last_became_peered":"2024-02-14T15:00:48.272520+0000","last_unstale":"2024-02-14T15:02:45.618389+0000","last_undegraded":"2024-02-14T15:02:45.618389+0000","last_fullsized":"2024-02-14T15:02:45.618389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:39.434577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:49.280994+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:49.280205+0000","last_became_peered":"2024-02-14T15:00:49.280205+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:53:48.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766150+0000","last_change":"2024-02-14T15:00:53.582599+0000","last_active":"2024-02-14T15:02:45.766150+0000","last_peered":"2024-02-14T15:02:45.766150+0000","last_clean":"2024-02-14T15:02:45.766150+0000","last_became_active":"2024-02-14T15:00:53.582231+0000","last_became_peered":"2024-02-14T15:00:53.582231+0000","last_unstale":"2024-02-14T15:02:45.766150+0000","last_undegraded":"2024-02-14T15:02:45.766150+0000","last_fullsized":"2024-02-14T15:02:45.766150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:19.126264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":319,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:07.740623+0000","last_change":"2024-02-14T15:01:44.615393+0000","last_active":"2024-02-14T15:03:07.740623+0000","last_peered":"2024-02-14T15:03:07.740623+0000","last_clean":"2024-02-14T15:03:07.740623+0000","last_became_active":"2024-02-14T15:01:44.604294+0000","last_became_peered":"2024-02-14T15:01:44.604294+0000","last_unstale":"2024-02-14T15:03:07.740623+0000","last_undegraded":"2024-02-14T15:03:07.740623+0000","last_fullsized":"2024-02-14T15:03:07.740623+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:00.459860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":180,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767208+0000","last_change":"2024-02-14T15:00:49.270144+0000","last_active":"2024-02-14T15:02:45.767208+0000","last_peered":"2024-02-14T15:02:45.767208+0000","last_clean":"2024-02-14T15:02:45.767208+0000","last_became_active":"2024-02-14T15:00:49.269806+0000","last_became_peered":"2024-02-14T15:00:49.269806+0000","last_unstale":"2024-02-14T15:02:45.767208+0000","last_undegraded":"2024-02-14T15:02:45.767208+0000","last_fullsized":"2024-02-14T15:02:45.767208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:42:18.559226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618862+0000","last_change":"2024-02-14T15:00:53.582755+0000","last_active":"2024-02-14T15:02:45.618862+0000","last_peered":"2024-02-14T15:02:45.618862+0000","last_clean":"2024-02-14T15:02:45.618862+0000","last_became_active":"2024-02-14T15:00:53.582375+0000","last_became_peered":"2024-02-14T15:00:53.582375+0000","last_unstale":"2024-02-14T15:02:45.618862+0000","last_undegraded":"2024-02-14T15:02:45.618862+0000","last_fullsized":"2024-02-14T15:02:45.618862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:11:49.115136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621377+0000","last_change":"2024-02-14T15:00:48.268982+0000","last_active":"2024-02-14T15:02:45.621377+0000","last_peered":"2024-02-14T15:02:45.621377+0000","last_clean":"2024-02-14T15:02:45.621377+0000","last_became_active":"2024-02-14T15:00:48.267137+0000","last_became_peered":"2024-02-14T15:00:48.267137+0000","last_unstale":"2024-02-14T15:02:45.621377+0000","last_undegraded":"2024-02-14T15:02:45.621377+0000","last_fullsized":"2024-02-14T15:02:45.621377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:44.351561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616587+0000","last_change":"2024-02-14T15:00:49.267493+0000","last_active":"2024-02-14T15:02:45.616587+0000","last_peered":"2024-02-14T15:02:45.616587+0000","last_clean":"2024-02-14T15:02:45.616587+0000","last_became_active":"2024-02-14T15:00:49.267161+0000","last_became_peered":"2024-02-14T15:00:49.267161+0000","last_unstale":"2024-02-14T15:02:45.616587+0000","last_undegraded":"2024-02-14T15:02:45.616587+0000","last_fullsized":"2024-02-14T15:02:45.616587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:35:22.997999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.768993+0000","last_change":"2024-02-14T15:02:45.768993+0000","last_active":"2024-02-14T15:02:45.768993+0000","last_peered":"2024-02-14T15:02:45.768993+0000","last_clean":"2024-02-14T15:02:45.768993+0000","last_became_active":"2024-02-14T15:02:45.768413+0000","last_became_peered":"2024-02-14T15:02:45.768413+0000","last_unstale":"2024-02-14T15:02:45.768993+0000","last_undegraded":"2024-02-14T15:02:45.768993+0000","last_fullsized":"2024-02-14T15:02:45.768993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:45:54.193127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618346+0000","last_change":"2024-02-14T15:00:48.282085+0000","last_active":"2024-02-14T15:02:45.618346+0000","last_peered":"2024-02-14T15:02:45.618346+0000","last_clean":"2024-02-14T15:02:45.618346+0000","last_became_active":"2024-02-14T15:00:48.281836+0000","last_became_peered":"2024-02-14T15:00:48.281836+0000","last_unstale":"2024-02-14T15:02:45.618346+0000","last_undegraded":"2024-02-14T15:02:45.618346+0000","last_fullsized":"2024-02-14T15:02:45.618346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:07:28.145074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767345+0000","last_change":"2024-02-14T15:00:49.280013+0000","last_active":"2024-02-14T15:02:45.767345+0000","last_peered":"2024-02-14T15:02:45.767345+0000","last_clean":"2024-02-14T15:02:45.767345+0000","last_became_active":"2024-02-14T15:00:49.279667+0000","last_became_peered":"2024-02-14T15:00:49.279667+0000","last_unstale":"2024-02-14T15:02:45.767345+0000","last_undegraded":"2024-02-14T15:02:45.767345+0000","last_fullsized":"2024-02-14T15:02:45.767345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:03:16.559295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766612+0000","last_change":"2024-02-14T15:00:53.580269+0000","last_active":"2024-02-14T15:02:45.766612+0000","last_peered":"2024-02-14T15:02:45.766612+0000","last_clean":"2024-02-14T15:02:45.766612+0000","last_became_active":"2024-02-14T15:00:53.579863+0000","last_became_peered":"2024-02-14T15:00:53.579863+0000","last_unstale":"2024-02-14T15:02:45.766612+0000","last_undegraded":"2024-02-14T15:02:45.766612+0000","last_fullsized":"2024-02-14T15:02:45.766612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:37:19.275654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.744250+0000","last_change":"2024-02-14T15:00:48.267310+0000","last_active":"2024-02-14T15:03:09.744250+0000","last_peered":"2024-02-14T15:03:09.744250+0000","last_clean":"2024-02-14T15:03:09.744250+0000","last_became_active":"2024-02-14T15:00:48.266864+0000","last_became_peered":"2024-02-14T15:00:48.266864+0000","last_unstale":"2024-02-14T15:03:09.744250+0000","last_undegraded":"2024-02-14T15:03:09.744250+0000","last_fullsized":"2024-02-14T15:03:09.744250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:53:07.703549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:49.288568+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:49.288222+0000","last_became_peered":"2024-02-14T15:00:49.288222+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:38:38.909769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616994+0000","last_change":"2024-02-14T15:00:53.585673+0000","last_active":"2024-02-14T15:02:45.616994+0000","last_peered":"2024-02-14T15:02:45.616994+0000","last_clean":"2024-02-14T15:02:45.616994+0000","last_became_active":"2024-02-14T15:00:53.585238+0000","last_became_peered":"2024-02-14T15:00:53.585238+0000","last_unstale":"2024-02-14T15:02:45.616994+0000","last_undegraded":"2024-02-14T15:02:45.616994+0000","last_fullsized":"2024-02-14T15:02:45.616994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:48:11.845921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766795+0000","last_change":"2024-02-14T15:00:48.275833+0000","last_active":"2024-02-14T15:02:45.766795+0000","last_peered":"2024-02-14T15:02:45.766795+0000","last_clean":"2024-02-14T15:02:45.766795+0000","last_became_active":"2024-02-14T15:00:48.275499+0000","last_became_peered":"2024-02-14T15:00:48.275499+0000","last_unstale":"2024-02-14T15:02:45.766795+0000","last_undegraded":"2024-02-14T15:02:45.766795+0000","last_fullsized":"2024-02-14T15:02:45.766795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:35:10.239156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618303+0000","last_change":"2024-02-14T15:01:44.605394+0000","last_active":"2024-02-14T15:02:45.618303+0000","last_peered":"2024-02-14T15:02:45.618303+0000","last_clean":"2024-02-14T15:02:45.618303+0000","last_became_active":"2024-02-14T15:01:44.604874+0000","last_became_peered":"2024-02-14T15:01:44.604874+0000","last_unstale":"2024-02-14T15:02:45.618303+0000","last_undegraded":"2024-02-14T15:02:45.618303+0000","last_fullsized":"2024-02-14T15:02:45.618303+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:41:47.508592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618214+0000","last_change":"2024-02-14T15:00:53.586406+0000","last_active":"2024-02-14T15:02:45.618214+0000","last_peered":"2024-02-14T15:02:45.618214+0000","last_clean":"2024-02-14T15:02:45.618214+0000","last_became_active":"2024-02-14T15:00:53.586034+0000","last_became_peered":"2024-02-14T15:00:53.586034+0000","last_unstale":"2024-02-14T15:02:45.618214+0000","last_undegraded":"2024-02-14T15:02:45.618214+0000","last_fullsized":"2024-02-14T15:02:45.618214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:46.886154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767472+0000","last_change":"2024-02-14T15:00:48.274305+0000","last_active":"2024-02-14T15:02:45.767472+0000","last_peered":"2024-02-14T15:02:45.767472+0000","last_clean":"2024-02-14T15:02:45.767472+0000","last_became_active":"2024-02-14T15:00:48.273966+0000","last_became_peered":"2024-02-14T15:00:48.273966+0000","last_unstale":"2024-02-14T15:02:45.767472+0000","last_undegraded":"2024-02-14T15:02:45.767472+0000","last_fullsized":"2024-02-14T15:02:45.767472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:28:49.198099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620493+0000","last_change":"2024-02-14T15:00:49.278473+0000","last_active":"2024-02-14T15:02:45.620493+0000","last_peered":"2024-02-14T15:02:45.620493+0000","last_clean":"2024-02-14T15:02:45.620493+0000","last_became_active":"2024-02-14T15:00:49.278145+0000","last_became_peered":"2024-02-14T15:00:49.278145+0000","last_unstale":"2024-02-14T15:02:45.620493+0000","last_undegraded":"2024-02-14T15:02:45.620493+0000","last_fullsized":"2024-02-14T15:02:45.620493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:49:11.718035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767006+0000","last_change":"2024-02-14T15:00:53.581164+0000","last_active":"2024-02-14T15:02:45.767006+0000","last_peered":"2024-02-14T15:02:45.767006+0000","last_clean":"2024-02-14T15:02:45.767006+0000","last_became_active":"2024-02-14T15:00:53.580796+0000","last_became_peered":"2024-02-14T15:00:53.580796+0000","last_unstale":"2024-02-14T15:02:45.767006+0000","last_undegraded":"2024-02-14T15:02:45.767006+0000","last_fullsized":"2024-02-14T15:02:45.767006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:37.245035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766486+0000","last_change":"2024-02-14T15:00:48.271427+0000","last_active":"2024-02-14T15:02:45.766486+0000","last_peered":"2024-02-14T15:02:45.766486+0000","last_clean":"2024-02-14T15:02:45.766486+0000","last_became_active":"2024-02-14T15:00:48.271115+0000","last_became_peered":"2024-02-14T15:00:48.271115+0000","last_unstale":"2024-02-14T15:02:45.766486+0000","last_undegraded":"2024-02-14T15:02:45.766486+0000","last_fullsized":"2024-02-14T15:02:45.766486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:56:01.008058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706430+0000","last_change":"2024-02-14T15:00:49.269964+0000","last_active":"2024-02-14T15:02:45.706430+0000","last_peered":"2024-02-14T15:02:45.706430+0000","last_clean":"2024-02-14T15:02:45.706430+0000","last_became_active":"2024-02-14T15:00:49.269628+0000","last_became_peered":"2024-02-14T15:00:49.269628+0000","last_unstale":"2024-02-14T15:02:45.706430+0000","last_undegraded":"2024-02-14T15:02:45.706430+0000","last_fullsized":"2024-02-14T15:02:45.706430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:04:26.722773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616880+0000","last_change":"2024-02-14T15:00:53.579843+0000","last_active":"2024-02-14T15:02:45.616880+0000","last_peered":"2024-02-14T15:02:45.616880+0000","last_clean":"2024-02-14T15:02:45.616880+0000","last_became_active":"2024-02-14T15:00:53.579457+0000","last_became_peered":"2024-02-14T15:00:53.579457+0000","last_unstale":"2024-02-14T15:02:45.616880+0000","last_undegraded":"2024-02-14T15:02:45.616880+0000","last_fullsized":"2024-02-14T15:02:45.616880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:34:49.438317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767597+0000","last_change":"2024-02-14T15:00:48.275016+0000","last_active":"2024-02-14T15:02:45.767597+0000","last_peered":"2024-02-14T15:02:45.767597+0000","last_clean":"2024-02-14T15:02:45.767597+0000","last_became_active":"2024-02-14T15:00:48.274636+0000","last_became_peered":"2024-02-14T15:00:48.274636+0000","last_unstale":"2024-02-14T15:02:45.767597+0000","last_undegraded":"2024-02-14T15:02:45.767597+0000","last_fullsized":"2024-02-14T15:02:45.767597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:12:12.961276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.706105+0000","last_change":"2024-02-14T15:01:44.602904+0000","last_active":"2024-02-14T15:02:45.706105+0000","last_peered":"2024-02-14T15:02:45.706105+0000","last_clean":"2024-02-14T15:02:45.706105+0000","last_became_active":"2024-02-14T15:01:44.602480+0000","last_became_peered":"2024-02-14T15:01:44.602480+0000","last_unstale":"2024-02-14T15:02:45.706105+0000","last_undegraded":"2024-02-14T15:02:45.706105+0000","last_fullsized":"2024-02-14T15:02:45.706105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:27:33.451581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616988+0000","last_change":"2024-02-14T15:00:53.584068+0000","last_active":"2024-02-14T15:02:45.616988+0000","last_peered":"2024-02-14T15:02:45.616988+0000","last_clean":"2024-02-14T15:02:45.616988+0000","last_became_active":"2024-02-14T15:00:53.583263+0000","last_became_peered":"2024-02-14T15:00:53.583263+0000","last_unstale":"2024-02-14T15:02:45.616988+0000","last_undegraded":"2024-02-14T15:02:45.616988+0000","last_fullsized":"2024-02-14T15:02:45.616988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:15:15.726442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617918+0000","last_change":"2024-02-14T15:00:49.271265+0000","last_active":"2024-02-14T15:02:45.617918+0000","last_peered":"2024-02-14T15:02:45.617918+0000","last_clean":"2024-02-14T15:02:45.617918+0000","last_became_active":"2024-02-14T15:00:49.270931+0000","last_became_peered":"2024-02-14T15:00:49.270931+0000","last_unstale":"2024-02-14T15:02:45.617918+0000","last_undegraded":"2024-02-14T15:02:45.617918+0000","last_fullsized":"2024-02-14T15:02:45.617918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:15:40.570730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:03:09.743769+0000","last_change":"2024-02-14T15:00:48.286669+0000","last_active":"2024-02-14T15:03:09.743769+0000","last_peered":"2024-02-14T15:03:09.743769+0000","last_clean":"2024-02-14T15:03:09.743769+0000","last_became_active":"2024-02-14T15:00:48.286252+0000","last_became_peered":"2024-02-14T15:00:48.286252+0000","last_unstale":"2024-02-14T15:03:09.743769+0000","last_undegraded":"2024-02-14T15:03:09.743769+0000","last_fullsized":"2024-02-14T15:03:09.743769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:49:40.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767025+0000","last_change":"2024-02-14T15:00:53.583353+0000","last_active":"2024-02-14T15:02:45.767025+0000","last_peered":"2024-02-14T15:02:45.767025+0000","last_clean":"2024-02-14T15:02:45.767025+0000","last_became_active":"2024-02-14T15:00:53.582449+0000","last_became_peered":"2024-02-14T15:00:53.582449+0000","last_unstale":"2024-02-14T15:02:45.767025+0000","last_undegraded":"2024-02-14T15:02:45.767025+0000","last_fullsized":"2024-02-14T15:02:45.767025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:09:01.514603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713191+0000","last_change":"2024-02-14T15:00:49.285579+0000","last_active":"2024-02-14T15:02:45.713191+0000","last_peered":"2024-02-14T15:02:45.713191+0000","last_clean":"2024-02-14T15:02:45.713191+0000","last_became_active":"2024-02-14T15:00:49.283741+0000","last_became_peered":"2024-02-14T15:00:49.283741+0000","last_unstale":"2024-02-14T15:02:45.713191+0000","last_undegraded":"2024-02-14T15:02:45.713191+0000","last_fullsized":"2024-02-14T15:02:45.713191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:05.813218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617501+0000","last_change":"2024-02-14T15:00:48.264434+0000","last_active":"2024-02-14T15:02:45.617501+0000","last_peered":"2024-02-14T15:02:45.617501+0000","last_clean":"2024-02-14T15:02:45.617501+0000","last_became_active":"2024-02-14T15:00:48.264048+0000","last_became_peered":"2024-02-14T15:00:48.264048+0000","last_unstale":"2024-02-14T15:02:45.617501+0000","last_undegraded":"2024-02-14T15:02:45.617501+0000","last_fullsized":"2024-02-14T15:02:45.617501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:52:32.498128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766621+0000","last_change":"2024-02-14T14:59:36.539524+0000","last_active":"2024-02-14T15:02:45.766621+0000","last_peered":"2024-02-14T15:02:45.766621+0000","last_clean":"2024-02-14T15:02:45.766621+0000","last_became_active":"2024-02-14T14:59:36.538922+0000","last_became_peered":"2024-02-14T14:59:36.538922+0000","last_unstale":"2024-02-14T15:02:45.766621+0000","last_undegraded":"2024-02-14T15:02:45.766621+0000","last_fullsized":"2024-02-14T15:02:45.766621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T14:56:09.567049+0000","last_clean_scrub_stamp":"2024-02-14T14:56:09.567049+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:58.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.707765+0000","last_change":"2024-02-14T15:02:45.707287+0000","last_active":"2024-02-14T15:02:45.707765+0000","last_peered":"2024-02-14T15:02:45.707765+0000","last_clean":"2024-02-14T15:02:45.707765+0000","last_became_active":"2024-02-14T15:02:45.706798+0000","last_became_peered":"2024-02-14T15:02:45.706798+0000","last_unstale":"2024-02-14T15:02:45.707765+0000","last_undegraded":"2024-02-14T15:02:45.707765+0000","last_fullsized":"2024-02-14T15:02:45.707765+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:12.237714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.616413+0000","last_change":"2024-02-14T15:00:49.272692+0000","last_active":"2024-02-14T15:02:45.616413+0000","last_peered":"2024-02-14T15:02:45.616413+0000","last_clean":"2024-02-14T15:02:45.616413+0000","last_became_active":"2024-02-14T15:00:49.272353+0000","last_became_peered":"2024-02-14T15:00:49.272353+0000","last_unstale":"2024-02-14T15:02:45.616413+0000","last_undegraded":"2024-02-14T15:02:45.616413+0000","last_fullsized":"2024-02-14T15:02:45.616413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:47.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766756+0000","last_change":"2024-02-14T15:00:48.272470+0000","last_active":"2024-02-14T15:02:45.766756+0000","last_peered":"2024-02-14T15:02:45.766756+0000","last_clean":"2024-02-14T15:02:45.766756+0000","last_became_active":"2024-02-14T15:00:48.272126+0000","last_became_peered":"2024-02-14T15:00:48.272126+0000","last_unstale":"2024-02-14T15:02:45.766756+0000","last_undegraded":"2024-02-14T15:02:45.766756+0000","last_fullsized":"2024-02-14T15:02:45.766756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:21:02.601411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620193+0000","last_change":"2024-02-14T15:00:53.585950+0000","last_active":"2024-02-14T15:02:45.620193+0000","last_peered":"2024-02-14T15:02:45.620193+0000","last_clean":"2024-02-14T15:02:45.620193+0000","last_became_active":"2024-02-14T15:00:53.585547+0000","last_became_peered":"2024-02-14T15:00:53.585547+0000","last_unstale":"2024-02-14T15:02:45.620193+0000","last_undegraded":"2024-02-14T15:02:45.620193+0000","last_fullsized":"2024-02-14T15:02:45.620193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:22:22.222243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767371+0000","last_change":"2024-02-14T15:00:48.274650+0000","last_active":"2024-02-14T15:02:45.767371+0000","last_peered":"2024-02-14T15:02:45.767371+0000","last_clean":"2024-02-14T15:02:45.767371+0000","last_became_active":"2024-02-14T15:00:48.274085+0000","last_became_peered":"2024-02-14T15:00:48.274085+0000","last_unstale":"2024-02-14T15:02:45.767371+0000","last_undegraded":"2024-02-14T15:02:45.767371+0000","last_fullsized":"2024-02-14T15:02:45.767371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:43:53.307596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617022+0000","last_change":"2024-02-14T15:00:49.268419+0000","last_active":"2024-02-14T15:02:45.617022+0000","last_peered":"2024-02-14T15:02:45.617022+0000","last_clean":"2024-02-14T15:02:45.617022+0000","last_became_active":"2024-02-14T15:00:49.268120+0000","last_became_peered":"2024-02-14T15:00:49.268120+0000","last_unstale":"2024-02-14T15:02:45.617022+0000","last_undegraded":"2024-02-14T15:02:45.617022+0000","last_fullsized":"2024-02-14T15:02:45.617022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:10:36.319337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713131+0000","last_change":"2024-02-14T15:00:53.586151+0000","last_active":"2024-02-14T15:02:45.713131+0000","last_peered":"2024-02-14T15:02:45.713131+0000","last_clean":"2024-02-14T15:02:45.713131+0000","last_became_active":"2024-02-14T15:00:53.585727+0000","last_became_peered":"2024-02-14T15:00:53.585727+0000","last_unstale":"2024-02-14T15:02:45.713131+0000","last_undegraded":"2024-02-14T15:02:45.713131+0000","last_fullsized":"2024-02-14T15:02:45.713131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:40:03.741063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":53,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766847+0000","last_change":"2024-02-14T15:01:44.615028+0000","last_active":"2024-02-14T15:02:45.766847+0000","last_peered":"2024-02-14T15:02:45.766847+0000","last_clean":"2024-02-14T15:02:45.766847+0000","last_became_active":"2024-02-14T15:01:44.606281+0000","last_became_peered":"2024-02-14T15:01:44.606281+0000","last_unstale":"2024-02-14T15:02:45.766847+0000","last_undegraded":"2024-02-14T15:02:45.766847+0000","last_fullsized":"2024-02-14T15:02:45.766847+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:20:37.706659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713701+0000","last_change":"2024-02-14T15:00:49.285726+0000","last_active":"2024-02-14T15:02:45.713701+0000","last_peered":"2024-02-14T15:02:45.713701+0000","last_clean":"2024-02-14T15:02:45.713701+0000","last_became_active":"2024-02-14T15:00:49.283728+0000","last_became_peered":"2024-02-14T15:00:49.283728+0000","last_unstale":"2024-02-14T15:02:45.713701+0000","last_undegraded":"2024-02-14T15:02:45.713701+0000","last_fullsized":"2024-02-14T15:02:45.713701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.764048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620850+0000","last_change":"2024-02-14T15:00:53.583644+0000","last_active":"2024-02-14T15:02:45.620850+0000","last_peered":"2024-02-14T15:02:45.620850+0000","last_clean":"2024-02-14T15:02:45.620850+0000","last_became_active":"2024-02-14T15:00:53.582801+0000","last_became_peered":"2024-02-14T15:00:53.582801+0000","last_unstale":"2024-02-14T15:02:45.620850+0000","last_undegraded":"2024-02-14T15:02:45.620850+0000","last_fullsized":"2024-02-14T15:02:45.620850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:41:40.622091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617759+0000","last_change":"2024-02-14T15:00:48.270231+0000","last_active":"2024-02-14T15:02:45.617759+0000","last_peered":"2024-02-14T15:02:45.617759+0000","last_clean":"2024-02-14T15:02:45.617759+0000","last_became_active":"2024-02-14T15:00:48.269880+0000","last_became_peered":"2024-02-14T15:00:48.269880+0000","last_unstale":"2024-02-14T15:02:45.617759+0000","last_undegraded":"2024-02-14T15:02:45.617759+0000","last_fullsized":"2024-02-14T15:02:45.617759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:08:47.376417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512291+0000","last_change":"2024-02-14T15:00:49.277227+0000","last_active":"2024-02-14T15:02:46.512291+0000","last_peered":"2024-02-14T15:02:46.512291+0000","last_clean":"2024-02-14T15:02:46.512291+0000","last_became_active":"2024-02-14T15:00:49.276904+0000","last_became_peered":"2024-02-14T15:00:49.276904+0000","last_unstale":"2024-02-14T15:02:46.512291+0000","last_undegraded":"2024-02-14T15:02:46.512291+0000","last_fullsized":"2024-02-14T15:02:46.512291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:01.091327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617891+0000","last_change":"2024-02-14T15:00:53.587855+0000","last_active":"2024-02-14T15:02:45.617891+0000","last_peered":"2024-02-14T15:02:45.617891+0000","last_clean":"2024-02-14T15:02:45.617891+0000","last_became_active":"2024-02-14T15:00:53.586313+0000","last_became_peered":"2024-02-14T15:00:53.586313+0000","last_unstale":"2024-02-14T15:02:45.617891+0000","last_undegraded":"2024-02-14T15:02:45.617891+0000","last_fullsized":"2024-02-14T15:02:45.617891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:36:41.279582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766114+0000","last_change":"2024-02-14T15:00:48.267385+0000","last_active":"2024-02-14T15:02:45.766114+0000","last_peered":"2024-02-14T15:02:45.766114+0000","last_clean":"2024-02-14T15:02:45.766114+0000","last_became_active":"2024-02-14T15:00:48.267036+0000","last_became_peered":"2024-02-14T15:00:48.267036+0000","last_unstale":"2024-02-14T15:02:45.766114+0000","last_undegraded":"2024-02-14T15:02:45.766114+0000","last_fullsized":"2024-02-14T15:02:45.766114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:29:24.840030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766956+0000","last_change":"2024-02-14T15:00:49.272233+0000","last_active":"2024-02-14T15:02:45.766956+0000","last_peered":"2024-02-14T15:02:45.766956+0000","last_clean":"2024-02-14T15:02:45.766956+0000","last_became_active":"2024-02-14T15:00:49.271813+0000","last_became_peered":"2024-02-14T15:00:49.271813+0000","last_unstale":"2024-02-14T15:02:45.766956+0000","last_undegraded":"2024-02-14T15:02:45.766956+0000","last_fullsized":"2024-02-14T15:02:45.766956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:30:32.521443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766727+0000","last_change":"2024-02-14T15:00:53.583385+0000","last_active":"2024-02-14T15:02:45.766727+0000","last_peered":"2024-02-14T15:02:45.766727+0000","last_clean":"2024-02-14T15:02:45.766727+0000","last_became_active":"2024-02-14T15:00:53.583023+0000","last_became_peered":"2024-02-14T15:00:53.583023+0000","last_unstale":"2024-02-14T15:02:45.766727+0000","last_undegraded":"2024-02-14T15:02:45.766727+0000","last_fullsized":"2024-02-14T15:02:45.766727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:40:53.500939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713137+0000","last_change":"2024-02-14T15:00:48.261376+0000","last_active":"2024-02-14T15:02:45.713137+0000","last_peered":"2024-02-14T15:02:45.713137+0000","last_clean":"2024-02-14T15:02:45.713137+0000","last_became_active":"2024-02-14T15:00:48.260461+0000","last_became_peered":"2024-02-14T15:00:48.260461+0000","last_unstale":"2024-02-14T15:02:45.713137+0000","last_undegraded":"2024-02-14T15:02:45.713137+0000","last_fullsized":"2024-02-14T15:02:45.713137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:35.052418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.619887+0000","last_change":"2024-02-14T15:00:49.273117+0000","last_active":"2024-02-14T15:02:45.619887+0000","last_peered":"2024-02-14T15:02:45.619887+0000","last_clean":"2024-02-14T15:02:45.619887+0000","last_became_active":"2024-02-14T15:00:49.272810+0000","last_became_peered":"2024-02-14T15:00:49.272810+0000","last_unstale":"2024-02-14T15:02:45.619887+0000","last_undegraded":"2024-02-14T15:02:45.619887+0000","last_fullsized":"2024-02-14T15:02:45.619887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:37:00.992212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617747+0000","last_change":"2024-02-14T15:00:53.583972+0000","last_active":"2024-02-14T15:02:45.617747+0000","last_peered":"2024-02-14T15:02:45.617747+0000","last_clean":"2024-02-14T15:02:45.617747+0000","last_became_active":"2024-02-14T15:00:53.583563+0000","last_became_peered":"2024-02-14T15:00:53.583563+0000","last_unstale":"2024-02-14T15:02:45.617747+0000","last_undegraded":"2024-02-14T15:02:45.617747+0000","last_fullsized":"2024-02-14T15:02:45.617747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:08:55.811082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621077+0000","last_change":"2024-02-14T15:00:48.270363+0000","last_active":"2024-02-14T15:02:45.621077+0000","last_peered":"2024-02-14T15:02:45.621077+0000","last_clean":"2024-02-14T15:02:45.621077+0000","last_became_active":"2024-02-14T15:00:48.270030+0000","last_became_peered":"2024-02-14T15:00:48.270030+0000","last_unstale":"2024-02-14T15:02:45.621077+0000","last_undegraded":"2024-02-14T15:02:45.621077+0000","last_fullsized":"2024-02-14T15:02:45.621077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:28.885451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713465+0000","last_change":"2024-02-14T15:00:49.285647+0000","last_active":"2024-02-14T15:02:45.713465+0000","last_peered":"2024-02-14T15:02:45.713465+0000","last_clean":"2024-02-14T15:02:45.713465+0000","last_became_active":"2024-02-14T15:00:49.285267+0000","last_became_peered":"2024-02-14T15:00:49.285267+0000","last_unstale":"2024-02-14T15:02:45.713465+0000","last_undegraded":"2024-02-14T15:02:45.713465+0000","last_fullsized":"2024-02-14T15:02:45.713465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:37:03.143177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617467+0000","last_change":"2024-02-14T15:00:53.591029+0000","last_active":"2024-02-14T15:02:45.617467+0000","last_peered":"2024-02-14T15:02:45.617467+0000","last_clean":"2024-02-14T15:02:45.617467+0000","last_became_active":"2024-02-14T15:00:53.590570+0000","last_became_peered":"2024-02-14T15:00:53.590570+0000","last_unstale":"2024-02-14T15:02:45.617467+0000","last_undegraded":"2024-02-14T15:02:45.617467+0000","last_fullsized":"2024-02-14T15:02:45.617467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:59:41.809227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620092+0000","last_change":"2024-02-14T15:01:44.611339+0000","last_active":"2024-02-14T15:02:45.620092+0000","last_peered":"2024-02-14T15:02:45.620092+0000","last_clean":"2024-02-14T15:02:45.620092+0000","last_became_active":"2024-02-14T15:01:44.605631+0000","last_became_peered":"2024-02-14T15:01:44.605631+0000","last_unstale":"2024-02-14T15:02:45.620092+0000","last_undegraded":"2024-02-14T15:02:45.620092+0000","last_fullsized":"2024-02-14T15:02:45.620092+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:24:19.853345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767748+0000","last_change":"2024-02-14T15:00:49.276346+0000","last_active":"2024-02-14T15:02:45.767748+0000","last_peered":"2024-02-14T15:02:45.767748+0000","last_clean":"2024-02-14T15:02:45.767748+0000","last_became_active":"2024-02-14T15:00:49.276000+0000","last_became_peered":"2024-02-14T15:00:49.276000+0000","last_unstale":"2024-02-14T15:02:45.767748+0000","last_undegraded":"2024-02-14T15:02:45.767748+0000","last_fullsized":"2024-02-14T15:02:45.767748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:14:03.811981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766244+0000","last_change":"2024-02-14T15:00:53.587021+0000","last_active":"2024-02-14T15:02:45.766244+0000","last_peered":"2024-02-14T15:02:45.766244+0000","last_clean":"2024-02-14T15:02:45.766244+0000","last_became_active":"2024-02-14T15:00:53.586492+0000","last_became_peered":"2024-02-14T15:00:53.586492+0000","last_unstale":"2024-02-14T15:02:45.766244+0000","last_undegraded":"2024-02-14T15:02:45.766244+0000","last_fullsized":"2024-02-14T15:02:45.766244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:51:02.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766373+0000","last_change":"2024-02-14T15:01:44.611916+0000","last_active":"2024-02-14T15:02:45.766373+0000","last_peered":"2024-02-14T15:02:45.766373+0000","last_clean":"2024-02-14T15:02:45.766373+0000","last_became_active":"2024-02-14T15:01:44.605791+0000","last_became_peered":"2024-02-14T15:01:44.605791+0000","last_unstale":"2024-02-14T15:02:45.766373+0000","last_undegraded":"2024-02-14T15:02:45.766373+0000","last_fullsized":"2024-02-14T15:02:45.766373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:50:54.066205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767679+0000","last_change":"2024-02-14T15:00:49.276110+0000","last_active":"2024-02-14T15:02:45.767679+0000","last_peered":"2024-02-14T15:02:45.767679+0000","last_clean":"2024-02-14T15:02:45.767679+0000","last_became_active":"2024-02-14T15:00:49.275729+0000","last_became_peered":"2024-02-14T15:00:49.275729+0000","last_unstale":"2024-02-14T15:02:45.767679+0000","last_undegraded":"2024-02-14T15:02:45.767679+0000","last_fullsized":"2024-02-14T15:02:45.767679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:07:29.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.511830+0000","last_change":"2024-02-14T15:00:53.582061+0000","last_active":"2024-02-14T15:02:46.511830+0000","last_peered":"2024-02-14T15:02:46.511830+0000","last_clean":"2024-02-14T15:02:46.511830+0000","last_became_active":"2024-02-14T15:00:53.581659+0000","last_became_peered":"2024-02-14T15:00:53.581659+0000","last_unstale":"2024-02-14T15:02:46.511830+0000","last_undegraded":"2024-02-14T15:02:46.511830+0000","last_fullsized":"2024-02-14T15:02:46.511830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:37:13.814496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.618666+0000","last_change":"2024-02-14T15:00:48.283300+0000","last_active":"2024-02-14T15:02:45.618666+0000","last_peered":"2024-02-14T15:02:45.618666+0000","last_clean":"2024-02-14T15:02:45.618666+0000","last_became_active":"2024-02-14T15:00:48.282939+0000","last_became_peered":"2024-02-14T15:00:48.282939+0000","last_unstale":"2024-02-14T15:02:45.618666+0000","last_undegraded":"2024-02-14T15:02:45.618666+0000","last_fullsized":"2024-02-14T15:02:45.618666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:56:52.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767479+0000","last_change":"2024-02-14T15:00:49.273435+0000","last_active":"2024-02-14T15:02:45.767479+0000","last_peered":"2024-02-14T15:02:45.767479+0000","last_clean":"2024-02-14T15:02:45.767479+0000","last_became_active":"2024-02-14T15:00:49.272980+0000","last_became_peered":"2024-02-14T15:00:49.272980+0000","last_unstale":"2024-02-14T15:02:45.767479+0000","last_undegraded":"2024-02-14T15:02:45.767479+0000","last_fullsized":"2024-02-14T15:02:45.767479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:19:05.294943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617563+0000","last_change":"2024-02-14T15:00:53.583509+0000","last_active":"2024-02-14T15:02:45.617563+0000","last_peered":"2024-02-14T15:02:45.617563+0000","last_clean":"2024-02-14T15:02:45.617563+0000","last_became_active":"2024-02-14T15:00:53.583148+0000","last_became_peered":"2024-02-14T15:00:53.583148+0000","last_unstale":"2024-02-14T15:02:45.617563+0000","last_undegraded":"2024-02-14T15:02:45.617563+0000","last_fullsized":"2024-02-14T15:02:45.617563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:52.555106+0000","last_clean_scrub_stamp":"2024-02-14T15:00:52.555106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:06:47.059643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617044+0000","last_change":"2024-02-14T15:00:48.286707+0000","last_active":"2024-02-14T15:02:45.617044+0000","last_peered":"2024-02-14T15:02:45.617044+0000","last_clean":"2024-02-14T15:02:45.617044+0000","last_became_active":"2024-02-14T15:00:48.286355+0000","last_became_peered":"2024-02-14T15:00:48.286355+0000","last_unstale":"2024-02-14T15:02:45.617044+0000","last_undegraded":"2024-02-14T15:02:45.617044+0000","last_fullsized":"2024-02-14T15:02:45.617044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:47.246995+0000","last_clean_scrub_stamp":"2024-02-14T15:00:47.246995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:48:31.024464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.767643+0000","last_change":"2024-02-14T15:00:49.271598+0000","last_active":"2024-02-14T15:02:45.767643+0000","last_peered":"2024-02-14T15:02:45.767643+0000","last_clean":"2024-02-14T15:02:45.767643+0000","last_became_active":"2024-02-14T15:00:49.270572+0000","last_became_peered":"2024-02-14T15:00:49.270572+0000","last_unstale":"2024-02-14T15:02:45.767643+0000","last_undegraded":"2024-02-14T15:02:45.767643+0000","last_fullsized":"2024-02-14T15:02:45.767643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:12.666557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.619943+0000","last_change":"2024-02-14T15:00:49.276697+0000","last_active":"2024-02-14T15:02:45.619943+0000","last_peered":"2024-02-14T15:02:45.619943+0000","last_clean":"2024-02-14T15:02:45.619943+0000","last_became_active":"2024-02-14T15:00:49.276279+0000","last_became_peered":"2024-02-14T15:00:49.276279+0000","last_unstale":"2024-02-14T15:02:45.619943+0000","last_undegraded":"2024-02-14T15:02:45.619943+0000","last_fullsized":"2024-02-14T15:02:45.619943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T16:11:31.824974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621107+0000","last_change":"2024-02-14T15:01:44.607605+0000","last_active":"2024-02-14T15:02:45.621107+0000","last_peered":"2024-02-14T15:02:45.621107+0000","last_clean":"2024-02-14T15:02:45.621107+0000","last_became_active":"2024-02-14T15:01:44.607171+0000","last_became_peered":"2024-02-14T15:01:44.607171+0000","last_unstale":"2024-02-14T15:02:45.621107+0000","last_undegraded":"2024-02-14T15:02:45.621107+0000","last_fullsized":"2024-02-14T15:02:45.621107+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:30:20.503814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621004+0000","last_change":"2024-02-14T15:00:49.274219+0000","last_active":"2024-02-14T15:02:45.621004+0000","last_peered":"2024-02-14T15:02:45.621004+0000","last_clean":"2024-02-14T15:02:45.621004+0000","last_became_active":"2024-02-14T15:00:49.273833+0000","last_became_peered":"2024-02-14T15:00:49.273833+0000","last_unstale":"2024-02-14T15:02:45.621004+0000","last_undegraded":"2024-02-14T15:02:45.621004+0000","last_fullsized":"2024-02-14T15:02:45.621004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:06:02.720273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:46.512067+0000","last_change":"2024-02-14T15:00:49.273576+0000","last_active":"2024-02-14T15:02:46.512067+0000","last_peered":"2024-02-14T15:02:46.512067+0000","last_clean":"2024-02-14T15:02:46.512067+0000","last_became_active":"2024-02-14T15:00:49.272920+0000","last_became_peered":"2024-02-14T15:00:49.272920+0000","last_unstale":"2024-02-14T15:02:46.512067+0000","last_undegraded":"2024-02-14T15:02:46.512067+0000","last_fullsized":"2024-02-14T15:02:46.512067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T15:22:03.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617100+0000","last_change":"2024-02-14T15:00:49.274549+0000","last_active":"2024-02-14T15:02:45.617100+0000","last_peered":"2024-02-14T15:02:45.617100+0000","last_clean":"2024-02-14T15:02:45.617100+0000","last_became_active":"2024-02-14T15:00:49.274097+0000","last_became_peered":"2024-02-14T15:00:49.274097+0000","last_unstale":"2024-02-14T15:02:45.617100+0000","last_undegraded":"2024-02-14T15:02:45.617100+0000","last_fullsized":"2024-02-14T15:02:45.617100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:47:44.017607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620430+0000","last_change":"2024-02-14T15:00:49.275699+0000","last_active":"2024-02-14T15:02:45.620430+0000","last_peered":"2024-02-14T15:02:45.620430+0000","last_clean":"2024-02-14T15:02:45.620430+0000","last_became_active":"2024-02-14T15:00:49.275361+0000","last_became_peered":"2024-02-14T15:00:49.275361+0000","last_unstale":"2024-02-14T15:02:45.620430+0000","last_undegraded":"2024-02-14T15:02:45.620430+0000","last_fullsized":"2024-02-14T15:02:45.620430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:34:02.638758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.621841+0000","last_change":"2024-02-14T15:00:49.275024+0000","last_active":"2024-02-14T15:02:45.621841+0000","last_peered":"2024-02-14T15:02:45.621841+0000","last_clean":"2024-02-14T15:02:45.621841+0000","last_became_active":"2024-02-14T15:00:49.274710+0000","last_became_peered":"2024-02-14T15:00:49.274710+0000","last_unstale":"2024-02-14T15:02:45.621841+0000","last_undegraded":"2024-02-14T15:02:45.621841+0000","last_fullsized":"2024-02-14T15:02:45.621841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:42.471112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.713794+0000","last_change":"2024-02-14T15:00:49.282638+0000","last_active":"2024-02-14T15:02:45.713794+0000","last_peered":"2024-02-14T15:02:45.713794+0000","last_clean":"2024-02-14T15:02:45.713794+0000","last_became_active":"2024-02-14T15:00:49.280223+0000","last_became_peered":"2024-02-14T15:00:49.280223+0000","last_unstale":"2024-02-14T15:02:45.713794+0000","last_undegraded":"2024-02-14T15:02:45.713794+0000","last_fullsized":"2024-02-14T15:02:45.713794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:40.250242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617680+0000","last_change":"2024-02-14T15:00:49.273025+0000","last_active":"2024-02-14T15:02:45.617680+0000","last_peered":"2024-02-14T15:02:45.617680+0000","last_clean":"2024-02-14T15:02:45.617680+0000","last_became_active":"2024-02-14T15:00:49.272700+0000","last_became_peered":"2024-02-14T15:00:49.272700+0000","last_unstale":"2024-02-14T15:02:45.617680+0000","last_undegraded":"2024-02-14T15:02:45.617680+0000","last_fullsized":"2024-02-14T15:02:45.617680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:08.677428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766013+0000","last_change":"2024-02-14T15:01:44.605957+0000","last_active":"2024-02-14T15:02:45.766013+0000","last_peered":"2024-02-14T15:02:45.766013+0000","last_clean":"2024-02-14T15:02:45.766013+0000","last_became_active":"2024-02-14T15:01:44.605398+0000","last_became_peered":"2024-02-14T15:01:44.605398+0000","last_unstale":"2024-02-14T15:02:45.766013+0000","last_undegraded":"2024-02-14T15:02:45.766013+0000","last_fullsized":"2024-02-14T15:02:45.766013+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:50:43.667496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.766620+0000","last_change":"2024-02-14T15:00:49.282178+0000","last_active":"2024-02-14T15:02:45.766620+0000","last_peered":"2024-02-14T15:02:45.766620+0000","last_clean":"2024-02-14T15:02:45.766620+0000","last_became_active":"2024-02-14T15:00:49.281804+0000","last_became_peered":"2024-02-14T15:00:49.281804+0000","last_unstale":"2024-02-14T15:02:45.766620+0000","last_undegraded":"2024-02-14T15:02:45.766620+0000","last_fullsized":"2024-02-14T15:02:45.766620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:36:58.672808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.617022+0000","last_change":"2024-02-14T15:00:49.269882+0000","last_active":"2024-02-14T15:02:45.617022+0000","last_peered":"2024-02-14T15:02:45.617022+0000","last_clean":"2024-02-14T15:02:45.617022+0000","last_became_active":"2024-02-14T15:00:49.269489+0000","last_became_peered":"2024-02-14T15:00:49.269489+0000","last_unstale":"2024-02-14T15:02:45.617022+0000","last_undegraded":"2024-02-14T15:02:45.617022+0000","last_fullsized":"2024-02-14T15:02:45.617022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:57:40.465044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-14T15:02:45.620120+0000","last_change":"2024-02-14T15:00:49.277688+0000","last_active":"2024-02-14T15:02:45.620120+0000","last_peered":"2024-02-14T15:02:45.620120+0000","last_clean":"2024-02-14T15:02:45.620120+0000","last_became_active":"2024-02-14T15:00:49.277348+0000","last_became_peered":"2024-02-14T15:00:49.277348+0000","last_unstale":"2024-02-14T15:02:45.620120+0000","last_undegraded":"2024-02-14T15:02:45.620120+0000","last_fullsized":"2024-02-14T15:02:45.620120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T15:00:48.248492+0000","last_clean_scrub_stamp":"2024-02-14T15:00:48.248492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:33.946319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68055,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":710,"num_read_kb":426,"num_write":170,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":63938,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":518659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72399999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.623},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.61299999999999999,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71999999999999997,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.52600000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.46899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81999999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.66700000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81200000000000006,"5min":1.155,"15min":1.155},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84799999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.42699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59099999999999997,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85099999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83799999999999997}]},{"osd":7,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.60499999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85699999999999998,"5min":1.121,"15min":1.121},"last":0.85699999999999998}]},{"osd":8,"last update":"Wed Feb 14 15:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.75700000000000001,"5min":0.875,"15min":0.875},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.89200000000000002,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.875}]},{"osd":9,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"min":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92000000000000004,"5min":1.002,"15min":1.002},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98499999999999999,"5min":1.012,"15min":1.012},"last":0.76200000000000001}]},{"osd":10,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.871,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.67600000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.95799999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.92100000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1744896,"data_stored":1170064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81399999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.65500000000000003},"min":{"1min":0.47499999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.625}]},{"osd":4,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65600000000000003,"15min":0.67300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1.054,"15min":1.054},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60399999999999998,"15min":0.64400000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":1.145,"15min":1.145},"last":0.35299999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97399999999999998,"5min":1.032,"15min":1.032},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98199999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":0.82999999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.071,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:00:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.59399999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0609999999999999,"5min":1.129,"15min":1.129},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81200000000000006,"15min":0.84999999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.872,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54000000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.04,"5min":1.169,"15min":1.169},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.83099999999999996,"15min":0.876},"min":{"1min":0.57199999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96199999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.66400000000000003}]},{"osd":11,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.873,"15min":0.876},"min":{"1min":0.71799999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0409999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.68200000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.083,"5min":1.141,"15min":1.141},"last":0.85099999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":522991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44900000000000001,"15min":0.46200000000000002},"min":{"1min":0.27900000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":1.22,"15min":1.22},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.745},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.54200000000000004,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.69499999999999995,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0629999999999999,"5min":1.141,"15min":1.141},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.69999999999999996,"15min":0.73699999999999999},"min":{"1min":0.45600000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97799999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.90600000000000003,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4370000000000001,"5min":19.152000000000001,"15min":19.152000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Feb 14 15:00:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.31},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.94199999999999995,"15min":0.86299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":18.734999999999999,"15min":18.734999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.86599999999999999,"15min":0.93999999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5069999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.94799999999999995,"15min":0.85999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.165,"5min":19.135999999999999,"15min":19.135999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0049999999999999,"5min":1.23,"15min":1.23},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed Feb 14 15:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.76400000000000001,"15min":0.79400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.79800000000000004,"15min":0.83099999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1100000000000001,"5min":1.264,"15min":1.264},"last":0.79600000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.58199999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0609999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96199999999999997,"5min":1.778,"15min":1.778},"last":0.78200000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":518815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80200000000000005,"5min":13.997,"15min":13.997},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79600000000000004}]},{"osd":2,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44,"15min":0.41799999999999998},"min":{"1min":0.36899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64000000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.32500000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64300000000000002,"15min":0.627},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.53200000000000003},"min":{"1min":0.373,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80800000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81200000000000006}]},{"osd":6,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63300000000000001,"15min":0.61599999999999999},"min":{"1min":0.47899999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.93700000000000006,"15min":1.175},"min":{"1min":0.56999999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93200000000000005,"5min":13.618,"15min":13.618},"last":0.86699999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.71699999999999997,"15min":0.68700000000000006},"min":{"1min":0.61499999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.69899999999999995,"15min":0.67100000000000004},"min":{"1min":0.627,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.59599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.68400000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.91100000000000003}]},{"osd":10,"last update":"Wed Feb 14 15:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71299999999999997,"15min":0.67300000000000004},"min":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.94899999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.73599999999999999,"15min":0.69999999999999996},"min":{"1min":0.67600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85599999999999998}]},{"osd":11,"last update":"Wed Feb 14 15:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.66300000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.98599999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.627,"5min":0.501,"15min":0.501},"max":{"1min":0.95099999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.96499999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1115955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.499}]},{"osd":1,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71599999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.48299999999999998},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86299999999999999,"5min":0.877,"15min":0.877},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.68899999999999995,"15min":0.65000000000000002},"min":{"1min":0.52900000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.32700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83199999999999996,"5min":0.872,"15min":0.872},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.58199999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91000000000000003,"5min":0.997,"15min":0.997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85499999999999998,"5min":1.095,"15min":1.095},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.78400000000000003,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:02:47 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.036,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.79300000000000004},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":1.139,"15min":1.139},"last":0.73599999999999999}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.94599999999999995,"5min":1.052,"15min":1.052},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.67300000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":4.0339999999999998,"5min":4.0339999999999998,"15min":4.0339999999999998},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.075,"5min":1.179,"15min":1.179},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.82699999999999996},"min":{"1min":0.626,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.014,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Feb 14 15:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90400000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.94399999999999995,"15min":0.94799999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.155,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84999999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":514071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.377},"min":{"1min":0.34000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.434,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63800000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.55400000000000005},"min":{"1min":0.47899999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80600000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78300000000000003,"15min":0.622},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1259999999999999,"5min":19.181000000000001,"15min":19.181000000000001},"last":0.626}]},{"osd":2,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.626,"15min":0.58499999999999996},"min":{"1min":0.53000000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85099999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.44600000000000001},"min":{"1min":0.32900000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.48499999999999999},"min":{"1min":0.308,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.79000000000000004,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93700000000000006,"5min":19.167000000000002,"15min":19.167000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.65800000000000003,"15min":0.623},"min":{"1min":0.52900000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Feb 14 15:02:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.56499999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.99199999999999999,"5min":1.107,"15min":1.107},"last":0.66800000000000004}]},{"osd":8,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0449999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.266,"15min":1.266},"last":0.76700000000000002}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68899999999999995,"15min":0.72499999999999998},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.92300000000000004,"5min":0.996,"15min":0.996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78600000000000003,"15min":0.77800000000000002},"min":{"1min":0.64100000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.748},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79700000000000004},"min":{"1min":0.57099999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.024,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.79500000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.66700000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":578476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.64900000000000002,"15min":0.53800000000000003},"min":{"1min":0.38300000000000001,"5min":0.33900000000000002,"15min":0.32800000000000001},"max":{"1min":6.5899999999999999,"5min":6.5899999999999999,"15min":6.5899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58399999999999996,"15min":0.51900000000000002},"min":{"1min":0.41599999999999998,"5min":0.33100000000000002,"15min":0.309},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.53600000000000003},"min":{"1min":0.47099999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93700000000000006,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.754},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.80800000000000005,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":18.332000000000001,"15min":18.332000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Wed Feb 14 15:02:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.315},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68899999999999995,"15min":0.61499999999999999},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.38200000000000001},"max":{"1min":1.0349999999999999,"5min":1.046,"15min":1.046},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:02:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91100000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.68799999999999994,"15min":0.59399999999999997},"min":{"1min":0.51300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70699999999999996,"15min":0.621},"min":{"1min":0.629,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95099999999999996,"5min":1.016,"15min":1.016},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Feb 14 15:03:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61299999999999999,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89400000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.73199999999999998,"15min":0.65800000000000003},"min":{"1min":0.53000000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65500000000000003,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0209999999999999,"5min":1.077,"15min":1.077},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60099999999999998,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55700000000000005}]},{"osd":8,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68500000000000005,"15min":0.66400000000000003},"min":{"1min":0.56299999999999994,"5min":0.376,"15min":0.376},"max":{"1min":0.996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.80000000000000004,"15min":0.748},"min":{"1min":0.69199999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0820000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67800000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.80400000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.0069999999999999,"5min":1.044,"15min":1.044},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.86399999999999999,"15min":0.84799999999999998},"min":{"1min":0.67800000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.65600000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0629999999999999,"5min":1.107,"15min":1.107},"last":0.81499999999999995}]},{"osd":11,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.93200000000000005,"15min":0.92100000000000004},"min":{"1min":0.84699999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.115,"5min":1.133,"15min":1.133},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.83199999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.84999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":515459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.31,"15min":0.253},"max":{"1min":0.76300000000000001,"5min":1.002,"15min":1.002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57499999999999996,"15min":0.52100000000000002},"min":{"1min":0.45200000000000001,"5min":0.32300000000000001,"15min":0.28199999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.53000000000000003},"min":{"1min":0.48999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.313,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72199999999999998,"5min":1.002,"15min":1.002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.58699999999999997,"15min":0.52500000000000002},"min":{"1min":0.47399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.5},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60699999999999998,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.87,"5min":1.694,"15min":1.694},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51300000000000001,"15min":0.53100000000000003},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76900000000000002,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.85599999999999998,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.36399999999999999}]},{"osd":7,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84399999999999997,"5min":1.139,"15min":1.139},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.1459999999999999,"5min":0.76000000000000001,"15min":0.69599999999999995},"min":{"1min":0.53500000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.8949999999999996,"5min":9.8949999999999996,"15min":9.8949999999999996},"last":0.53900000000000003}]},{"osd":8,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84599999999999997,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.71599999999999997,"15min":0.70999999999999996},"min":{"1min":0.498,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66700000000000004,"15min":0.69299999999999995},"min":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.109,"5min":1.153,"15min":1.153},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0649999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58699999999999997}]},{"osd":10,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.60499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.63900000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.67000000000000004}]},{"osd":11,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.751,"15min":0.748},"min":{"1min":0.61199999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94899999999999995,"5min":1.802,"15min":1.802},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75900000000000001,"15min":0.73299999999999998},"min":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.97399999999999998,"5min":1.242,"15min":1.242},"last":0.75900000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":514749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.47799999999999998},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77400000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.501}]},{"osd":1,"last update":"Wed Feb 14 15:02:22 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58099999999999996,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Feb 14 15:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.55800000000000005},"min":{"1min":0.45400000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Wed Feb 14 15:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91200000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.874}]},{"osd":5,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.46500000000000002,"15min":0.436},"min":{"1min":0.36099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.372,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93799999999999994,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.625},{"interface":"front","average":{"1min":1.284,"5min":0.748,"15min":0.65600000000000003},"min":{"1min":0.54000000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":13.955,"5min":13.955,"15min":13.955},"last":0.66400000000000003}]},{"osd":7,"last update":"Wed Feb 14 15:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93400000000000005,"5min":1.046,"15min":1.046},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81100000000000005,"15min":0.81599999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Feb 14 15:02:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94999999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70999999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.498,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":1.01,"15min":1.01},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.53700000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.749}]},{"osd":10,"last update":"Wed Feb 14 15:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.621,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0409999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.753,"15min":0.71799999999999997},"min":{"1min":0.54900000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.89200000000000002}]},{"osd":11,"last update":"Wed Feb 14 15:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.64700000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76900000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":514265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.375,"15min":0.32700000000000001},"min":{"1min":0.29599999999999999,"5min":0.20399999999999999,"15min":0.191},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.35599999999999998},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.66900000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.76500000000000001,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.33600000000000002},"max":{"1min":0.91000000000000003,"5min":19.896999999999998,"15min":19.896999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.77400000000000002,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.99399999999999999,"5min":19.911999999999999,"15min":19.911999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Feb 14 15:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59499999999999997,"15min":0.53700000000000003},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":0.995,"15min":0.995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60899999999999999,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.751,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Feb 14 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60399999999999998,"15min":0.57999999999999996},"min":{"1min":0.46400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.879,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.498}]},{"osd":5,"last update":"Wed Feb 14 15:02:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.80500000000000005,"15min":1.3919999999999999},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":19.879999999999999,"15min":19.879999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Feb 14 15:02:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.32300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.66500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.441},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.501,"15min":0.48499999999999999},"min":{"1min":0.33500000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.39400000000000002}]},{"osd":7,"last update":"Wed Feb 14 15:02:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.56999999999999995}]},{"osd":8,"last update":"Wed Feb 14 15:00:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70499999999999996}]},{"osd":9,"last update":"Wed Feb 14 15:02:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64300000000000002,"15min":0.67200000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.64500000000000002,"15min":0.67100000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47299999999999998}]},{"osd":10,"last update":"Wed Feb 14 15:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92700000000000005,"5min":1.391,"15min":1.391},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.93600000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.628}]},{"osd":11,"last update":"Wed Feb 14 15:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86199999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67800000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":583166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53100000000000003,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.42199999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72299999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.379}]},{"osd":2,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.64200000000000002,"15min":0.59999999999999998},"min":{"1min":0.56100000000000005,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.91900000000000004,"5min":0.997,"15min":0.997},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89500000000000002,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89100000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Feb 14 15:02:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65400000000000003,"15min":0.58499999999999996},"min":{"1min":0.52500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.67900000000000005,"15min":0.621},"min":{"1min":0.50700000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.92400000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":0.79000000000000004}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.012,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52700000000000002}]},{"osd":8,"last update":"Wed Feb 14 15:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.74399999999999999,"15min":0.71699999999999997},"min":{"1min":0.626,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0629999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.1299999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.84099999999999997}]},{"osd":9,"last update":"Wed Feb 14 15:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":1.04,"15min":1.04},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0609999999999999,"5min":1.105,"15min":1.105},"last":0.90200000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.56999999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.012,"5min":1.032,"15min":1.032},"last":0.749}]},{"osd":11,"last update":"Wed Feb 14 15:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.876},{"interface":"front","average":{"1min":0.877,"5min":0.872,"15min":0.871},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71699999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1671168,"data_stored":1110229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53600000000000003,"15min":0.49199999999999999},"min":{"1min":0.33700000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.48099999999999998},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79700000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56599999999999995,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.84899999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.52700000000000002},"min":{"1min":0.46500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78300000000000003,"5min":0.876,"15min":0.876},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40600000000000003,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.61199999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41599999999999998,"15min":0.39300000000000002},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.505},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83999999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Feb 14 15:02:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83199999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":0.878,"15min":0.878},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Feb 14 15:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.45600000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81899999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63200000000000001,"15min":0.58399999999999996},"min":{"1min":0.55100000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82699999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Wed Feb 14 15:02:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89200000000000002,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70999999999999996}]},{"osd":8,"last update":"Wed Feb 14 15:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60199999999999998,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59699999999999998,"15min":0.56399999999999995},"min":{"1min":0.45200000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Feb 14 15:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95799999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.038,"15min":1.038},"last":0.90200000000000002}]},{"osd":10,"last update":"Wed Feb 14 15:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.59299999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91900000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75,"15min":0.73099999999999998},"min":{"1min":0.65600000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93000000000000005,"5min":1.032,"15min":1.032},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T15:03:14.671 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:03:14.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 -- 172.21.15.17:0/3845109954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c40ff350 msgr2=0x7f14c40ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 --2- 172.21.15.17:0/3845109954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c40ff350 0x7f14c40ff7d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009900 tx=0x7f14b402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:14.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 -- 172.21.15.17:0/3845109954 shutdown_connections 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 --2- 172.21.15.17:0/3845109954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c40ff350 0x7f14c40ff7d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 --2- 172.21.15.17:0/3845109954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14c40ffd10 0x7f14c41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 --2- 172.21.15.17:0/3845109954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14c41028e0 0x7f14c4102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 -- 172.21.15.17:0/3845109954 >> 172.21.15.17:0/3845109954 conn(0x7f14c40faeb0 msgr2=0x7f14c40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 -- 172.21.15.17:0/3845109954 shutdown_connections 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.807+0000 7f14cb33e640 1 -- 172.21.15.17:0/3845109954 wait complete. 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 Processor -- start 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 -- start start 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14c40ff350 0x7f14c41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14c40ffd10 0x7f14c4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c41028e0 0x7f14c419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c419edc0 con 0x7f14c41028e0 2024-02-14T15:03:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c419ef30 con 0x7f14c40ffd10 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c419f0f0 con 0x7f14c40ff350 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c41028e0 0x7f14c419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c41028e0 0x7f14c419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59080/0 (socket says 172.21.15.17:59080) 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 -- 172.21.15.17:0/2911646062 learned_addr learned my addr 172.21.15.17:0/2911646062 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c88d9640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14c40ffd10 0x7f14c4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c90da640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14c40ff350 0x7f14c41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 -- 172.21.15.17:0/2911646062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14c40ff350 msgr2=0x7f14c41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14c40ff350 0x7f14c41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 -- 172.21.15.17:0/2911646062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14c40ffd10 msgr2=0x7f14c4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14c40ffd10 0x7f14c4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 -- 172.21.15.17:0/2911646062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4009560 con 0x7f14c41028e0 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c90da640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14c40ff350 0x7f14c41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c88d9640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14c40ffd10 0x7f14c4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14c98db640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c41028e0 0x7f14c419e880 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00eaa0 tx=0x7f14bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14ba7fc640 1 -- 172.21.15.17:0/2911646062 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc00cdb0 con 0x7f14c41028e0 2024-02-14T15:03:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14ba7fc640 1 -- 172.21.15.17:0/2911646062 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14bc005020 con 0x7f14c41028e0 2024-02-14T15:03:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.808+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c419f3d0 con 0x7f14c41028e0 2024-02-14T15:03:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.809+0000 7f14ba7fc640 1 -- 172.21.15.17:0/2911646062 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0116b0 con 0x7f14c41028e0 2024-02-14T15:03:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.809+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c419f890 con 0x7f14c41028e0 2024-02-14T15:03:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.809+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c4108820 con 0x7f14c41028e0 2024-02-14T15:03:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.810+0000 7f14ba7fc640 1 -- 172.21.15.17:0/2911646062 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f14bc005520 con 0x7f14c41028e0 2024-02-14T15:03:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.811+0000 7f14ba7fc640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f14b00a2150 0x7f14b00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.811+0000 7f14c90da640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f14b00a2150 0x7f14b00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.811+0000 7f14ba7fc640 1 -- 172.21.15.17:0/2911646062 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f14bc0ce470 con 0x7f14c41028e0 2024-02-14T15:03:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.812+0000 7f14c90da640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f14b00a2150 0x7f14b00a4610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f14c4069650 tx=0x7f14c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:14.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.816+0000 7f14ba7fc640 1 -- 172.21.15.17:0/2911646062 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14bc096f90 con 0x7f14c41028e0 2024-02-14T15:03:14.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.997+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14c410cca0 con 0x7f14c41028e0 2024-02-14T15:03:14.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:14.999+0000 7f14ba7fc640 1 -- 172.21.15.17:0/2911646062 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f14bc01f090 con 0x7f14c41028e0 2024-02-14T15:03:15.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:15.000 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:02:45.606961+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:03:15.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f14b00a2150 msgr2=0x7f14b00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f14b00a2150 0x7f14b00a4610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f14c4069650 tx=0x7f14c0005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c41028e0 msgr2=0x7f14c419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c41028e0 0x7f14c419e880 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00eaa0 tx=0x7f14bc00ef60 comp rx=0 tx=0).stop 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 shutdown_connections 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f14b00a2150 0x7f14b00a4610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14c41028e0 0x7f14c419e880 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14c40ffd10 0x7f14c4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 --2- 172.21.15.17:0/2911646062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14c40ff350 0x7f14c41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 >> 172.21.15.17:0/2911646062 conn(0x7f14c40faeb0 msgr2=0x7f14c41059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 shutdown_connections 2024-02-14T15:03:15.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.001+0000 7f14cb33e640 1 -- 172.21.15.17:0/2911646062 wait complete. 2024-02-14T15:03:15.011 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:14 smithi017 ceph-mon[32549]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:15.011 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:14 smithi017 ceph-mon[32549]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:15.013 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:03:15.013 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:03:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 -- 172.21.15.17:0/1253227690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 msgr2=0x7f62a81017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 --2- 172.21.15.17:0/1253227690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81017c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f629c009900 tx=0x7f629c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 -- 172.21.15.17:0/1253227690 shutdown_connections 2024-02-14T15:03:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 --2- 172.21.15.17:0/1253227690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81017c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 --2- 172.21.15.17:0/1253227690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a8101d00 0x7f62a8108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 --2- 172.21.15.17:0/1253227690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a8100150 0x7f62a8100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 -- 172.21.15.17:0/1253227690 >> 172.21.15.17:0/1253227690 conn(0x7f62a80fb6c0 msgr2=0x7f62a80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 -- 172.21.15.17:0/1253227690 shutdown_connections 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 -- 172.21.15.17:0/1253227690 wait complete. 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 Processor -- start 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.118+0000 7f62acef4640 1 -- start start 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a8100150 0x7f62a81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a8101d00 0x7f62a819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a819ed90 con 0x7f62a8101340 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a819ef00 con 0x7f62a8101d00 2024-02-14T15:03:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a819f0c0 con 0x7f62a8100150 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59096/0 (socket says 172.21.15.17:59096) 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 -- 172.21.15.17:0/1949298212 learned_addr learned my addr 172.21.15.17:0/1949298212 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a6d9d640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a8101d00 0x7f62a819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a659c640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a8100150 0x7f62a81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 -- 172.21.15.17:0/1949298212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a8100150 msgr2=0x7f62a81972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a8100150 0x7f62a81972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 -- 172.21.15.17:0/1949298212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a8101d00 msgr2=0x7f62a819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a8101d00 0x7f62a819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 -- 172.21.15.17:0/1949298212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c009560 con 0x7f62a8101340 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a6d9d640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a8101d00 0x7f62a819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62a5d9b640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81977e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f629c005ab0 tx=0x7f629c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62977fe640 1 -- 172.21.15.17:0/1949298212 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0044e0 con 0x7f62a8101340 2024-02-14T15:03:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62977fe640 1 -- 172.21.15.17:0/1949298212 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f629c0054e0 con 0x7f62a8101340 2024-02-14T15:03:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a819f340 con 0x7f62a8101340 2024-02-14T15:03:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62977fe640 1 -- 172.21.15.17:0/1949298212 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c03e450 con 0x7f62a8101340 2024-02-14T15:03:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.119+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a819f8a0 con 0x7f62a8101340 2024-02-14T15:03:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.121+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a810bb60 con 0x7f62a8101340 2024-02-14T15:03:15.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.122+0000 7f62977fe640 1 -- 172.21.15.17:0/1949298212 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f629c0035f0 con 0x7f62a8101340 2024-02-14T15:03:15.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.122+0000 7f62977fe640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f62900a21d0 0x7f62900a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.122+0000 7f62a659c640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f62900a21d0 0x7f62900a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.123+0000 7f62977fe640 1 -- 172.21.15.17:0/1949298212 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f629c0f05f0 con 0x7f62a8101340 2024-02-14T15:03:15.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.123+0000 7f62a659c640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f62900a21d0 0x7f62900a4690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6298005c40 tx=0x7f6298005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.127+0000 7f62977fe640 1 -- 172.21.15.17:0/1949298212 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f629c0b9220 con 0x7f62a8101340 2024-02-14T15:03:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:14 smithi050 ceph-mon[36512]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:14 smithi050 ceph-mon[36512]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:14 smithi076 ceph-mon[37057]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:14 smithi076 ceph-mon[37057]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:15.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.307+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62a819fd30 con 0x7f62a8101340 2024-02-14T15:03:15.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.307+0000 7f62977fe640 1 -- 172.21.15.17:0/1949298212 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f629c0bc8e0 con 0x7f62a8101340 2024-02-14T15:03:15.308 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:03:15.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f62900a21d0 msgr2=0x7f62900a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f62900a21d0 0x7f62900a4690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6298005c40 tx=0x7f6298005bd0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 msgr2=0x7f62a81977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81977e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f629c005ab0 tx=0x7f629c0042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 shutdown_connections 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f62900a21d0 0x7f62900a4690 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62a8101340 0x7f62a81977e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62a8101d00 0x7f62a819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 --2- 172.21.15.17:0/1949298212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a8100150 0x7f62a81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.309+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 >> 172.21.15.17:0/1949298212 conn(0x7f62a80fb6c0 msgr2=0x7f62a8068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.310+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 shutdown_connections 2024-02-14T15:03:15.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.310+0000 7f62acef4640 1 -- 172.21.15.17:0/1949298212 wait complete. 2024-02-14T15:03:15.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:03:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.426+0000 7f8e13655640 1 -- 172.21.15.17:0/2829328475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102df0 msgr2=0x7f8e0c109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.426+0000 7f8e13655640 1 --2- 172.21.15.17:0/2829328475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102df0 0x7f8e0c109680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009900 tx=0x7f8dfc02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- 172.21.15.17:0/2829328475 shutdown_connections 2024-02-14T15:03:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 --2- 172.21.15.17:0/2829328475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102df0 0x7f8e0c109680 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 --2- 172.21.15.17:0/2829328475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0c102430 0x7f8e0c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 --2- 172.21.15.17:0/2829328475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e0c101240 0x7f8e0c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- 172.21.15.17:0/2829328475 >> 172.21.15.17:0/2829328475 conn(0x7f8e0c0fc7b0 msgr2=0x7f8e0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- 172.21.15.17:0/2829328475 shutdown_connections 2024-02-14T15:03:15.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- 172.21.15.17:0/2829328475 wait complete. 2024-02-14T15:03:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 Processor -- start 2024-02-14T15:03:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- start start 2024-02-14T15:03:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0c101240 0x7f8e0c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102430 0x7f8e0c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e0c102df0 0x7f8e0c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c19eda0 con 0x7f8e0c102430 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c19ef10 con 0x7f8e0c102df0 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e13655640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c19f0d0 con 0x7f8e0c101240 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.427+0000 7f8e10bf0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102430 0x7f8e0c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e113f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0c101240 0x7f8e0c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102430 0x7f8e0c1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59124/0 (socket says 172.21.15.17:59124) 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 -- 172.21.15.17:0/3562073566 learned_addr learned my addr 172.21.15.17:0/3562073566 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e11bf2640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e0c102df0 0x7f8e0c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 -- 172.21.15.17:0/3562073566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0c101240 msgr2=0x7f8e0c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0c101240 0x7f8e0c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 -- 172.21.15.17:0/3562073566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e0c102df0 msgr2=0x7f8e0c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e0c102df0 0x7f8e0c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 -- 172.21.15.17:0/3562073566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc009560 con 0x7f8e0c102430 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e10bf0640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102430 0x7f8e0c1977f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000c970 tx=0x7f8e0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e0a7fc640 1 -- 172.21.15.17:0/3562073566 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00007bb0 con 0x7f8e0c102430 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e0a7fc640 1 -- 172.21.15.17:0/3562073566 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e00007d10 con 0x7f8e0c102430 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c19f3b0 con 0x7f8e0c102430 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e0a7fc640 1 -- 172.21.15.17:0/3562073566 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0000f450 con 0x7f8e0c102430 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.428+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c19f870 con 0x7f8e0c102430 2024-02-14T15:03:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.429+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0c108490 con 0x7f8e0c102430 2024-02-14T15:03:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.430+0000 7f8e0a7fc640 1 -- 172.21.15.17:0/3562073566 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8e00004f70 con 0x7f8e0c102430 2024-02-14T15:03:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.431+0000 7f8e0a7fc640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8df40a2150 0x7f8df40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.431+0000 7f8e113f1640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8df40a2150 0x7f8df40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.431+0000 7f8e0a7fc640 1 -- 172.21.15.17:0/3562073566 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f8e000cde10 con 0x7f8e0c102430 2024-02-14T15:03:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.431+0000 7f8e113f1640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8df40a2150 0x7f8df40a4610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c102290 tx=0x7f8e04007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.435+0000 7f8e0a7fc640 1 -- 172.21.15.17:0/3562073566 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8e000d1030 con 0x7f8e0c102430 2024-02-14T15:03:15.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.615+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e0c02cdb0 con 0x7f8e0c102430 2024-02-14T15:03:15.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.616+0000 7f8e0a7fc640 1 -- 172.21.15.17:0/3562073566 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e00096980 con 0x7f8e0c102430 2024-02-14T15:03:15.617 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8df40a2150 msgr2=0x7f8df40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8df40a2150 0x7f8df40a4610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c102290 tx=0x7f8e04007600 comp rx=0 tx=0).stop 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102430 msgr2=0x7f8e0c1977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102430 0x7f8e0c1977f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000c970 tx=0x7f8e0000ce30 comp rx=0 tx=0).stop 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 shutdown_connections 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f8df40a2150 0x7f8df40a4610 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c102430 0x7f8e0c1977f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e0c102df0 0x7f8e0c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 --2- 172.21.15.17:0/3562073566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0c101240 0x7f8e0c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 >> 172.21.15.17:0/3562073566 conn(0x7f8e0c0fc7b0 msgr2=0x7f8e0c105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 shutdown_connections 2024-02-14T15:03:15.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.618+0000 7f8e13655640 1 -- 172.21.15.17:0/3562073566 wait complete. 2024-02-14T15:03:15.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:03:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.734+0000 7f352772c640 1 -- 172.21.15.17:0/1872333765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 msgr2=0x7f35201017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.734+0000 7f352772c640 1 --2- 172.21.15.17:0/1872333765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f35201017c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3510009900 tx=0x7f351002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.734+0000 7f352772c640 1 -- 172.21.15.17:0/1872333765 shutdown_connections 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.734+0000 7f352772c640 1 --2- 172.21.15.17:0/1872333765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f35201017c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.734+0000 7f352772c640 1 --2- 172.21.15.17:0/1872333765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3520101d00 0x7f3520108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.734+0000 7f352772c640 1 --2- 172.21.15.17:0/1872333765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3520100150 0x7f3520100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.734+0000 7f352772c640 1 -- 172.21.15.17:0/1872333765 >> 172.21.15.17:0/1872333765 conn(0x7f35200fb6c0 msgr2=0x7f35200fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 -- 172.21.15.17:0/1872333765 shutdown_connections 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 -- 172.21.15.17:0/1872333765 wait complete. 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 Processor -- start 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 -- start start 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3520100150 0x7f35201972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f3520197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3520101d00 0x7f352019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019ede0 con 0x7f3520101340 2024-02-14T15:03:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019ef50 con 0x7f3520100150 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f352772c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019f110 con 0x7f3520101d00 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f3524cc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f3520197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f3524cc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f3520197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59152/0 (socket says 172.21.15.17:59152) 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.735+0000 7f3524cc7640 1 -- 172.21.15.17:0/3109139893 learned_addr learned my addr 172.21.15.17:0/3109139893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3525cc9640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3520101d00 0x7f352019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f35254c8640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3520100150 0x7f35201972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3524cc7640 1 -- 172.21.15.17:0/3109139893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3520101d00 msgr2=0x7f352019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3524cc7640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3520101d00 0x7f352019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3524cc7640 1 -- 172.21.15.17:0/3109139893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3520100150 msgr2=0x7f35201972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3524cc7640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3520100150 0x7f35201972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3524cc7640 1 -- 172.21.15.17:0/3109139893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3510009560 con 0x7f3520101340 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3525cc9640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3520101d00 0x7f352019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f3524cc7640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f3520197830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3510005ab0 tx=0x7f35100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f35167fc640 1 -- 172.21.15.17:0/3109139893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100044e0 con 0x7f3520101340 2024-02-14T15:03:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352019f390 con 0x7f3520101340 2024-02-14T15:03:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f35167fc640 1 -- 172.21.15.17:0/3109139893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35100054e0 con 0x7f3520101340 2024-02-14T15:03:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f35167fc640 1 -- 172.21.15.17:0/3109139893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351003e450 con 0x7f3520101340 2024-02-14T15:03:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.736+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352019f8a0 con 0x7f3520101340 2024-02-14T15:03:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.737+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35201074e0 con 0x7f3520101340 2024-02-14T15:03:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.738+0000 7f35167fc640 1 -- 172.21.15.17:0/3109139893 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f35100035f0 con 0x7f3520101340 2024-02-14T15:03:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.739+0000 7f35167fc640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f350c0a2150 0x7f350c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.739+0000 7f35254c8640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f350c0a2150 0x7f350c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.739+0000 7f35167fc640 1 -- 172.21.15.17:0/3109139893 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f35100f06a0 con 0x7f3520101340 2024-02-14T15:03:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.739+0000 7f35254c8640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f350c0a2150 0x7f350c0a4610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f351c005db0 tx=0x7f351c005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:15.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.743+0000 7f35167fc640 1 -- 172.21.15.17:0/3109139893 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f35100b92d0 con 0x7f3520101340 2024-02-14T15:03:15.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.926+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f352010ceb0 con 0x7f3520101340 2024-02-14T15:03:15.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.926+0000 7f35167fc640 1 -- 172.21.15.17:0/3109139893 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f35100bc990 con 0x7f3520101340 2024-02-14T15:03:15.928 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:03:15.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f350c0a2150 msgr2=0x7f350c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f350c0a2150 0x7f350c0a4610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f351c005db0 tx=0x7f351c005d40 comp rx=0 tx=0).stop 2024-02-14T15:03:15.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 msgr2=0x7f3520197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f3520197830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3510005ab0 tx=0x7f35100042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 shutdown_connections 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f350c0a2150 0x7f350c0a4610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3520101340 0x7f3520197830 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3520101d00 0x7f352019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 --2- 172.21.15.17:0/3109139893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3520100150 0x7f35201972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 >> 172.21.15.17:0/3109139893 conn(0x7f35200fb6c0 msgr2=0x7f35201046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 shutdown_connections 2024-02-14T15:03:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:15.928+0000 7f352772c640 1 -- 172.21.15.17:0/3109139893 wait complete. 2024-02-14T15:03:15.939 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:15 smithi017 ceph-mon[32549]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:15.939 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:15 smithi017 ceph-mon[32549]: pgmap v369: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-02-14T15:03:15.939 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2911646062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:15.939 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1949298212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:15.939 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:15 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3562073566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:15.940 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:03:16.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.045+0000 7efe9690c640 1 -- 172.21.15.17:0/1642436217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90100150 msgr2=0x7efe90100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.045+0000 7efe9690c640 1 --2- 172.21.15.17:0/1642436217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90100150 0x7efe90100570 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efe80009900 tx=0x7efe8002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- 172.21.15.17:0/1642436217 shutdown_connections 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 --2- 172.21.15.17:0/1642436217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe90101d00 0x7efe90108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 --2- 172.21.15.17:0/1642436217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe90101340 0x7efe901017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 --2- 172.21.15.17:0/1642436217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90100150 0x7efe90100570 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- 172.21.15.17:0/1642436217 >> 172.21.15.17:0/1642436217 conn(0x7efe900fb6c0 msgr2=0x7efe900fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- 172.21.15.17:0/1642436217 shutdown_connections 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- 172.21.15.17:0/1642436217 wait complete. 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 Processor -- start 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- start start 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe90100150 0x7efe9010aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe90101340 0x7efe90109050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90101d00 0x7efe90109590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90109cf0 con 0x7efe90101d00 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90109e60 con 0x7efe90101340 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe9690c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe901a13d0 con 0x7efe90100150 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.046+0000 7efe94ea9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90101d00 0x7efe90109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90101d00 0x7efe90109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59162/0 (socket says 172.21.15.17:59162) 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 -- 172.21.15.17:0/2952840039 learned_addr learned my addr 172.21.15.17:0/2952840039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe8ffff640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe90100150 0x7efe9010aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe8f7fe640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe90101340 0x7efe90109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 -- 172.21.15.17:0/2952840039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe90100150 msgr2=0x7efe9010aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe90100150 0x7efe9010aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 -- 172.21.15.17:0/2952840039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe90101340 msgr2=0x7efe90109050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe90101340 0x7efe90109050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 -- 172.21.15.17:0/2952840039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80009560 con 0x7efe90101d00 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe8f7fe640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe90101340 0x7efe90109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe94ea9640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90101d00 0x7efe90109590 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efe8800db20 tx=0x7efe8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe8d7fa640 1 -- 172.21.15.17:0/2952840039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800bea0 con 0x7efe90101d00 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe8d7fa640 1 -- 172.21.15.17:0/2952840039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe88009940 con 0x7efe90101d00 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe901a1680 con 0x7efe90101d00 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe8d7fa640 1 -- 172.21.15.17:0/2952840039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880106c0 con 0x7efe90101d00 2024-02-14T15:03:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.047+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe901a1b40 con 0x7efe90101d00 2024-02-14T15:03:16.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.048+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe901074e0 con 0x7efe90101d00 2024-02-14T15:03:16.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.049+0000 7efe8d7fa640 1 -- 172.21.15.17:0/2952840039 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7efe88004240 con 0x7efe90101d00 2024-02-14T15:03:16.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.050+0000 7efe8d7fa640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe780a2150 0x7efe780a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.050+0000 7efe8ffff640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe780a2150 0x7efe780a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.050+0000 7efe8d7fa640 1 -- 172.21.15.17:0/2952840039 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7efe880cdfe0 con 0x7efe90101d00 2024-02-14T15:03:16.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.050+0000 7efe8ffff640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe780a2150 0x7efe780a4610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe80006010 tx=0x7efe80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:16.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.054+0000 7efe8d7fa640 1 -- 172.21.15.17:0/2952840039 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe88096aa0 con 0x7efe90101d00 2024-02-14T15:03:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:15 smithi050 ceph-mon[36512]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:15 smithi050 ceph-mon[36512]: pgmap v369: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-02-14T15:03:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2911646062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1949298212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:15 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3562073566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.233+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efe9002ce50 con 0x7efe90101d00 2024-02-14T15:03:16.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.233+0000 7efe8d7fa640 1 -- 172.21.15.17:0/2952840039 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7efe88017020 con 0x7efe90101d00 2024-02-14T15:03:16.234 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:03:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:15 smithi076 ceph-mon[37057]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T15:03:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:15 smithi076 ceph-mon[37057]: pgmap v369: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-02-14T15:03:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2911646062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1949298212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:03:16.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:15 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3562073566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe780a2150 msgr2=0x7efe780a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe780a2150 0x7efe780a4610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe80006010 tx=0x7efe80000bc0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90101d00 msgr2=0x7efe90109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90101d00 0x7efe90109590 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efe8800db20 tx=0x7efe8800b260 comp rx=0 tx=0).stop 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 shutdown_connections 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7efe780a2150 0x7efe780a4610 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe90101d00 0x7efe90109590 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe90101340 0x7efe90109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 --2- 172.21.15.17:0/2952840039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe90100150 0x7efe9010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 >> 172.21.15.17:0/2952840039 conn(0x7efe900fb6c0 msgr2=0x7efe901046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 shutdown_connections 2024-02-14T15:03:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.235+0000 7efe9690c640 1 -- 172.21.15.17:0/2952840039 wait complete. 2024-02-14T15:03:16.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:03:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 -- 172.21.15.17:0/1568913520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 msgr2=0x7fa668104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 --2- 172.21.15.17:0/1568913520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa668104140 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa664009900 tx=0x7fa66402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 -- 172.21.15.17:0/1568913520 shutdown_connections 2024-02-14T15:03:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 --2- 172.21.15.17:0/1568913520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa668106fb0 0x7fa668109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 --2- 172.21.15.17:0/1568913520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa668104680 0x7fa668106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 --2- 172.21.15.17:0/1568913520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa668104140 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 -- 172.21.15.17:0/1568913520 >> 172.21.15.17:0/1568913520 conn(0x7fa6680fb760 msgr2=0x7fa6680fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 -- 172.21.15.17:0/1568913520 shutdown_connections 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.349+0000 7fa6702eb640 1 -- 172.21.15.17:0/1568913520 wait complete. 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 Processor -- start 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 -- start start 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa6681016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa668104680 0x7fa6680ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa668106fb0 0x7fa668100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6681009a0 con 0x7fa668101d50 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668100b10 con 0x7fa668106fb0 2024-02-14T15:03:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa6702eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6681a1320 con 0x7fa668104680 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa6681016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa6681016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59178/0 (socket says 172.21.15.17:59178) 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 -- 172.21.15.17:0/471781738 learned_addr learned my addr 172.21.15.17:0/471781738 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 -- 172.21.15.17:0/471781738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa668104680 msgr2=0x7fa6680ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa668104680 0x7fa6680ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e888640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa668106fb0 0x7fa668100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 -- 172.21.15.17:0/471781738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa668106fb0 msgr2=0x7fa668100240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa668106fb0 0x7fa668100240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e087640 1 -- 172.21.15.17:0/471781738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa664009560 con 0x7fa668101d50 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.350+0000 7fa66e888640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa668106fb0 0x7fa668100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.351+0000 7fa66e087640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa6681016c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa664036040 tx=0x7fa6640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.351+0000 7fa65f7fe640 1 -- 172.21.15.17:0/471781738 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640044e0 con 0x7fa668101d50 2024-02-14T15:03:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.351+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6681a1570 con 0x7fa668101d50 2024-02-14T15:03:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.351+0000 7fa65f7fe640 1 -- 172.21.15.17:0/471781738 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6640054e0 con 0x7fa668101d50 2024-02-14T15:03:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.351+0000 7fa65f7fe640 1 -- 172.21.15.17:0/471781738 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66403e450 con 0x7fa668101d50 2024-02-14T15:03:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.351+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6681a1a80 con 0x7fa668101d50 2024-02-14T15:03:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.352+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6680fd250 con 0x7fa668101d50 2024-02-14T15:03:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.353+0000 7fa65f7fe640 1 -- 172.21.15.17:0/471781738 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa664003680 con 0x7fa668101d50 2024-02-14T15:03:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.354+0000 7fa65f7fe640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6540a2150 0x7fa6540a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.354+0000 7fa65f7fe640 1 -- 172.21.15.17:0/471781738 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fa6640f0cd0 con 0x7fa668101d50 2024-02-14T15:03:16.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.354+0000 7fa66d886640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6540a2150 0x7fa6540a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.354+0000 7fa66d886640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6540a2150 0x7fa6540a4610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa668101430 tx=0x7fa658005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:16.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.357+0000 7fa65f7fe640 1 -- 172.21.15.17:0/471781738 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6640b98a0 con 0x7fa668101d50 2024-02-14T15:03:16.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.578+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa66810cc70 con 0x7fa668101d50 2024-02-14T15:03:16.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.578+0000 7fa65f7fe640 1 -- 172.21.15.17:0/471781738 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11317 (secure 0 0 0) 0x7fa6640b9020 con 0x7fa668101d50 2024-02-14T15:03:16.579 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:16.579 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:03:09.607099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:03:16.579 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2024-02-14T15:03:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.580+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6540a2150 msgr2=0x7fa6540a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6540a2150 0x7fa6540a4610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa668101430 tx=0x7fa658005bf0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 msgr2=0x7fa6681016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa6681016c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa664036040 tx=0x7fa6640042f0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 shutdown_connections 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fa6540a2150 0x7fa6540a4610 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa668106fb0 0x7fa668100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa668104680 0x7fa6680ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 --2- 172.21.15.17:0/471781738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa668101d50 0x7fa6681016c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 >> 172.21.15.17:0/471781738 conn(0x7fa6680fb760 msgr2=0x7fa668102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 shutdown_connections 2024-02-14T15:03:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.581+0000 7fa6702eb640 1 -- 172.21.15.17:0/471781738 wait complete. 2024-02-14T15:03:16.593 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:03:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 -- 172.21.15.17:0/1834424754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c106f40 msgr2=0x7fcd5c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/1834424754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c106f40 0x7fcd5c109390 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009ca0 tx=0x7fcd5402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 -- 172.21.15.17:0/1834424754 shutdown_connections 2024-02-14T15:03:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/1834424754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c106f40 0x7fcd5c109390 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/1834424754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd5c104610 0x7fcd5c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/1834424754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd5c101ce0 0x7fcd5c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 -- 172.21.15.17:0/1834424754 >> 172.21.15.17:0/1834424754 conn(0x7fcd5c0fb6f0 msgr2=0x7fcd5c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 -- 172.21.15.17:0/1834424754 shutdown_connections 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.694+0000 7fcd63d0c640 1 -- 172.21.15.17:0/1834424754 wait complete. 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 Processor -- start 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 -- start start 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c101ce0 0x7fcd5c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd5c104610 0x7fcd5c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd5c106f40 0x7fcd5c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c19eda0 con 0x7fcd5c101ce0 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c19ef10 con 0x7fcd5c106f40 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd63d0c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c19f0d0 con 0x7fcd5c104610 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c101ce0 0x7fcd5c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c101ce0 0x7fcd5c1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59194/0 (socket says 172.21.15.17:59194) 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 -- 172.21.15.17:0/3931562403 learned_addr learned my addr 172.21.15.17:0/3931562403 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd612a7640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd5c104610 0x7fcd5c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd622a9640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd5c106f40 0x7fcd5c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 -- 172.21.15.17:0/3931562403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd5c104610 msgr2=0x7fcd5c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd5c104610 0x7fcd5c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 -- 172.21.15.17:0/3931562403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd5c106f40 msgr2=0x7fcd5c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd5c106f40 0x7fcd5c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd61aa8640 1 -- 172.21.15.17:0/3931562403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd54009900 con 0x7fcd5c101ce0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.695+0000 7fcd612a7640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd5c104610 0x7fcd5c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.696+0000 7fcd61aa8640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c101ce0 0x7fcd5c1972b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800d950 tx=0x7fcd5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.696+0000 7fcd52ffd640 1 -- 172.21.15.17:0/3931562403 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58009a30 con 0x7fcd5c101ce0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.696+0000 7fcd52ffd640 1 -- 172.21.15.17:0/3931562403 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd58009b90 con 0x7fcd5c101ce0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.696+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c19f3b0 con 0x7fcd5c101ce0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.696+0000 7fcd52ffd640 1 -- 172.21.15.17:0/3931562403 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004a00 con 0x7fcd5c101ce0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.696+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c19f910 con 0x7fcd5c101ce0 2024-02-14T15:03:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.697+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c0fd1e0 con 0x7fcd5c101ce0 2024-02-14T15:03:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.698+0000 7fcd52ffd640 1 -- 172.21.15.17:0/3931562403 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fcd58009d00 con 0x7fcd5c101ce0 2024-02-14T15:03:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.698+0000 7fcd52ffd640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd480a21d0 0x7fcd480a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.698+0000 7fcd612a7640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd480a21d0 0x7fcd480a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.698+0000 7fcd52ffd640 1 -- 172.21.15.17:0/3931562403 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fcd580cdc80 con 0x7fcd5c101ce0 2024-02-14T15:03:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.699+0000 7fcd612a7640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd480a21d0 0x7fcd480a4690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c1987c0 tx=0x7fcd4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.703+0000 7fcd52ffd640 1 -- 172.21.15.17:0/3931562403 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd580967a0 con 0x7fcd5c101ce0 2024-02-14T15:03:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.886+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd5c02cdb0 con 0x7fcd5c101ce0 2024-02-14T15:03:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.888+0000 7fcd52ffd640 1 -- 172.21.15.17:0/3931562403 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fcd58099e60 con 0x7fcd5c101ce0 2024-02-14T15:03:16.889 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:16.889 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":93,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:02:45.606961+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd480a21d0 msgr2=0x7fcd480a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd480a21d0 0x7fcd480a4690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c1987c0 tx=0x7fcd4c005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c101ce0 msgr2=0x7fcd5c1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c101ce0 0x7fcd5c1972b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800d950 tx=0x7fcd5800de10 comp rx=0 tx=0).stop 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 shutdown_connections 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcd480a21d0 0x7fcd480a4690 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd5c106f40 0x7fcd5c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd5c104610 0x7fcd5c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 --2- 172.21.15.17:0/3931562403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd5c101ce0 0x7fcd5c1972b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 >> 172.21.15.17:0/3931562403 conn(0x7fcd5c0fb6f0 msgr2=0x7fcd5c102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 shutdown_connections 2024-02-14T15:03:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:16.890+0000 7fcd63d0c640 1 -- 172.21.15.17:0/3931562403 wait complete. 2024-02-14T15:03:16.901 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:16 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3109139893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:16.901 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:16 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2952840039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:16.901 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:16 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/471781738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:16.902 DEBUG:tasks.fwd_scrub:Yielding 2024-02-14T15:03:16.903 INFO:teuthology.run_tasks:Running task exec... 2024-02-14T15:03:16.915 INFO:teuthology.task.exec:Executing custom commands... 2024-02-14T15:03:16.915 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:03:16.915 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-02-14T15:03:16.916 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-14T15:03:16.916 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-14T15:03:16.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:16.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 -- 172.21.15.17:0/1494102455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc101280 msgr2=0x7f92dc1016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 --2- 172.21.15.17:0/1494102455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc101280 0x7f92dc1016a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009ca0 tx=0x7f92d402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 -- 172.21.15.17:0/1494102455 shutdown_connections 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 --2- 172.21.15.17:0/1494102455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92dc102e30 0x7f92dc1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 --2- 172.21.15.17:0/1494102455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92dc102470 0x7f92dc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 --2- 172.21.15.17:0/1494102455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc101280 0x7f92dc1016a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 -- 172.21.15.17:0/1494102455 >> 172.21.15.17:0/1494102455 conn(0x7f92dc0fc7f0 msgr2=0x7f92dc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 -- 172.21.15.17:0/1494102455 shutdown_connections 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.018+0000 7f92e3091640 1 -- 172.21.15.17:0/1494102455 wait complete. 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 Processor -- start 2024-02-14T15:03:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 -- start start 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92dc101280 0x7f92dc197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92dc102470 0x7f92dc197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc102e30 0x7f92dc19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc19ee00 con 0x7f92dc102e30 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc19ef70 con 0x7f92dc101280 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e3091640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc19f130 con 0x7f92dc102470 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc102e30 0x7f92dc19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc102e30 0x7f92dc19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59220/0 (socket says 172.21.15.17:59220) 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 -- 172.21.15.17:0/2891827071 learned_addr learned my addr 172.21.15.17:0/2891827071 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e0e2d640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92dc101280 0x7f92dc197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92dbfff640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92dc102470 0x7f92dc197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 -- 172.21.15.17:0/2891827071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92dc102470 msgr2=0x7f92dc197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92dc102470 0x7f92dc197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 -- 172.21.15.17:0/2891827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92dc101280 msgr2=0x7f92dc197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92dc101280 0x7f92dc197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.019+0000 7f92e162e640 1 -- 172.21.15.17:0/2891827071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4009900 con 0x7f92dc102e30 2024-02-14T15:03:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.020+0000 7f92e162e640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc102e30 0x7f92dc19e8c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00d750 tx=0x7f92cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.020+0000 7f92d9ffb640 1 -- 172.21.15.17:0/2891827071 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc009bd0 con 0x7f92dc102e30 2024-02-14T15:03:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.020+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc19f410 con 0x7f92dc102e30 2024-02-14T15:03:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.020+0000 7f92d9ffb640 1 -- 172.21.15.17:0/2891827071 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92cc00de60 con 0x7f92dc102e30 2024-02-14T15:03:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.020+0000 7f92d9ffb640 1 -- 172.21.15.17:0/2891827071 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004e70 con 0x7f92dc102e30 2024-02-14T15:03:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.020+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc19f920 con 0x7f92dc102e30 2024-02-14T15:03:17.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.021+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc108640 con 0x7f92dc102e30 2024-02-14T15:03:17.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7f92d9ffb640 1 -- 172.21.15.17:0/2891827071 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f92cc005020 con 0x7f92dc102e30 2024-02-14T15:03:17.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 -- 172.21.15.17:0/3581247285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34100150 msgr2=0x7fbd34100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/3581247285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34100150 0x7fbd34100570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009860 tx=0x7fbd2802bb70 comp rx=0 tx=0).stop 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 -- 172.21.15.17:0/3581247285 shutdown_connections 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/3581247285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34101340 0x7fbd341017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/3581247285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd34101d00 0x7fbd34108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/3581247285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34100150 0x7fbd34100570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 -- 172.21.15.17:0/3581247285 >> 172.21.15.17:0/3581247285 conn(0x7fbd340fb6c0 msgr2=0x7fbd340fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 -- 172.21.15.17:0/3581247285 shutdown_connections 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.022+0000 7fbd3ae50640 1 -- 172.21.15.17:0/3581247285 wait complete. 2024-02-14T15:03:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7f92d9ffb640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f92c80a2140 0x7f92c80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 Processor -- start 2024-02-14T15:03:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7f92e0e2d640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f92c80a2140 0x7f92c80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 -- start start 2024-02-14T15:03:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7f92d9ffb640 1 -- 172.21.15.17:0/2891827071 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f92cc0cda60 con 0x7f92dc102e30 2024-02-14T15:03:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34100150 0x7fbd34196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd34101340 0x7fbd34196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34101d00 0x7fbd3419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3419dd10 con 0x7fbd34100150 2024-02-14T15:03:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3419de80 con 0x7fbd34101d00 2024-02-14T15:03:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd3ae50640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3419e040 con 0x7fbd34101340 2024-02-14T15:03:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd38bec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34100150 0x7fbd34196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd38bec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34100150 0x7fbd34196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59252/0 (socket says 172.21.15.17:59252) 2024-02-14T15:03:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.023+0000 7fbd38bec640 1 -- 172.21.15.17:0/1187188735 learned_addr learned my addr 172.21.15.17:0/1187188735 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7f92e0e2d640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f92c80a2140 0x7f92c80a4600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92d402c700 tx=0x7f92d40326b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd33fff640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd34101340 0x7fbd34196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd393ed640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34101d00 0x7fbd3419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd38bec640 1 -- 172.21.15.17:0/1187188735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd34101340 msgr2=0x7fbd34196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd38bec640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd34101340 0x7fbd34196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd38bec640 1 -- 172.21.15.17:0/1187188735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34101d00 msgr2=0x7fbd3419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd38bec640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34101d00 0x7fbd3419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd38bec640 1 -- 172.21.15.17:0/1187188735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20009560 con 0x7fbd34100150 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.024+0000 7fbd38bec640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34100150 0x7fbd34196220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbd2802c0b0 tx=0x7fbd280045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.027+0000 7fbd31ffb640 1 -- 172.21.15.17:0/1187188735 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2802c510 con 0x7fbd34100150 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.027+0000 7fbd31ffb640 1 -- 172.21.15.17:0/1187188735 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd2802cb50 con 0x7fbd34100150 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.027+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd28009490 con 0x7fbd34100150 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.027+0000 7fbd31ffb640 1 -- 172.21.15.17:0/1187188735 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28032600 con 0x7fbd34100150 2024-02-14T15:03:17.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.027+0000 7f92d9ffb640 1 -- 172.21.15.17:0/2891827071 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92cc09a160 con 0x7f92dc102e30 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.028+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3419e4f0 con 0x7fbd34100150 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 Processor -- start 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 -- start start 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734eeb00 0x55c3734f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734f3c70 0x55c3734f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd31ffb640 1 -- 172.21.15.17:0/1187188735 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fbd2803b040 con 0x7fbd34100150 2024-02-14T15:03:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3734aabd0 con 0x55c3734ee110 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c37345d030 con 0x55c3734eeb00 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.030+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c37345a560 con 0x55c3734f3c70 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734ee530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59260/0 (socket says 172.21.15.17:59260) 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 -- 172.21.15.17:0/3962826435 learned_addr learned my addr 172.21.15.17:0/3962826435 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd33fff640 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734f3c70 0x55c3734f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 -- 172.21.15.17:0/3962826435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734f3c70 msgr2=0x55c3734f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734f3c70 0x55c3734f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd38bec640 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734eeb00 0x55c3734f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 -- 172.21.15.17:0/3962826435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734eeb00 msgr2=0x55c3734f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734eeb00 0x55c3734f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 -- 172.21.15.17:0/3962826435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c373418290 con 0x55c3734ee110 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd38bec640 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734eeb00 0x55c3734f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd31ffb640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbd240a21d0 0x7fbd240a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd393ed640 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734ee530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009fc0 tx=0x7fbd2c0109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445c2cce6d93aa43 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd38bec640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbd240a21d0 0x7fbd240a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd1effd640 1 -- 172.21.15.17:0/3962826435 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c015070 con 0x55c3734ee110 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd31ffb640 1 -- 172.21.15.17:0/1187188735 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fbd280efd40 con 0x7fbd34100150 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd1effd640 1 -- 172.21.15.17:0/3962826435 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c003e50 con 0x55c3734ee110 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.031+0000 7fbd1effd640 1 -- 172.21.15.17:0/3962826435 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c019690 con 0x55c3734ee110 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3962826435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 msgr2=0x55c3734ee530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd38bec640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbd240a21d0 0x7fbd240a4690 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55c3734ef570 tx=0x7fbd280a7040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734ee530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009fc0 tx=0x7fbd2c0109f0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3962826435 shutdown_connections 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734f3c70 0x55c3734f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734eeb00 0x55c3734f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3962826435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734ee530 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3962826435 >> 172.21.15.17:0/3962826435 conn(0x55c3734e9680 msgr2=0x55c3734e9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3962826435 shutdown_connections 2024-02-14T15:03:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3962826435 wait complete. 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.032+0000 7fbd3da1b740 1 -- start start 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 1 Processor -- start 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c3734ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c373455d40 con 0x55c3734f3c70 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3733fae60 con 0x55c3734ee110 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3731294b0 con 0x55c3734eeb00 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd3da1b740 10 client.0 _ll_register_callbacks cb 0x55c3734118d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c3734ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c3734ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59272/0 (socket says 172.21.15.17:59272) 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 -- 172.21.15.17:0/273496261 learned_addr learned my addr 172.21.15.17:0/273496261 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd33fff640 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 -- 172.21.15.17:0/273496261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 msgr2=0x55c3734f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 -- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 msgr2=0x55c3734f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.033+0000 7fbd393ed640 1 -- 172.21.15.17:0/273496261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c009c70 con 0x55c3734f3c70 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd393ed640 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c3734ff8a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c019730 tx=0x7fbd2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ded2bbb43f05985 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd13fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd13fff640 1 -- 172.21.15.17:0/273496261 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c015070 con 0x55c3734f3c70 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd13fff640 1 -- 172.21.15.17:0/273496261 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fbd2c019ba0 con 0x55c3734f3c70 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c37347bdd0 con 0x55c3734f3c70 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd13fff640 1 -- 172.21.15.17:0/273496261 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd2c00c470 con 0x55c3734f3c70 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 10 client.15639 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c3734543c0 con 0x55c3734f3c70 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd13fff640 1 -- 172.21.15.17:0/273496261 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c37347bdd0 con 0x55c3734f3c70 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd13fff640 1 -- 172.21.15.17:0/273496261 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd2c020070 con 0x55c3734f3c70 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 10 client.15639 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 20 client.15639 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c373504020 0x55c373506440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 4 client.15639 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.034+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c373428cb0 con 0x55c373504020 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.035+0000 7fbd33fff640 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c373504020 0x55c373506440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.035+0000 7fbd13fff640 10 client.15639 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.035+0000 7fbd33fff640 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c373504020 0x55c373506440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556734457f960aca server_cookie=eaf0952b05e3e28e in_seq=0 out_seq=0 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd13fff640 1 -- 172.21.15.17:0/273496261 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c373428cb0 con 0x55c373504020 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd13fff640 10 client.15639 handle_command_reply: tid=0 2024-02-14T15:03:17.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 client.15639 shutdown 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 msgr2=0x55c3734ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c3734ff8a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c019730 tx=0x7fbd2c009bd0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 shutdown_connections 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c373504020 0x55c373506440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c3734ff8a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd13fff640 0 client.15639 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/273496261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 >> 172.21.15.17:0/273496261 conn(0x55c3734e9680 msgr2=0x55c3734e8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 shutdown_connections 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 1 -- 172.21.15.17:0/273496261 wait complete. 2024-02-14T15:03:17.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.036+0000 7fbd3da1b740 20 client.15639 trim_cache size 0 max 16384 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 Processor -- start 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 -- start start 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734f3c70 0x55c3734ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3733db670 con 0x55c3734ee110 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3734ea170 con 0x55c3734f3c70 2024-02-14T15:03:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.051+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3734f67c0 con 0x55c3734eeb00 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59284/0 (socket says 172.21.15.17:59284) 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 -- 172.21.15.17:0/3460871774 learned_addr learned my addr 172.21.15.17:0/3460871774 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd33fff640 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 -- 172.21.15.17:0/3460871774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 msgr2=0x55c3734f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 -- 172.21.15.17:0/3460871774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734f3c70 msgr2=0x55c3734ff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734f3c70 0x55c3734ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 -- 172.21.15.17:0/3460871774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3733f0c70 con 0x55c3734ee110 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd33fff640 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd393ed640 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734f8320 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c019760 tx=0x7fbd2c0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a82e2c983e3fdc6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd1e7fc640 1 -- 172.21.15.17:0/3460871774 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c015040 con 0x55c3734ee110 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd1e7fc640 1 -- 172.21.15.17:0/3460871774 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c009500 con 0x55c3734ee110 2024-02-14T15:03:17.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.052+0000 7fbd1e7fc640 1 -- 172.21.15.17:0/3460871774 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c004b40 con 0x55c3734ee110 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3460871774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 msgr2=0x55c3734f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734f8320 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c019760 tx=0x7fbd2c0039b0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3460871774 shutdown_connections 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734f3c70 0x55c3734ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/3460871774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734ee110 0x55c3734f8320 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3460871774 >> 172.21.15.17:0/3460871774 conn(0x55c3734e9680 msgr2=0x55c3734e9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3460871774 shutdown_connections 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 -- 172.21.15.17:0/3460871774 wait complete. 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.053+0000 7fbd3da1b740 1 -- start start 2024-02-14T15:03:17.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 1 Processor -- start 2024-02-14T15:03:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c37355a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c37355ab80 con 0x55c3734f3c70 2024-02-14T15:03:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c37355acc0 con 0x55c3734ee110 2024-02-14T15:03:17.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c37355ae00 con 0x55c3734eeb00 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd3da1b740 10 client.0 _ll_register_callbacks cb 0x7fbd2c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c37355a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c37355a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59298/0 (socket says 172.21.15.17:59298) 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 -- 172.21.15.17:0/1721642697 learned_addr learned my addr 172.21.15.17:0/1721642697 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd33fff640 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 -- 172.21.15.17:0/1721642697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 msgr2=0x55c3734feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 -- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 msgr2=0x55c3734fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.054+0000 7fbd393ed640 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd393ed640 1 -- 172.21.15.17:0/1721642697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3734543c0 con 0x55c3734f3c70 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd393ed640 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c37355a640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c002230 tx=0x7fbd2c019430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e781916238dd2da server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd13fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd13fff640 1 -- 172.21.15.17:0/1721642697 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c015070 con 0x55c3734f3c70 2024-02-14T15:03:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd13fff640 1 -- 172.21.15.17:0/1721642697 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fbd2c0047e0 con 0x55c3734f3c70 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd2c003e50 con 0x55c3734f3c70 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd13fff640 1 -- 172.21.15.17:0/1721642697 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd2c003cd0 con 0x55c3734f3c70 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 10 client.15645 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c373416af0 con 0x55c3734f3c70 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd13fff640 1 -- 172.21.15.17:0/1721642697 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c04b080 con 0x55c3734f3c70 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd13fff640 1 -- 172.21.15.17:0/1721642697 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd2c03d040 con 0x55c3734f3c70 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 10 client.15645 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 20 client.15645 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c37355c470 0x55c37355e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 4 client.15645 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.055+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c373155860 con 0x55c37355c470 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.056+0000 7fbd33fff640 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c37355c470 0x55c37355e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.056+0000 7fbd33fff640 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c37355c470 0x55c37355e890 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f728812b2f2026 server_cookie=115c1d38a794af4a in_seq=0 out_seq=0 2024-02-14T15:03:17.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.056+0000 7fbd13fff640 10 client.15645 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.057+0000 7fbd13fff640 1 -- 172.21.15.17:0/1721642697 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c373155860 con 0x55c37355c470 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.057+0000 7fbd13fff640 10 client.15645 handle_command_reply: tid=0 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.057+0000 7fbd3da1b740 1 client.15645 shutdown 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.057+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 msgr2=0x55c37355a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.057+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c37355a640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c002230 tx=0x7fbd2c019430 comp rx=0 tx=0).stop 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 shutdown_connections 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c37355c470 0x55c37355e890 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd13fff640 0 client.15645 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3734f3c70 0x55c37355a640 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3734eeb00 0x55c3734feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 --2- 172.21.15.17:0/1721642697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3734ee110 0x55c3734fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 >> 172.21.15.17:0/1721642697 conn(0x55c3734e9680 msgr2=0x55c3734f0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 shutdown_connections 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 1 -- 172.21.15.17:0/1721642697 wait complete. 2024-02-14T15:03:17.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3da1b740 20 client.15645 trim_cache size 0 max 16384 2024-02-14T15:03:17.064 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2024-02-14T15:03:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_tag": "0777424c-b4c4-4dd2-80e5-99c3b054d92f", 2024-02-14T15:03:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": "asynchronous" 2024-02-14T15:03:17.064 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbd240a21d0 msgr2=0x7fbd240a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.058+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbd240a21d0 0x7fbd240a4690 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55c3734ef570 tx=0x7fbd280a7040 comp rx=0 tx=0).stop 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34100150 msgr2=0x7fbd34196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34100150 0x7fbd34196220 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbd2802c0b0 tx=0x7fbd280045a0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 shutdown_connections 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fbd240a21d0 0x7fbd240a4690 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd34101d00 0x7fbd3419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd34101340 0x7fbd34196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 --2- 172.21.15.17:0/1187188735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd34100150 0x7fbd34196220 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 >> 172.21.15.17:0/1187188735 conn(0x7fbd340fb6c0 msgr2=0x7fbd341046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 shutdown_connections 2024-02-14T15:03:17.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.059+0000 7fbd3ae50640 1 -- 172.21.15.17:0/1187188735 wait complete. 2024-02-14T15:03:17.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:03:17.071 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:03:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:16 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3109139893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:17.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:16 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2952840039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:17.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:16 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/471781738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 -- 172.21.15.17:0/538828558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c101d00 msgr2=0x7f835c108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 --2- 172.21.15.17:0/538828558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c101d00 0x7f835c108590 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8354009ca0 tx=0x7f835402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 -- 172.21.15.17:0/538828558 shutdown_connections 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 --2- 172.21.15.17:0/538828558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c101d00 0x7f835c108590 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 --2- 172.21.15.17:0/538828558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f835c101340 0x7f835c1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 --2- 172.21.15.17:0/538828558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f835c100150 0x7f835c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 -- 172.21.15.17:0/538828558 >> 172.21.15.17:0/538828558 conn(0x7f835c0fb6c0 msgr2=0x7f835c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 -- 172.21.15.17:0/538828558 shutdown_connections 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.175+0000 7f8360e39640 1 -- 172.21.15.17:0/538828558 wait complete. 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 Processor -- start 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 -- start start 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c100150 0x7f835c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f835c101340 0x7f835c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f835c101d00 0x7f835c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1a0fa0 con 0x7f835c100150 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1a1110 con 0x7f835c101d00 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8360e39640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1a12d0 con 0x7f835c101340 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c100150 0x7f835c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c100150 0x7f835c1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59322/0 (socket says 172.21.15.17:59322) 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 -- 172.21.15.17:0/1282049118 learned_addr learned my addr 172.21.15.17:0/1282049118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f835c101340 0x7f835c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 -- 172.21.15.17:0/1282049118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f835c101340 msgr2=0x7f835c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f835c101340 0x7f835c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 -- 172.21.15.17:0/1282049118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f835c101d00 msgr2=0x7f835c1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f835c101d00 0x7f835c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.176+0000 7f835a59c640 1 -- 172.21.15.17:0/1282049118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8354009900 con 0x7f835c100150 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.177+0000 7f835a59c640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c100150 0x7f835c1994b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f834c00e9c0 tx=0x7f834c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.177+0000 7f834b7fe640 1 -- 172.21.15.17:0/1282049118 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0096e0 con 0x7f835c100150 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.177+0000 7f834b7fe640 1 -- 172.21.15.17:0/1282049118 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f834c00cd80 con 0x7f835c100150 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.177+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c1a15b0 con 0x7f835c100150 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.177+0000 7f834b7fe640 1 -- 172.21.15.17:0/1282049118 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c010430 con 0x7f835c100150 2024-02-14T15:03:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.177+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c1a1a70 con 0x7f835c100150 2024-02-14T15:03:17.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.179+0000 7f834b7fe640 1 -- 172.21.15.17:0/1282049118 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f834c003680 con 0x7f835c100150 2024-02-14T15:03:17.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.179+0000 7f8363a07740 1 Processor -- start 2024-02-14T15:03:17.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8363a07740 1 -- start start 2024-02-14T15:03:17.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8363a07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 0x564a66cb6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8363a07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb7040 0x564a66cbbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f834b7fe640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f83440a2150 0x7f83440a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8363a07740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cbe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8363a07740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66c725b0 con 0x564a66cbc1b0 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8363a07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66c262b0 con 0x564a66cb6650 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8363a07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66c237e0 con 0x564a66cb7040 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cbe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cbe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59336/0 (socket says 172.21.15.17:59336) 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 -- 172.21.15.17:0/4225976954 learned_addr learned my addr 172.21.15.17:0/4225976954 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f834b7fe640 1 -- 172.21.15.17:0/1282049118 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f834c014070 con 0x7f835c100150 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835a59c640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f83440a2150 0x7f83440a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 0x564a66cb6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f8359d9b640 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb7040 0x564a66cbbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 -- 172.21.15.17:0/4225976954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb7040 msgr2=0x564a66cbbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb7040 0x564a66cbbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 -- 172.21.15.17:0/4225976954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 msgr2=0x564a66cb6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 0x564a66cb6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.180+0000 7f835ad9d640 1 -- 172.21.15.17:0/4225976954 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a66be1020 con 0x564a66cbc1b0 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.181+0000 7f835a59c640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f83440a2150 0x7f83440a4610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f834c09ae80 tx=0x7f834c00efb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.181+0000 7f835ad9d640 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cbe640 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x564a66cb78a0 tx=0x7f8354005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204b2aa4cc8d2579 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.181+0000 7f8348ff9640 1 -- 172.21.15.17:0/4225976954 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354046030 con 0x564a66cbc1b0 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.181+0000 7f8348ff9640 1 -- 172.21.15.17:0/4225976954 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8354034640 con 0x564a66cbc1b0 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.181+0000 7f8348ff9640 1 -- 172.21.15.17:0/4225976954 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354003bb0 con 0x564a66cbc1b0 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.181+0000 7f8363a07740 1 -- 172.21.15.17:0/4225976954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 msgr2=0x564a66cbe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.181+0000 7f8363a07740 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cbe640 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x564a66cb78a0 tx=0x7f8354005da0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- 172.21.15.17:0/4225976954 shutdown_connections 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cbe640 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb7040 0x564a66cbbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 --2- 172.21.15.17:0/4225976954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 0x564a66cb6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- 172.21.15.17:0/4225976954 >> 172.21.15.17:0/4225976954 conn(0x564a66cb1bc0 msgr2=0x564a66cb1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- 172.21.15.17:0/4225976954 shutdown_connections 2024-02-14T15:03:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- 172.21.15.17:0/4225976954 wait complete. 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- start start 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 Processor -- start 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66c1efc0 con 0x564a66cbc1b0 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a668f2430 con 0x564a66cb7040 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a668aee00 con 0x564a66cb6650 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f8363a07740 10 client.0 _ll_register_callbacks cb 0x564a66ba0870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f835ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f835ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59340/0 (socket says 172.21.15.17:59340) 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.182+0000 7f835ad9d640 1 -- 172.21.15.17:0/1394015704 learned_addr learned my addr 172.21.15.17:0/1394015704 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f835ad9d640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f835ad9d640 1 -- 172.21.15.17:0/1394015704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 msgr2=0x564a66cc0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f835ad9d640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f835ad9d640 1 -- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 msgr2=0x564a66cc0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f835ad9d640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f835ad9d640 1 -- 172.21.15.17:0/1394015704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c108e90 con 0x564a66cbc1b0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f835ad9d640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc7d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8354005fb0 tx=0x7f8354005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfd6c55d9b2b9de server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8341ffb640 1 -- 172.21.15.17:0/1394015704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354046070 con 0x564a66cbc1b0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a6691e7e0 con 0x564a66cbc1b0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8341ffb640 1 -- 172.21.15.17:0/1394015704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f8354047070 con 0x564a66cbc1b0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8341ffb640 1 -- 172.21.15.17:0/1394015704 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8354034440 con 0x564a66cbc1b0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8363a07740 10 client.15657 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564a66c1d640 con 0x564a66cbc1b0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8341ffb640 1 -- 172.21.15.17:0/1394015704 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564a6691e7e0 con 0x564a66cbc1b0 2024-02-14T15:03:17.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.183+0000 7f8341ffb640 1 -- 172.21.15.17:0/1394015704 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f835403e5e0 con 0x564a66cbc1b0 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8363a07740 10 client.15657 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8363a07740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8363a07740 20 client.15657 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8363a07740 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66ccc6b0 0x564a66ccead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8363a07740 4 client.15657 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a6695b720 con 0x564a66ccc6b0 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66ccc6b0 0x564a66ccead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66ccc6b0 0x564a66ccead0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8ffec3c7006c2e server_cookie=2ed3439da4abd75b in_seq=0 out_seq=0 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.184+0000 7f8341ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8341ffb640 1 -- 172.21.15.17:0/1394015704 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a6695b720 con 0x564a66ccc6b0 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8341ffb640 10 client.15657 handle_command_reply: tid=0 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 client.15657 shutdown 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 msgr2=0x564a66cc7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc7d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8354005fb0 tx=0x7f8354005c30 comp rx=0 tx=0).stop 2024-02-14T15:03:17.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 shutdown_connections 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66ccc6b0 0x564a66ccead0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc7d90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8341ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 --2- 172.21.15.17:0/1394015704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 >> 172.21.15.17:0/1394015704 conn(0x564a66cb1bc0 msgr2=0x564a66cb0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 shutdown_connections 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 1 -- 172.21.15.17:0/1394015704 wait complete. 2024-02-14T15:03:17.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.185+0000 7f8363a07740 20 client.15657 trim_cache size 0 max 16384 2024-02-14T15:03:17.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 Processor -- start 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 -- start start 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 0x564a66cc0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cbc1b0 0x564a66cc7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66ba3fd0 con 0x564a66cb7040 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66cb6210 con 0x564a66cb6650 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8363a07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66cc1880 con 0x564a66cbc1b0 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8359d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8359d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59352/0 (socket says 172.21.15.17:59352) 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.200+0000 7f8359d9b640 1 -- 172.21.15.17:0/1357691007 learned_addr learned my addr 172.21.15.17:0/1357691007 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f835ad9d640 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cbc1b0 0x564a66cc7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8359d9b640 1 -- 172.21.15.17:0/1357691007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cbc1b0 msgr2=0x564a66cc7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cbc1b0 0x564a66cc7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8359d9b640 1 -- 172.21.15.17:0/1357691007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 msgr2=0x564a66cc0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 0x564a66cc0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8359d9b640 1 -- 172.21.15.17:0/1357691007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a66bb9b30 con 0x564a66cb7040 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f835ad9d640 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cbc1b0 0x564a66cc7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8359d9b640 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x564a66cb7ab0 tx=0x7f8350014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b29e8e40c82e6f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8343fff640 1 -- 172.21.15.17:0/1357691007 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350018040 con 0x564a66cb7040 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8343fff640 1 -- 172.21.15.17:0/1357691007 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8350021020 con 0x564a66cb7040 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8343fff640 1 -- 172.21.15.17:0/1357691007 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350012d50 con 0x564a66cb7040 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8363a07740 1 -- 172.21.15.17:0/1357691007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cb7040 msgr2=0x564a66cc0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8363a07740 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x564a66cb7ab0 tx=0x7f8350014840 comp rx=0 tx=0).stop 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8363a07740 1 -- 172.21.15.17:0/1357691007 shutdown_connections 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8363a07740 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cb7040 0x564a66cc0d50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8363a07740 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cbc1b0 0x564a66cc7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8363a07740 1 --2- 172.21.15.17:0/1357691007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb6650 0x564a66cc0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.201+0000 7f8363a07740 1 -- 172.21.15.17:0/1357691007 >> 172.21.15.17:0/1357691007 conn(0x564a66cb1bc0 msgr2=0x564a66cb85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 -- 172.21.15.17:0/1357691007 shutdown_connections 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 -- 172.21.15.17:0/1357691007 wait complete. 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 -- start start 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 Processor -- start 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66cc7140 con 0x564a66cbc1b0 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66cc72b0 con 0x564a66cb7040 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a66d22bd0 con 0x564a66cb6650 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8363a07740 10 client.0 _ll_register_callbacks cb 0x7f8354037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59356/0 (socket says 172.21.15.17:59356) 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 -- 172.21.15.17:0/2067307197 learned_addr learned my addr 172.21.15.17:0/2067307197 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f8359d9b640 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 -- 172.21.15.17:0/2067307197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 msgr2=0x564a66cc7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 -- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 msgr2=0x564a66cc6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.202+0000 7f835ad9d640 1 -- 172.21.15.17:0/2067307197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8350009530 con 0x564a66cbc1b0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f835ad9d640 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc6a40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8354009c70 tx=0x7f8354032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8798599f27e89b97 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8341ffb640 1 -- 172.21.15.17:0/2067307197 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354046070 con 0x564a66cbc1b0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8341ffb640 1 -- 172.21.15.17:0/2067307197 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f8354034440 con 0x564a66cbc1b0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8354034640 con 0x564a66cbc1b0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8341ffb640 1 -- 172.21.15.17:0/2067307197 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8354004d50 con 0x564a66cbc1b0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 10 client.15663 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564a66bdf880 con 0x564a66cbc1b0 2024-02-14T15:03:17.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8341ffb640 1 -- 172.21.15.17:0/2067307197 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8354064080 con 0x564a66cbc1b0 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8341ffb640 1 -- 172.21.15.17:0/2067307197 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f83540619d0 con 0x564a66cbc1b0 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 10 client.15663 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 20 client.15663 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66d24260 0x564a66d26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 4 client.15663 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.203+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a669aa890 con 0x564a66d24260 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8359d9b640 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66d24260 0x564a66d26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8341ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8359d9b640 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66d24260 0x564a66d26680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42bfec53819cda8 server_cookie=cc31f35115f62a5b in_seq=0 out_seq=0 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8341ffb640 1 -- 172.21.15.17:0/2067307197 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564a669aa890 con 0x564a66d24260 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8341ffb640 10 client.15663 handle_command_reply: tid=0 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8363a07740 1 client.15663 shutdown 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 msgr2=0x564a66cc6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.204+0000 7f8363a07740 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc6a40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8354009c70 tx=0x7f8354032440 comp rx=0 tx=0).stop 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 shutdown_connections 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564a66d24260 0x564a66d26680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564a66cbc1b0 0x564a66cc6a40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a66cb7040 0x564a66cc6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8341ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 --2- 172.21.15.17:0/2067307197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564a66cb6650 0x564a66cc7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 >> 172.21.15.17:0/2067307197 conn(0x564a66cb1bc0 msgr2=0x564a66cc07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 shutdown_connections 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 1 -- 172.21.15.17:0/2067307197 wait complete. 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8363a07740 20 client.15663 trim_cache size 0 max 16384 2024-02-14T15:03:17.211 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": { 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: "0777424c-b4c4-4dd2-80e5-99c3b054d92f": { 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/", 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: "tag": "0777424c-b4c4-4dd2-80e5-99c3b054d92f", 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: "options": "recursive,force" 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:03:17.212 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f83440a2150 msgr2=0x7f83440a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f83440a2150 0x7f83440a4610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f834c09ae80 tx=0x7f834c00efb0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c100150 msgr2=0x7f835c1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c100150 0x7f835c1994b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f834c00e9c0 tx=0x7f834c00ee80 comp rx=0 tx=0).stop 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 shutdown_connections 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f83440a2150 0x7f83440a4610 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f835c101d00 0x7f835c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f835c101340 0x7f835c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 --2- 172.21.15.17:0/1282049118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f835c100150 0x7f835c1994b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 >> 172.21.15.17:0/1282049118 conn(0x7f835c0fb6c0 msgr2=0x7f835c0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 shutdown_connections 2024-02-14T15:03:17.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.205+0000 7f8360e39640 1 -- 172.21.15.17:0/1282049118 wait complete. 2024-02-14T15:03:17.217 INFO:tasks.cephfs.filesystem:scrub status for tag:0777424c-b4c4-4dd2-80e5-99c3b054d92f - {'path': '/', 'tag': '0777424c-b4c4-4dd2-80e5-99c3b054d92f', 'options': 'recursive,force'} 2024-02-14T15:03:17.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:16 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3109139893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:03:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:16 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2952840039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:03:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:16 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/471781738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.270+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f92dc10ce40 con 0x7f92dc102e30 2024-02-14T15:03:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.808+0000 7f92d9ffb640 1 -- 172.21.15.17:0/2891827071 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7f92cc0200a0 con 0x7f92dc102e30 2024-02-14T15:03:17.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.811+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f92c80a2140 msgr2=0x7f92c80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.811+0000 7f92e3091640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f92c80a2140 0x7f92c80a4600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92d402c700 tx=0x7f92d40326b0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.811+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc102e30 msgr2=0x7f92dc19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.811+0000 7f92e3091640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc102e30 0x7f92dc19e8c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00d750 tx=0x7f92cc00dc10 comp rx=0 tx=0).stop 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 shutdown_connections 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f92c80a2140 0x7f92c80a4600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92dc102e30 0x7f92dc19e8c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92dc102470 0x7f92dc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 --2- 172.21.15.17:0/2891827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92dc101280 0x7f92dc197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 >> 172.21.15.17:0/2891827071 conn(0x7f92dc0fc7f0 msgr2=0x7f92dc105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 shutdown_connections 2024-02-14T15:03:17.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.813+0000 7f92e3091640 1 -- 172.21.15.17:0/2891827071 wait complete. 2024-02-14T15:03:17.831 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-02-14T15:03:17.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.940+0000 7f7ddbd02640 1 -- 172.21.15.17:0/731659795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd41046b0 msgr2=0x7f7dd4106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.940+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/731659795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd41046b0 0x7f7dd4106aa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009900 tx=0x7f7dc402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:17.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.940+0000 7f7ddbd02640 1 -- 172.21.15.17:0/731659795 shutdown_connections 2024-02-14T15:03:17.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.940+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/731659795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd41046b0 0x7f7dd4106aa0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.940+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/731659795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dd4106fe0 0x7f7dd41093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.940+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/731659795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd4101d80 0x7f7dd4104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.940+0000 7f7ddbd02640 1 -- 172.21.15.17:0/731659795 >> 172.21.15.17:0/731659795 conn(0x7f7dd40fb6d0 msgr2=0x7f7dd40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:17.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 -- 172.21.15.17:0/731659795 shutdown_connections 2024-02-14T15:03:17.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 -- 172.21.15.17:0/731659795 wait complete. 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 Processor -- start 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 -- start start 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd4101d80 0x7f7dd4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dd41046b0 0x7f7dd4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd4106fe0 0x7f7dd419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd419ee00 con 0x7f7dd4106fe0 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd419ef70 con 0x7f7dd4101d80 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7ddbd02640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd419f130 con 0x7f7dd41046b0 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7dda29f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd4106fe0 0x7f7dd419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7dda29f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd4106fe0 0x7f7dd419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59382/0 (socket says 172.21.15.17:59382) 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7dda29f640 1 -- 172.21.15.17:0/677010918 learned_addr learned my addr 172.21.15.17:0/677010918 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7dd9a9e640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd4101d80 0x7f7dd4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.941+0000 7f7dd929d640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dd41046b0 0x7f7dd4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dd9a9e640 1 -- 172.21.15.17:0/677010918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dd41046b0 msgr2=0x7f7dd4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dd9a9e640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dd41046b0 0x7f7dd4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dd9a9e640 1 -- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd4106fe0 msgr2=0x7f7dd419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dd9a9e640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd4106fe0 0x7f7dd419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dd9a9e640 1 -- 172.21.15.17:0/677010918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4009560 con 0x7f7dd4101d80 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dd9a9e640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd4101d80 0x7f7dd4197310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000e9f0 tx=0x7f7dd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dcaffd640 1 -- 172.21.15.17:0/677010918 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000ccc0 con 0x7f7dd4101d80 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd419f410 con 0x7f7dd4101d80 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dcaffd640 1 -- 172.21.15.17:0/677010918 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7dd000ce20 con 0x7f7dd4101d80 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7dcaffd640 1 -- 172.21.15.17:0/677010918 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0018920 con 0x7f7dd4101d80 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.942+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd419f8f0 con 0x7f7dd4101d80 2024-02-14T15:03:17.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.943+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd40fd1c0 con 0x7f7dd4101d80 2024-02-14T15:03:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.944+0000 7f7dcaffd640 1 -- 172.21.15.17:0/677010918 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) v1 ==== 136836+0+0 (secure 0 0 0) 0x7f7dd0018a80 con 0x7f7dd4101d80 2024-02-14T15:03:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.945+0000 7f7dcaffd640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7dc00a2190 0x7f7dc00a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.945+0000 7f7dd929d640 1 -- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7dc00a2190 msgr2=0x7f7dc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:17.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.945+0000 7f7dcaffd640 1 -- 172.21.15.17:0/677010918 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f7dd0014070 con 0x7f7dd4101d80 2024-02-14T15:03:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.945+0000 7f7dd929d640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7dc00a2190 0x7f7dc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T15:03:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:17.948+0000 7f7dcaffd640 1 -- 172.21.15.17:0/677010918 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7dd00970c0 con 0x7f7dd4101d80 2024-02-14T15:03:18.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.132+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7dd410cc90 con 0x7f7dd4101d80 2024-02-14T15:03:18.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.137+0000 7f7dcaffd640 1 -- 172.21.15.17:0/677010918 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7dd009a780 con 0x7f7dd4101d80 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7dc00a2190 msgr2=0x7f7dc00a4650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7dc00a2190 0x7f7dc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd4101d80 msgr2=0x7f7dd4197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd4101d80 0x7f7dd4197310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000e9f0 tx=0x7f7dd000eeb0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 shutdown_connections 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7dc00a2190 0x7f7dc00a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7dd4106fe0 0x7f7dd419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dd41046b0 0x7f7dd4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 --2- 172.21.15.17:0/677010918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd4101d80 0x7f7dd4197310 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 >> 172.21.15.17:0/677010918 conn(0x7f7dd40fb6d0 msgr2=0x7f7dd4102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 shutdown_connections 2024-02-14T15:03:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.139+0000 7f7ddbd02640 1 -- 172.21.15.17:0/677010918 wait complete. 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: ignoring --setuser ceph since I am not root 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: ignoring --setgroup ceph since I am not root 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mgr[37941]: -- 172.21.15.50:0/4180554991 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac9f289520 con 0x55ac9f25f000 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mon[36512]: pgmap v370: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3931562403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mon[36512]: scrub queued for path: / 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mon[36512]: scrub summary: active paths [/] 2024-02-14T15:03:18.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:18.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:17 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2891827071' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-14T15:03:18.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-mon[32549]: pgmap v370: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-02-14T15:03:18.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3931562403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:18.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-mon[32549]: scrub queued for path: / 2024-02-14T15:03:18.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:18.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-mon[32549]: scrub summary: active paths [/] 2024-02-14T15:03:18.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:18.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2891827071' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-14T15:03:18.151 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: ignoring --setuser ceph since I am not root 2024-02-14T15:03:18.151 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: ignoring --setgroup ceph since I am not root 2024-02-14T15:03:18.152 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: ignoring --setuser ceph since I am not root 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: ignoring --setgroup ceph since I am not root 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mgr[38815]: -- 172.21.15.76:0/1084442126 <== mon.1 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55734c5fb520 con 0x55734c5d0c00 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mon[37057]: pgmap v370: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3931562403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mon[37057]: scrub queued for path: / 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mon[37057]: scrub summary: active paths [/] 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:17 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2891827071' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 -- 172.21.15.17:0/63436981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 msgr2=0x7f7fd81017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 --2- 172.21.15.17:0/63436981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd81017d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009900 tx=0x7f7fcc02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 -- 172.21.15.17:0/63436981 shutdown_connections 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 --2- 172.21.15.17:0/63436981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd81017d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 --2- 172.21.15.17:0/63436981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd8101d10 0x7f7fd81085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 --2- 172.21.15.17:0/63436981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd8100160 0x7f7fd8100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 -- 172.21.15.17:0/63436981 >> 172.21.15.17:0/63436981 conn(0x7f7fd80fb6d0 msgr2=0x7f7fd80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 -- 172.21.15.17:0/63436981 shutdown_connections 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.255+0000 7f7fde09f640 1 -- 172.21.15.17:0/63436981 wait complete. 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 Processor -- start 2024-02-14T15:03:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 -- start start 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd8100160 0x7f7fd8199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd8101d10 0x7f7fd81a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81a1000 con 0x7f7fd8101350 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81a1170 con 0x7f7fd8100160 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fde09f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81a1330 con 0x7f7fd8101d10 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fd6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fd6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd8199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59398/0 (socket says 172.21.15.17:59398) 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fd6ffd640 1 -- 172.21.15.17:0/1909350675 learned_addr learned my addr 172.21.15.17:0/1909350675 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fd7fff640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd8101d10 0x7f7fd81a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.256+0000 7f7fd77fe640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd8100160 0x7f7fd8199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd6ffd640 1 -- 172.21.15.17:0/1909350675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd8101d10 msgr2=0x7f7fd81a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd6ffd640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd8101d10 0x7f7fd81a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd6ffd640 1 -- 172.21.15.17:0/1909350675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd8100160 msgr2=0x7f7fd8199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd6ffd640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd8100160 0x7f7fd8199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd6ffd640 1 -- 172.21.15.17:0/1909350675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc009560 con 0x7f7fd8101350 2024-02-14T15:03:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd6ffd640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd8199a50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc02c6a0 tx=0x7f7fcc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd4ff9640 1 -- 172.21.15.17:0/1909350675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0044c0 con 0x7f7fd8101350 2024-02-14T15:03:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd81a15b0 con 0x7f7fd8101350 2024-02-14T15:03:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd4ff9640 1 -- 172.21.15.17:0/1909350675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc0052d0 con 0x7f7fd8101350 2024-02-14T15:03:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fd4ff9640 1 -- 172.21.15.17:0/1909350675 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc003e80 con 0x7f7fd8101350 2024-02-14T15:03:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.257+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd81a1ac0 con 0x7f7fd8101350 2024-02-14T15:03:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.258+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd810bbc0 con 0x7f7fd8101350 2024-02-14T15:03:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.259+0000 7f7fd4ff9640 1 -- 172.21.15.17:0/1909350675 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 20) v1 ==== 136836+0+0 (secure 0 0 0) 0x7f7fcc005440 con 0x7f7fd8101350 2024-02-14T15:03:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.260+0000 7f7fd4ff9640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7fc00a2190 0x7f7fc00a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.260+0000 7f7fd77fe640 1 -- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7fc00a2190 msgr2=0x7f7fc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.260+0000 7f7fd4ff9640 1 -- 172.21.15.17:0/1909350675 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7f7fcc0f0d60 con 0x7f7fd8101350 2024-02-14T15:03:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.260+0000 7f7fd77fe640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7fc00a2190 0x7f7fc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T15:03:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.266+0000 7f7fd4ff9640 1 -- 172.21.15.17:0/1909350675 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7fcc0b9990 con 0x7f7fd8101350 2024-02-14T15:03:18.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.449+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f7fd81a1c50 con 0x7f7fd8101350 2024-02-14T15:03:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.452+0000 7f7fd4ff9640 1 -- 172.21.15.17:0/1909350675 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7fd81a1c50 con 0x7f7fd8101350 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7fc00a2190 msgr2=0x7f7fc00a4650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7fc00a2190 0x7f7fc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 msgr2=0x7f7fd8199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd8199a50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc02c6a0 tx=0x7f7fcc0040c0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 shutdown_connections 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7f7fc00a2190 0x7f7fc00a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fd8101350 0x7f7fd8199a50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd8101d10 0x7f7fd81a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 --2- 172.21.15.17:0/1909350675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fd8100160 0x7f7fd8199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 >> 172.21.15.17:0/1909350675 conn(0x7f7fd80fb6d0 msgr2=0x7f7fd80fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 shutdown_connections 2024-02-14T15:03:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.455+0000 7f7fde09f640 1 -- 172.21.15.17:0/1909350675 wait complete. 2024-02-14T15:03:18.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-02-14T15:03:18.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 -- 172.21.15.17:0/52191250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102e30 msgr2=0x7fcbd41096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 --2- 172.21.15.17:0/52191250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102e30 0x7fcbd41096c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009ca0 tx=0x7fcbcc02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 -- 172.21.15.17:0/52191250 shutdown_connections 2024-02-14T15:03:18.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 --2- 172.21.15.17:0/52191250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102e30 0x7fcbd41096c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 --2- 172.21.15.17:0/52191250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbd4102470 0x7fcbd41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 --2- 172.21.15.17:0/52191250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbd4101280 0x7fcbd41016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 -- 172.21.15.17:0/52191250 >> 172.21.15.17:0/52191250 conn(0x7fcbd40fc7f0 msgr2=0x7fcbd40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 -- 172.21.15.17:0/52191250 shutdown_connections 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.573+0000 7fcbdc28f640 1 -- 172.21.15.17:0/52191250 wait complete. 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 Processor -- start 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 -- start start 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbd4101280 0x7fcbd41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102470 0x7fcbd4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbd4102e30 0x7fcbd419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd419edd0 con 0x7fcbd4102470 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd419ef40 con 0x7fcbd4102e30 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbdc28f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd419f100 con 0x7fcbd4101280 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102470 0x7fcbd4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102470 0x7fcbd4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59402/0 (socket says 172.21.15.17:59402) 2024-02-14T15:03:18.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 -- 172.21.15.17:0/3684992409 learned_addr learned my addr 172.21.15.17:0/3684992409 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbda02b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbd4101280 0x7fcbd41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbda82c640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbd4102e30 0x7fcbd419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbd4101280 msgr2=0x7fcbd41972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbd4101280 0x7fcbd41972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbd4102e30 msgr2=0x7fcbd419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcbd4102e30 0x7fcbd419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 -- 172.21.15.17:0/3684992409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc009900 con 0x7fcbd4102470 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.574+0000 7fcbd982a640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102470 0x7fcbd4197820 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400e9f0 tx=0x7fcbc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.575+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4009700 con 0x7fcbd4102470 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.575+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd419f3e0 con 0x7fcbd4102470 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.575+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbc400cd80 con 0x7fcbd4102470 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.575+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4018920 con 0x7fcbd4102470 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.575+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd419f8f0 con 0x7fcbd4102470 2024-02-14T15:03:18.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.576+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd41085a0 con 0x7fcbd4102470 2024-02-14T15:03:18.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.577+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 20) v1 ==== 136836+0+0 (secure 0 0 0) 0x7fcbc4018a80 con 0x7fcbd4102470 2024-02-14T15:03:18.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.577+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:18.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.577+0000 7fcbda02b640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 msgr2=0x7fcbc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:18.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.578+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9086+0+0 (secure 0 0 0) 0x7fcbc4014070 con 0x7fcbd4102470 2024-02-14T15:03:18.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.578+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T15:03:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.581+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcbc40975d0 con 0x7fcbd4102470 2024-02-14T15:03:18.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:18.378+0000 7f5a37c41200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T15:03:18.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:18.471+0000 7f5a37c41200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T15:03:18.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:18.411+0000 7f9142f49200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T15:03:18.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:18.511+0000 7f9142f49200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T15:03:18.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.779+0000 7fcbda02b640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 msgr2=0x7fcbc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:18.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.779+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T15:03:18.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:18.788+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 --> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbd402cec0 con 0x7fcbc00a2190 2024-02-14T15:03:18.809 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:18.399+0000 7fcf11c5e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T15:03:18.809 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:18.495+0000 7fcf11c5e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T15:03:19.113 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:18 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2891827071' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-14T15:03:19.113 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:18 smithi076 ceph-mon[37057]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-02-14T15:03:19.113 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:18 smithi076 ceph-mon[37057]: from='client.? ' entity='client.admin' 2024-02-14T15:03:19.114 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:18 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1909350675' entity='client.admin' 2024-02-14T15:03:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:18 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2891827071' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-14T15:03:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:18 smithi050 ceph-mon[36512]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-02-14T15:03:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:18 smithi050 ceph-mon[36512]: from='client.? ' entity='client.admin' 2024-02-14T15:03:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:18 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1909350675' entity='client.admin' 2024-02-14T15:03:19.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:19.180+0000 7fcbda02b640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 msgr2=0x7fcbc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:19.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:19.180+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-14T15:03:19.181 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:18 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2891827071' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-14T15:03:19.181 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:18 smithi017 ceph-mon[32549]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-02-14T15:03:19.181 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:18 smithi017 ceph-mon[32549]: from='client.? ' entity='client.admin' 2024-02-14T15:03:19.181 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:18 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1909350675' entity='client.admin' 2024-02-14T15:03:19.443 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.112+0000 7f5a37c41200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T15:03:19.443 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.205+0000 7f5a37c41200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T15:03:19.443 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.347+0000 7f5a37c41200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T15:03:19.501 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:19.159+0000 7fcf11c5e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T15:03:19.501 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:19.256+0000 7fcf11c5e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T15:03:19.501 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:19.402+0000 7fcf11c5e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T15:03:19.501 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:19.500+0000 7fcf11c5e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T15:03:19.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:19.180+0000 7f9142f49200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T15:03:19.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:19.277+0000 7f9142f49200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T15:03:19.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:19.421+0000 7f9142f49200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T15:03:19.733 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.441+0000 7f5a37c41200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T15:03:19.734 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.524+0000 7f5a37c41200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T15:03:19.859 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:19.586+0000 7fcf11c5e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T15:03:19.877 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:19.520+0000 7f9142f49200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T15:03:19.877 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:19.606+0000 7f9142f49200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T15:03:19.878 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:19.877+0000 7f9142f49200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T15:03:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:19.981+0000 7fcbda02b640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 msgr2=0x7fcbc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:19.981+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-14T15:03:20.080 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.789+0000 7f5a37c41200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T15:03:20.080 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.898+0000 7f5a37c41200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T15:03:20.080 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:19.984+0000 7f5a37c41200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T15:03:20.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:19.858+0000 7fcf11c5e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T15:03:20.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:19.971+0000 7fcf11c5e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T15:03:20.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:20.060+0000 7fcf11c5e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T15:03:20.179 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:19.991+0000 7f9142f49200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T15:03:20.179 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:20.080+0000 7f9142f49200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T15:03:20.179 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:20.178+0000 7f9142f49200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T15:03:20.484 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:20 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:20.078+0000 7f5a37c41200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T15:03:20.484 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:20 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:20.254+0000 7f5a37c41200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T15:03:20.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:20.360+0000 7f9142f49200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T15:03:20.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:20.158+0000 7fcf11c5e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T15:03:20.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:20.341+0000 7fcf11c5e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T15:03:21.484 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:21.222+0000 7f5a37c41200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T15:03:21.484 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:21.355+0000 7f5a37c41200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T15:03:21.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:21.582+0000 7fcbda02b640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 msgr2=0x7fcbc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:21.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:21.582+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-14T15:03:21.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:21.361+0000 7fcf11c5e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T15:03:21.642 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:21.498+0000 7fcf11c5e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T15:03:21.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:21.365+0000 7f9142f49200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T15:03:21.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:21.502+0000 7f9142f49200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T15:03:21.976 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:21.689+0000 7f5a37c41200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T15:03:21.977 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:21.780+0000 7f5a37c41200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T15:03:22.136 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:21.842+0000 7fcf11c5e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T15:03:22.137 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:21.935+0000 7fcf11c5e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T15:03:22.144 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:21.844+0000 7f9142f49200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T15:03:22.144 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:21.939+0000 7f9142f49200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T15:03:22.144 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:22 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:22.143+0000 7f9142f49200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T15:03:22.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:21.975+0000 7f5a37c41200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T15:03:22.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:22.162+0000 7f5a37c41200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T15:03:22.392 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:22.135+0000 7fcf11c5e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T15:03:22.392 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:22.325+0000 7fcf11c5e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T15:03:22.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:22 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:22.337+0000 7f9142f49200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T15:03:22.705 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:22.444+0000 7f5a37c41200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T15:03:22.887 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:22.616+0000 7fcf11c5e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T15:03:22.900 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:22 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:22.634+0000 7f9142f49200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T15:03:22.900 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:22 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:22.899+0000 7f9142f49200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T15:03:22.983 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:22.703+0000 7f5a37c41200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T15:03:22.984 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:22.939+0000 7f5a37c41200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T15:03:23.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:22.886+0000 7fcf11c5e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T15:03:23.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:23.129+0000 7fcf11c5e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T15:03:23.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:23 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:23.146+0000 7f9142f49200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T15:03:24.234 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:23.740+0000 7f5a37c41200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T15:03:24.266 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:23 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:23.990+0000 7f9142f49200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T15:03:24.392 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:23.964+0000 7fcf11c5e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T15:03:24.626 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:24.311+0000 7f5a37c41200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T15:03:24.627 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:24.409+0000 7f5a37c41200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T15:03:24.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:24.700+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 21) v1 ==== 136836+0+0 (secure 0 0 0) 0x7fcbc40979a0 con 0x7fcbd4102470 2024-02-14T15:03:24.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:24.786+0000 7fcbda02b640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 msgr2=0x7fcbc00a4650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/3796717329 2024-02-14T15:03:24.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:24.786+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-14T15:03:24.886 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:24.559+0000 7fcf11c5e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T15:03:24.886 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:24.660+0000 7fcf11c5e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T15:03:24.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-mon[36512]: Standby manager daemon z restarted 2024-02-14T15:03:24.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-mon[36512]: Standby manager daemon z started 2024-02-14T15:03:24.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T15:03:24.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T15:03:24.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T15:03:24.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T15:03:24.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-mon[32549]: Standby manager daemon z restarted 2024-02-14T15:03:24.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-mon[32549]: Standby manager daemon z started 2024-02-14T15:03:24.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T15:03:24.918 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T15:03:24.918 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T15:03:24.918 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T15:03:24.918 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:24.580+0000 7f9142f49200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T15:03:24.918 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:24.684+0000 7f9142f49200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T15:03:24.918 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:24 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:24.916+0000 7f9142f49200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T15:03:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:24.941+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mgrmap(e 22) v1 ==== 136453+0+0 (secure 0 0 0) 0x7fcbc409b590 con 0x7fcbd4102470 2024-02-14T15:03:24.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:24.941+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 msgr2=0x7fcbc00a4650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T15:03:24.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:24.941+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/3796717329,v1:172.21.15.17:6801/3796717329] conn(0x7fcbc00a2190 0x7fcbc00a4650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:24.984 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z[38765]: 2024-02-14T15:03:24.625+0000 7f5a37c41200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T15:03:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-mon[37057]: Standby manager daemon z restarted 2024-02-14T15:03:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-mon[37057]: Standby manager daemon z started 2024-02-14T15:03:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-14T15:03:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T15:03:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-14T15:03:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:24 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.76:0/3236013434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T15:03:25.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:03:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y[37937]: 2024-02-14T15:03:24.885+0000 7fcf11c5e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T15:03:25.516 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.174+0000 7f90a493e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.174+0000 7f90a493e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.174+0000 7f90a493e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.174+0000 7f90a493e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.174+0000 7f90a493e700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.175+0000 7f90a1938700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.175+0000 7f90a1938700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.175+0000 7f90a1938700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.175+0000 7f90a1938700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.517 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:03:25.175+0000 7f90a1938700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:03:25.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:25.946+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mon.0 v2:172.21.15.17:3300/0 9 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7fcbc40498a0 con 0x7fcbd4102470 2024-02-14T15:03:25.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:25.947+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcbc0034560 0x7fcbc0034940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:25.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:25.947+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 --> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbd402cec0 con 0x7fcbc0034560 2024-02-14T15:03:25.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:25.949+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcbc0034560 0x7fcbc0034940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:25.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:25.953+0000 7fcbda02b640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcbc0034560 0x7fcbc0034940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcbd41022d0 tx=0x7fcbd0005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: mgrmap e21: x(active, since 9m), standbys: y, z 2024-02-14T15:03:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: Standby manager daemon y restarted 2024-02-14T15:03:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: Standby manager daemon y started 2024-02-14T15:03:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: Active manager daemon x restarted 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: Activating manager daemon x 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: osdmap e94: 12 total, 12 up, 12 in 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: mgrmap e22: x(active, starting, since 0.0125445s), standbys: y, z 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-14T15:03:25.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T15:03:25.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: Manager daemon x is now available 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:25.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-14T15:03:25.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:25 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-14T15:03:26.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.013+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3684992409 <== mgr.15681 v2:172.21.15.17:6800/877978339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcbd402cec0 con 0x7fcbc0034560 2024-02-14T15:03:26.014 INFO:teuthology.orchestra.run.smithi017.stdout:Schedule set for path / 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcbc0034560 msgr2=0x7fcbc0034940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcbc0034560 0x7fcbc0034940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcbd41022d0 tx=0x7fcbd0005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102470 msgr2=0x7fcbd4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 --2- 172.21.15.17:0/3684992409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbd4102470 0x7fcbd4197820 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400e9f0 tx=0x7fcbc400eeb0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbda82c640 1 -- 172.21.15.17:0/3684992409 reap_dead start 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 shutdown_connections 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 >> 172.21.15.17:0/3684992409 conn(0x7fcbd40fc7f0 msgr2=0x7fcbd4105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:26.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 shutdown_connections 2024-02-14T15:03:26.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.015+0000 7fcbdc28f640 1 -- 172.21.15.17:0/3684992409 wait complete. 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: mgrmap e21: x(active, since 9m), standbys: y, z 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: Standby manager daemon y restarted 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: Standby manager daemon y started 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: Active manager daemon x restarted 2024-02-14T15:03:26.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: Activating manager daemon x 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: osdmap e94: 12 total, 12 up, 12 in 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: mgrmap e22: x(active, starting, since 0.0125445s), standbys: y, z 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:03:26.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T15:03:26.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: Manager daemon x is now available 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-14T15:03:26.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:25 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-14T15:03:26.027 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-02-14T15:03:26.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.133+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/4269800049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 msgr2=0x7f0c94100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.133+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/4269800049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94100590 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009900 tx=0x7f0c9002bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:26.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.133+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/4269800049 shutdown_connections 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.133+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/4269800049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c94101d20 0x7f0c941085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.133+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/4269800049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c94101360 0x7f0c941017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.133+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/4269800049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94100590 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.133+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/4269800049 >> 172.21.15.17:0/4269800049 conn(0x7f0c940fb700 msgr2=0x7f0c940fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/4269800049 shutdown_connections 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/4269800049 wait complete. 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 Processor -- start 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 -- start start 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c94101360 0x7f0c94102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c94101d20 0x7f0c94102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c941033e0 con 0x7f0c94100170 2024-02-14T15:03:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c94103550 con 0x7f0c94101360 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9b8c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c941a13d0 con 0x7f0c94101d20 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9965c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9965c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94104100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54484/0 (socket says 172.21.15.17:54484) 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.134+0000 7f0c9965c640 1 -- 172.21.15.17:0/856703887 learned_addr learned my addr 172.21.15.17:0/856703887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c99e5d640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c94101d20 0x7f0c94102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c98e5b640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c94101360 0x7f0c94102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9965c640 1 -- 172.21.15.17:0/856703887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c94101d20 msgr2=0x7f0c94102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9965c640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c94101d20 0x7f0c94102c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9965c640 1 -- 172.21.15.17:0/856703887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c94101360 msgr2=0x7f0c94102740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9965c640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c94101360 0x7f0c94102740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9965c640 1 -- 172.21.15.17:0/856703887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009560 con 0x7f0c94100170 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c99e5d640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c94101d20 0x7f0c94102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9965c640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94104100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0c90000c00 tx=0x7f0c90005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:26.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c8a7fc640 1 -- 172.21.15.17:0/856703887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900043d0 con 0x7f0c94100170 2024-02-14T15:03:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c8a7fc640 1 -- 172.21.15.17:0/856703887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c900052d0 con 0x7f0c94100170 2024-02-14T15:03:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c8a7fc640 1 -- 172.21.15.17:0/856703887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90003d70 con 0x7f0c94100170 2024-02-14T15:03:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c941a1620 con 0x7f0c94100170 2024-02-14T15:03:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.135+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c941a1b30 con 0x7f0c94100170 2024-02-14T15:03:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.136+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9410bb80 con 0x7f0c94100170 2024-02-14T15:03:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.137+0000 7f0c8a7fc640 1 -- 172.21.15.17:0/856703887 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f0c90005440 con 0x7f0c94100170 2024-02-14T15:03:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.138+0000 7f0c8a7fc640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c800a1ea0 0x7f0c800a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.138+0000 7f0c8a7fc640 1 -- 172.21.15.17:0/856703887 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0c900effe0 con 0x7f0c94100170 2024-02-14T15:03:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.138+0000 7f0c98e5b640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c800a1ea0 0x7f0c800a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.138+0000 7f0c98e5b640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c800a1ea0 0x7f0c800a4360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0c94103e70 tx=0x7f0c84005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: mgrmap e21: x(active, since 9m), standbys: y, z 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: Standby manager daemon y restarted 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: Standby manager daemon y started 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.? 172.21.15.50:0/1946023802' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: Active manager daemon x restarted 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: Activating manager daemon x 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: osdmap e94: 12 total, 12 up, 12 in 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: mgrmap e22: x(active, starting, since 0.0125445s), standbys: y, z 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T15:03:26.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T15:03:26.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: Manager daemon x is now available 2024-02-14T15:03:26.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:26.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:26.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-14T15:03:26.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:25 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-14T15:03:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.142+0000 7f0c8a7fc640 1 -- 172.21.15.17:0/856703887 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c900b8e20 con 0x7f0c94100170 2024-02-14T15:03:26.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.354+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 --> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c940fd900 con 0x7f0c800a1ea0 2024-02-14T15:03:26.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.412+0000 7f0c8a7fc640 1 -- 172.21.15.17:0/856703887 <== mgr.15681 v2:172.21.15.17:6800/877978339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f0c940fd900 con 0x7f0c800a1ea0 2024-02-14T15:03:26.413 INFO:teuthology.orchestra.run.smithi017.stdout:Retention added to path / 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c800a1ea0 msgr2=0x7f0c800a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c800a1ea0 0x7f0c800a4360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0c94103e70 tx=0x7f0c84005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 msgr2=0x7f0c94104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94104100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0c90000c00 tx=0x7f0c90005e50 comp rx=0 tx=0).stop 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 shutdown_connections 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c800a1ea0 0x7f0c800a4360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c94101d20 0x7f0c94102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c94101360 0x7f0c94102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 --2- 172.21.15.17:0/856703887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c94100170 0x7f0c94104100 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 >> 172.21.15.17:0/856703887 conn(0x7f0c940fb700 msgr2=0x7f0c940fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:26.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 shutdown_connections 2024-02-14T15:03:26.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.414+0000 7f0c9b8c0640 1 -- 172.21.15.17:0/856703887 wait complete. 2024-02-14T15:03:26.426 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/602372164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102e30 msgr2=0x7f5e981096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/602372164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102e30 0x7f5e981096c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009ca0 tx=0x7f5e9002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/602372164 shutdown_connections 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/602372164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102e30 0x7f5e981096c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/602372164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e98102470 0x7f5e981028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/602372164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98101280 0x7f5e981016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/602372164 >> 172.21.15.17:0/602372164 conn(0x7f5e980fc7f0 msgr2=0x7f5e980fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:26.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/602372164 shutdown_connections 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.530+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/602372164 wait complete. 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 Processor -- start 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 -- start start 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e98101280 0x7f5e98074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102470 0x7f5e98073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98102e30 0x7f5e98073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980742c0 con 0x7f5e98102470 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e98074430 con 0x7f5e98102e30 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e9dd8c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e981a1340 con 0x7f5e98101280 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e96ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102470 0x7f5e98073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e96ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102470 0x7f5e98073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54498/0 (socket says 172.21.15.17:54498) 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e96ffd640 1 -- 172.21.15.17:0/606691589 learned_addr learned my addr 172.21.15.17:0/606691589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e977fe640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e98101280 0x7f5e98074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e97fff640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98102e30 0x7f5e98073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e96ffd640 1 -- 172.21.15.17:0/606691589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e98101280 msgr2=0x7f5e98074fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e96ffd640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e98101280 0x7f5e98074fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e96ffd640 1 -- 172.21.15.17:0/606691589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98102e30 msgr2=0x7f5e98073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.531+0000 7f5e96ffd640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98102e30 0x7f5e98073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.532+0000 7f5e96ffd640 1 -- 172.21.15.17:0/606691589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e90009900 con 0x7f5e98102470 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.532+0000 7f5e96ffd640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102470 0x7f5e98073620 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00ea20 tx=0x7f5e8c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.532+0000 7f5e94ff9640 1 -- 172.21.15.17:0/606691589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c009720 con 0x7f5e98102470 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.532+0000 7f5e94ff9640 1 -- 172.21.15.17:0/606691589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e8c00cd80 con 0x7f5e98102470 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.532+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e981a15f0 con 0x7f5e98102470 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.532+0000 7f5e94ff9640 1 -- 172.21.15.17:0/606691589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c010490 con 0x7f5e98102470 2024-02-14T15:03:26.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.532+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e981a1ab0 con 0x7f5e98102470 2024-02-14T15:03:26.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.533+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e981085a0 con 0x7f5e98102470 2024-02-14T15:03:26.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.534+0000 7f5e94ff9640 1 -- 172.21.15.17:0/606691589 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f5e8c005000 con 0x7f5e98102470 2024-02-14T15:03:26.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.535+0000 7f5e94ff9640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5e800a1ea0 0x7f5e800a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.535+0000 7f5e977fe640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5e800a1ea0 0x7f5e800a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.535+0000 7f5e94ff9640 1 -- 172.21.15.17:0/606691589 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f5e8c014070 con 0x7f5e98102470 2024-02-14T15:03:26.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.535+0000 7f5e977fe640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5e800a1ea0 0x7f5e800a4360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e981022d0 tx=0x7f5e88005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:26.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.538+0000 7f5e94ff9640 1 -- 172.21.15.17:0/606691589 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e8c097160 con 0x7f5e98102470 2024-02-14T15:03:26.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.751+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 --> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e9802cec0 con 0x7f5e800a1ea0 2024-02-14T15:03:26.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.759+0000 7f5e94ff9640 1 -- 172.21.15.17:0/606691589 <== mgr.15681 v2:172.21.15.17:6800/877978339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f5e9802cec0 con 0x7f5e800a1ea0 2024-02-14T15:03:26.760 INFO:teuthology.orchestra.run.smithi017.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-14T00:00:00", "created": "2024-02-14T15:03:25", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5e800a1ea0 msgr2=0x7f5e800a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5e800a1ea0 0x7f5e800a4360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e981022d0 tx=0x7f5e88005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102470 msgr2=0x7f5e98073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102470 0x7f5e98073620 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00ea20 tx=0x7f5e8c00ee20 comp rx=0 tx=0).stop 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 shutdown_connections 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5e800a1ea0 0x7f5e800a4360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e98102470 0x7f5e98073620 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e98102e30 0x7f5e98073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 --2- 172.21.15.17:0/606691589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e98101280 0x7f5e98074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 >> 172.21.15.17:0/606691589 conn(0x7f5e980fc7f0 msgr2=0x7f5e98105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:26.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 shutdown_connections 2024-02-14T15:03:26.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.761+0000 7f5e9dd8c640 1 -- 172.21.15.17:0/606691589 wait complete. 2024-02-14T15:03:26.774 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-14T15:03:26.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 -- 172.21.15.17:0/976472037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 msgr2=0x7f7914109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 --2- 172.21.15.17:0/976472037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f7914109390 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ca0 tx=0x7f790c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 -- 172.21.15.17:0/976472037 shutdown_connections 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 --2- 172.21.15.17:0/976472037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f7914109390 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 --2- 172.21.15.17:0/976472037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7914104610 0x7f7914106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 --2- 172.21.15.17:0/976472037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7914101ce0 0x7f79141040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 -- 172.21.15.17:0/976472037 >> 172.21.15.17:0/976472037 conn(0x7f79140fb6d0 msgr2=0x7f79140fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 -- 172.21.15.17:0/976472037 shutdown_connections 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.887+0000 7f791bb3e640 1 -- 172.21.15.17:0/976472037 wait complete. 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 Processor -- start 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 -- start start 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7914101ce0 0x7f7914101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7914104610 0x7f79140ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f79141001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914100930 con 0x7f7914106f40 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914100aa0 con 0x7f7914104610 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791bb3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79141a1300 con 0x7f7914101ce0 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f79141001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f79141001d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54512/0 (socket says 172.21.15.17:54512) 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 -- 172.21.15.17:0/3335795650 learned_addr learned my addr 172.21.15.17:0/3335795650 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f79190d9640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7914104610 0x7f79140ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f79198da640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7914101ce0 0x7f7914101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 -- 172.21.15.17:0/3335795650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7914101ce0 msgr2=0x7f7914101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7914101ce0 0x7f7914101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 -- 172.21.15.17:0/3335795650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7914104610 msgr2=0x7f79140ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7914104610 0x7f79140ffc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 -- 172.21.15.17:0/3335795650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c009900 con 0x7f7914106f40 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.888+0000 7f791a0db640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f79141001d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f790c033000 tx=0x7f790c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:26.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.889+0000 7f790affd640 1 -- 172.21.15.17:0/3335795650 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c0329b0 con 0x7f7914106f40 2024-02-14T15:03:26.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.889+0000 7f790affd640 1 -- 172.21.15.17:0/3335795650 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f790c003e50 con 0x7f7914106f40 2024-02-14T15:03:26.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.889+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79141a1550 con 0x7f7914106f40 2024-02-14T15:03:26.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.889+0000 7f790affd640 1 -- 172.21.15.17:0/3335795650 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c040450 con 0x7f7914106f40 2024-02-14T15:03:26.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.889+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79141a1a50 con 0x7f7914106f40 2024-02-14T15:03:26.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:26 smithi050 ceph-mon[36512]: [14/Feb/2024:15:03:25] ENGINE Bus STARTING 2024-02-14T15:03:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:26 smithi050 ceph-mon[36512]: [14/Feb/2024:15:03:25] ENGINE Serving on https://172.21.15.17:7150 2024-02-14T15:03:26.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:26 smithi050 ceph-mon[36512]: mgrmap e23: x(active, since 1.02039s), standbys: y, z 2024-02-14T15:03:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.890+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79140fd1c0 con 0x7f7914106f40 2024-02-14T15:03:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.891+0000 7f790affd640 1 -- 172.21.15.17:0/3335795650 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f790c00a040 con 0x7f7914106f40 2024-02-14T15:03:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.891+0000 7f790affd640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f79000a1f20 0x7f79000a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.891+0000 7f79198da640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f79000a1f20 0x7f79000a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:26.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.892+0000 7f790affd640 1 -- 172.21.15.17:0/3335795650 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f790c0f0a40 con 0x7f7914106f40 2024-02-14T15:03:26.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.892+0000 7f79198da640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f79000a1f20 0x7f79000a43e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7910009bd0 tx=0x7f7910005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:26.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:26.896+0000 7f790affd640 1 -- 172.21.15.17:0/3335795650 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f790c0b9880 con 0x7f7914106f40 2024-02-14T15:03:26.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:26 smithi076 ceph-mon[37057]: [14/Feb/2024:15:03:25] ENGINE Bus STARTING 2024-02-14T15:03:26.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:26 smithi076 ceph-mon[37057]: [14/Feb/2024:15:03:25] ENGINE Serving on https://172.21.15.17:7150 2024-02-14T15:03:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:26 smithi076 ceph-mon[37057]: mgrmap e23: x(active, since 1.02039s), standbys: y, z 2024-02-14T15:03:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:26 smithi017 ceph-mon[32549]: [14/Feb/2024:15:03:25] ENGINE Bus STARTING 2024-02-14T15:03:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:26 smithi017 ceph-mon[32549]: [14/Feb/2024:15:03:25] ENGINE Serving on https://172.21.15.17:7150 2024-02-14T15:03:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:26 smithi017 ceph-mon[32549]: mgrmap e23: x(active, since 1.02039s), standbys: y, z 2024-02-14T15:03:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.135+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 --> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f791402cec0 con 0x7f79000a1f20 2024-02-14T15:03:27.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.142+0000 7f790affd640 1 -- 172.21.15.17:0/3335795650 <== mgr.15681 v2:172.21.15.17:6800/877978339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f791402cec0 con 0x7f79000a1f20 2024-02-14T15:03:27.143 INFO:teuthology.orchestra.run.smithi017.stdout:/ 1m 6m3h 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f79000a1f20 msgr2=0x7f79000a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f79000a1f20 0x7f79000a43e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7910009bd0 tx=0x7f7910005f90 comp rx=0 tx=0).stop 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 msgr2=0x7f79141001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f79141001d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f790c033000 tx=0x7f790c009360 comp rx=0 tx=0).stop 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 shutdown_connections 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f79000a1f20 0x7f79000a43e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7914106f40 0x7f79141001d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7914104610 0x7f79140ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 --2- 172.21.15.17:0/3335795650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7914101ce0 0x7f7914101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 >> 172.21.15.17:0/3335795650 conn(0x7f79140fb6d0 msgr2=0x7f7914102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 shutdown_connections 2024-02-14T15:03:27.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:27.145+0000 7f791bb3e640 1 -- 172.21.15.17:0/3335795650 wait complete. 2024-02-14T15:03:27.157 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-02-14T15:03:27.186 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-14T15:03:27.197 INFO:teuthology.run_tasks:Running task background_exec... 2024-02-14T15:03:27.219 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:03:27.219 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-02-14T15:03:27.229 INFO:teuthology.run_tasks:Running task workunit... 2024-02-14T15:03:27.255 INFO:tasks.workunit:Pulling workunits from ref a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T15:03:27.255 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-14T15:03:27.255 INFO:tasks.workunit:timeout=3h 2024-02-14T15:03:27.255 INFO:tasks.workunit:cleanup=True 2024-02-14T15:03:27.256 DEBUG:teuthology.orchestra.run.smithi017:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:27.271 INFO:teuthology.orchestra.run.smithi017.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:27.271 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-14T15:03:27.271 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-02-14T15:03:27.271 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T15:03:27.271 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-14T15:03:27.272 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-02-14 15:01:16.969437404 +0000 2024-02-14T15:03:27.272 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-02-14 15:01:16.969437404 +0000 2024-02-14T15:03:27.272 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-02-14 15:03:08.933610463 +0000 2024-02-14T15:03:27.272 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-02-14 15:01:16.969437404 +0000 2024-02-14T15:03:27.272 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-14T15:03:27.272 DEBUG:teuthology.orchestra.run.smithi017:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-14T15:03:27.343 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 2024-02-14T15:03:27.400 INFO:tasks.workunit.client.0.smithi017.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: [14/Feb/2024:15:03:25] ENGINE Serving on http://172.21.15.17:8765 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: [14/Feb/2024:15:03:25] ENGINE Bus STARTED 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='client.15699 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='client.15741 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:27.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:27 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: [14/Feb/2024:15:03:25] ENGINE Serving on http://172.21.15.17:8765 2024-02-14T15:03:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: [14/Feb/2024:15:03:25] ENGINE Bus STARTED 2024-02-14T15:03:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='client.15699 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='client.15741 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:28.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:27 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: [14/Feb/2024:15:03:25] ENGINE Serving on http://172.21.15.17:8765 2024-02-14T15:03:28.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: [14/Feb/2024:15:03:25] ENGINE Bus STARTED 2024-02-14T15:03:28.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='client.15699 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='client.15741 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:27 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-02-14T15:03:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: scrub summary: idle 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:28.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:03:28.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:28 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:03:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-02-14T15:03:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: scrub summary: idle 2024-02-14T15:03:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:03:29.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:28 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:03:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:03:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-02-14T15:03:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: scrub summary: idle 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:03:29.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:03:29.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:28 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:03:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-02-14T15:03:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-14T15:03:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-02-14T15:03:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Unable to set osd_memory_target on smithi017 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-14T15:03:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi050 to 175.2M 2024-02-14T15:03:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Unable to set osd_memory_target on smithi050 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-02-14T15:03:29.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Updating smithi017:/etc/ceph/ceph.conf 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:03:29.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-02-14T15:03:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-14T15:03:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-02-14T15:03:30.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Unable to set osd_memory_target on smithi017 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi050 to 175.2M 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Unable to set osd_memory_target on smithi050 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Updating smithi017:/etc/ceph/ceph.conf 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:30.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:03:30.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:03:30.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-02-14T15:03:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-14T15:03:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-02-14T15:03:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Unable to set osd_memory_target on smithi017 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-14T15:03:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi050 to 175.2M 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Unable to set osd_memory_target on smithi050 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Updating smithi017:/etc/ceph/ceph.conf 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Updating smithi076:/etc/ceph/ceph.conf 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:03:30.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:03:30.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:03:30.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:30 smithi076 ceph-mon[37057]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-14T15:03:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:31.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:31.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:30 smithi017 ceph-mon[32549]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-14T15:03:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-02-14T15:03:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: Updating smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: Updating smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: Updating smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T15:03:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:30 smithi050 ceph-mon[36512]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-14T15:03:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:32 smithi076 ceph-mon[37057]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:32 smithi017 ceph-mon[32549]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:32 smithi050 ceph-mon[36512]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-14T15:03:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:34 smithi076 ceph-mon[37057]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-02-14T15:03:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:34 smithi017 ceph-mon[32549]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-02-14T15:03:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:34 smithi050 ceph-mon[36512]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-02-14T15:03:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:36 smithi076 ceph-mon[37057]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-02-14T15:03:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:36 smithi017 ceph-mon[32549]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-02-14T15:03:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:36 smithi050 ceph-mon[36512]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-02-14T15:03:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:38 smithi076 ceph-mon[37057]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-02-14T15:03:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:38 smithi017 ceph-mon[32549]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-02-14T15:03:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:38 smithi050 ceph-mon[36512]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-02-14T15:03:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 -- 172.21.15.17:0/1065998658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 msgr2=0x7f258c10d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 --2- 172.21.15.17:0/1065998658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c10d9f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f257c009930 tx=0x7f257c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 -- 172.21.15.17:0/1065998658 shutdown_connections 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 --2- 172.21.15.17:0/1065998658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c10d9f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 --2- 172.21.15.17:0/1065998658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f258c10df30 0x7f258c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 --2- 172.21.15.17:0/1065998658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f258c069860 0x7f258c069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 -- 172.21.15.17:0/1065998658 >> 172.21.15.17:0/1065998658 conn(0x7f258c076f80 msgr2=0x7f258c077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 -- 172.21.15.17:0/1065998658 shutdown_connections 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.371+0000 7f259373e640 1 -- 172.21.15.17:0/1065998658 wait complete. 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 Processor -- start 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 -- start start 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f258c069860 0x7f258c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f258c10df30 0x7f258c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c199ff0 con 0x7f258c06a180 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c19a160 con 0x7f258c069860 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f259373e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c19a320 con 0x7f258c10df30 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.372+0000 7f2590cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c19fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49360/0 (socket says 172.21.15.17:49360) 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 -- 172.21.15.17:0/1416343690 learned_addr learned my addr 172.21.15.17:0/1416343690 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2591cdb640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f258c10df30 0x7f258c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f25914da640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f258c069860 0x7f258c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 -- 172.21.15.17:0/1416343690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f258c10df30 msgr2=0x7f258c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f258c10df30 0x7f258c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 -- 172.21.15.17:0/1416343690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f258c069860 msgr2=0x7f258c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f258c069860 0x7f258c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 -- 172.21.15.17:0/1416343690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c009560 con 0x7f258c06a180 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f2590cd9640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c19fd30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f257c009230 tx=0x7f257c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f25827fc640 1 -- 172.21.15.17:0/1416343690 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c004440 con 0x7f258c06a180 2024-02-14T15:03:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f25827fc640 1 -- 172.21.15.17:0/1416343690 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f257c0052d0 con 0x7f258c06a180 2024-02-14T15:03:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c19a5a0 con 0x7f258c06a180 2024-02-14T15:03:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f25827fc640 1 -- 172.21.15.17:0/1416343690 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c003d70 con 0x7f258c06a180 2024-02-14T15:03:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.373+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c1a6890 con 0x7f258c06a180 2024-02-14T15:03:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.376+0000 7f25827fc640 1 -- 172.21.15.17:0/1416343690 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f257c005440 con 0x7f258c06a180 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.376+0000 7f25963ef740 1 Processor -- start 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.376+0000 7f25963ef740 1 -- start start 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.376+0000 7f25827fc640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25780a22b0 0x7f25780a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25963ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ab9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2591cdb640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25780a22b0 0x7f25780a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25963ef740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25963ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25827fc640 1 -- 172.21.15.17:0/1416343690 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f257c0f0620 con 0x7f258c06a180 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25963ef740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05a279e0 con 0x557b05aba040 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25963ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05a24f10 con 0x557b05ac2c20 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25963ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05a206f0 con 0x557b05ab9720 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49368/0 (socket says 172.21.15.17:49368) 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 -- 172.21.15.17:0/1215665468 learned_addr learned my addr 172.21.15.17:0/1215665468 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f25914da640 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ab9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2591cdb640 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 -- 172.21.15.17:0/1215665468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 msgr2=0x557b05ab9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ab9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 -- 172.21.15.17:0/1215665468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 msgr2=0x557b05ac5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 -- 172.21.15.17:0/1215665468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b05731920 con 0x557b05aba040 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2591cdb640 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2591cdb640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25780a22b0 0x7f25780a4770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f258c069650 tx=0x7f2584010740 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.377+0000 7f2590cd9640 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac26e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f257c0ee010 tx=0x7f257c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f721fe3bd4f000fa server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25777fe640 1 -- 172.21.15.17:0/1215665468 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0bcc50 con 0x557b05aba040 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25777fe640 1 -- 172.21.15.17:0/1215665468 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f257c0c0da0 con 0x557b05aba040 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25777fe640 1 -- 172.21.15.17:0/1215665468 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0f04c0 con 0x557b05aba040 2024-02-14T15:03:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 -- 172.21.15.17:0/1215665468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 msgr2=0x557b05ac26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac26e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f257c0ee010 tx=0x7f257c005fd0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 -- 172.21.15.17:0/1215665468 shutdown_connections 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac26e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 --2- 172.21.15.17:0/1215665468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ab9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 -- 172.21.15.17:0/1215665468 >> 172.21.15.17:0/1215665468 conn(0x557b05ab3e50 msgr2=0x557b05ab4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 -- 172.21.15.17:0/1215665468 shutdown_connections 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.378+0000 7f25963ef740 1 -- 172.21.15.17:0/1215665468 wait complete. 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 -- start start 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 Processor -- start 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ab9720 0x557b05ac2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05aba040 0x557b05abf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ac2c20 0x557b05abfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b059bcfa0 con 0x557b05ac2c20 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b056f4250 con 0x557b05ab9720 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b059a6200 con 0x557b05aba040 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25963ef740 10 client.0 _ll_register_callbacks cb 0x557b05a15b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f2591cdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ac2c20 0x557b05abfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f2591cdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ac2c20 0x557b05abfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49376/0 (socket says 172.21.15.17:49376) 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f2591cdb640 1 -- 172.21.15.17:0/634444033 learned_addr learned my addr 172.21.15.17:0/634444033 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f2590cd9640 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05aba040 0x557b05abf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.379+0000 7f25914da640 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ab9720 0x557b05ac2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2591cdb640 1 -- 172.21.15.17:0/634444033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05aba040 msgr2=0x557b05abf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2591cdb640 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05aba040 0x557b05abf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2591cdb640 1 -- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ab9720 msgr2=0x557b05ac2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2591cdb640 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ab9720 0x557b05ac2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2591cdb640 1 -- 172.21.15.17:0/634444033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c1a6890 con 0x557b05ac2c20 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2591cdb640 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ac2c20 0x557b05abfe20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2584012c10 tx=0x7f2584004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98243d928636962 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2574ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2574ff9640 1 -- 172.21.15.17:0/634444033 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584020650 con 0x557b05ac2c20 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2574ff9640 1 -- 172.21.15.17:0/634444033 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2584019030 con 0x557b05ac2c20 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b05720600 con 0x557b05ac2c20 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f2574ff9640 1 -- 172.21.15.17:0/634444033 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2584012e50 con 0x557b05ac2c20 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f25963ef740 10 client.15771 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.380+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557b05a1ed70 con 0x557b05ac2c20 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f2574ff9640 1 -- 172.21.15.17:0/634444033 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557b05720600 con 0x557b05ac2c20 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f2574ff9640 1 -- 172.21.15.17:0/634444033 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2584027060 con 0x557b05ac2c20 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25963ef740 10 client.15771 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25963ef740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25963ef740 20 client.15771 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25963ef740 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05ad2780 0x557b05ad4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25963ef740 4 client.15771 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b057711b0 con 0x557b05ad2780 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25914da640 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05ad2780 0x557b05ad4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f25914da640 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05ad2780 0x557b05ad4b70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99db07218df89d8b server_cookie=258f9bd45841311b in_seq=0 out_seq=0 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.381+0000 7f2574ff9640 10 client.15771 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f2574ff9640 1 -- 172.21.15.17:0/634444033 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b057711b0 con 0x557b05ad2780 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f2574ff9640 10 client.15771 handle_command_reply: tid=0 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 client.15771 shutdown 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ac2c20 msgr2=0x557b05abfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ac2c20 0x557b05abfe20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2584012c10 tx=0x7f2584004ab0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 shutdown_connections 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05ad2780 0x557b05ad4b70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ac2c20 0x557b05abfe20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05aba040 0x557b05abf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 --2- 172.21.15.17:0/634444033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ab9720 0x557b05ac2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f2574ff9640 0 client.15771 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 >> 172.21.15.17:0/634444033 conn(0x557b05ab3e50 msgr2=0x557b05ab5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 shutdown_connections 2024-02-14T15:03:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.382+0000 7f25963ef740 1 -- 172.21.15.17:0/634444033 wait complete. 2024-02-14T15:03:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.383+0000 7f25963ef740 20 client.15771 trim_cache size 0 max 16384 2024-02-14T15:03:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 Processor -- start 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 -- start start 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ab9720 0x557b05abe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05aba040 0x557b05ac2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ac2c20 0x557b05ac2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05ab8ec0 con 0x557b05ab9720 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05ac0420 con 0x557b05aba040 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25963ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05ad2580 con 0x557b05ac2c20 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ab9720 0x557b05abe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ab9720 0x557b05abe820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49380/0 (socket says 172.21.15.17:49380) 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 -- 172.21.15.17:0/3083892531 learned_addr learned my addr 172.21.15.17:0/3083892531 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 -- 172.21.15.17:0/3083892531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ac2c20 msgr2=0x557b05ac2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ac2c20 0x557b05ac2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 -- 172.21.15.17:0/3083892531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05aba040 msgr2=0x557b05ac2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f2590cd9640 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05aba040 0x557b05ac2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05aba040 0x557b05ac2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.396+0000 7f25914da640 1 -- 172.21.15.17:0/3083892531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b059bb820 con 0x557b05ab9720 2024-02-14T15:03:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25914da640 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ab9720 0x557b05abe820 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2588005fd0 tx=0x7f258800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551cda5e5e16d8b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f2576ffd640 1 -- 172.21.15.17:0/3083892531 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588016040 con 0x557b05ab9720 2024-02-14T15:03:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f2576ffd640 1 -- 172.21.15.17:0/3083892531 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2588017040 con 0x557b05ab9720 2024-02-14T15:03:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f2576ffd640 1 -- 172.21.15.17:0/3083892531 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880047a0 con 0x557b05ab9720 2024-02-14T15:03:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 -- 172.21.15.17:0/3083892531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ab9720 msgr2=0x557b05abe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ab9720 0x557b05abe820 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2588005fd0 tx=0x7f258800b390 comp rx=0 tx=0).stop 2024-02-14T15:03:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 -- 172.21.15.17:0/3083892531 shutdown_connections 2024-02-14T15:03:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ac2c20 0x557b05ac2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05aba040 0x557b05ac2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 --2- 172.21.15.17:0/3083892531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05ab9720 0x557b05abe820 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 -- 172.21.15.17:0/3083892531 >> 172.21.15.17:0/3083892531 conn(0x557b05ab3e50 msgr2=0x557b05ab2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 -- 172.21.15.17:0/3083892531 shutdown_connections 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.397+0000 7f25963ef740 1 -- 172.21.15.17:0/3083892531 wait complete. 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 -- start start 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 Processor -- start 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ac85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05b28dc0 con 0x557b05aba040 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05b28f30 con 0x557b05ac2c20 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b05b290f0 con 0x557b05ab9720 2024-02-14T15:03:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25963ef740 10 client.0 _ll_register_callbacks cb 0x7f257c0ed8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f2590cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f2590cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49394/0 (socket says 172.21.15.17:49394) 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f2590cd9640 1 -- 172.21.15.17:0/4128831494 learned_addr learned my addr 172.21.15.17:0/4128831494 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.398+0000 7f25914da640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ac85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2591cdb640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2590cd9640 1 -- 172.21.15.17:0/4128831494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 msgr2=0x557b05ac85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2590cd9640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ac85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2590cd9640 1 -- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 msgr2=0x557b05ac9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2590cd9640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2590cd9640 1 -- 172.21.15.17:0/4128831494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2588011440 con 0x557b05aba040 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2591cdb640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2590cd9640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac8ae0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f257c0bca40 tx=0x7f257c0bdf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0d7eabedba59d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2574ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2574ff9640 1 -- 172.21.15.17:0/4128831494 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0db1e0 con 0x557b05aba040 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2574ff9640 1 -- 172.21.15.17:0/4128831494 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f257c0a7040 con 0x557b05aba040 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f257c0c0da0 con 0x557b05aba040 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2574ff9640 1 -- 172.21.15.17:0/4128831494 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257c0ed9a0 con 0x557b05aba040 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f25963ef740 10 client.15777 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557b059e1720 con 0x557b05aba040 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.399+0000 7f2574ff9640 1 -- 172.21.15.17:0/4128831494 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f257c0c9020 con 0x557b05aba040 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f2574ff9640 1 -- 172.21.15.17:0/4128831494 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f257c03c070 con 0x557b05aba040 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25963ef740 10 client.15777 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25963ef740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25963ef740 20 client.15777 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25963ef740 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05b2a420 0x557b05b2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25963ef740 4 client.15777 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b0575d550 con 0x557b05b2a420 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25914da640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05b2a420 0x557b05b2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f25914da640 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05b2a420 0x557b05b2c840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3683f36347dee79 server_cookie=b71a46c0dfcfffcc in_seq=0 out_seq=0 2024-02-14T15:03:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.400+0000 7f2574ff9640 10 client.15777 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f2574ff9640 1 -- 172.21.15.17:0/4128831494 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b0575d550 con 0x557b05b2a420 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f2574ff9640 10 client.15777 handle_command_reply: tid=0 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 client.15777 shutdown 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 msgr2=0x557b05ac8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac8ae0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f257c0bca40 tx=0x7f257c0bdf80 comp rx=0 tx=0).stop 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 shutdown_connections 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b05b2a420 0x557b05b2c840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b05aba040 0x557b05ac8ae0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b05ac2c20 0x557b05ac9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 --2- 172.21.15.17:0/4128831494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b05ab9720 0x557b05ac85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f2574ff9640 0 client.15777 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.414+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 >> 172.21.15.17:0/4128831494 conn(0x557b05ab3e50 msgr2=0x557b05aba560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 shutdown_connections 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f25963ef740 1 -- 172.21.15.17:0/4128831494 wait complete. 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f25963ef740 20 client.15777 trim_cache size 0 max 16384 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:03:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:03:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25780a22b0 msgr2=0x7f25780a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25780a22b0 0x7f25780a4770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f258c069650 tx=0x7f2584010740 comp rx=0 tx=0).stop 2024-02-14T15:03:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 msgr2=0x7f258c19fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c19fd30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f257c009230 tx=0x7f257c004040 comp rx=0 tx=0).stop 2024-02-14T15:03:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 shutdown_connections 2024-02-14T15:03:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25780a22b0 0x7f25780a4770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f258c06a180 0x7f258c19fd30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f258c10df30 0x7f258c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 --2- 172.21.15.17:0/1416343690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f258c069860 0x7f258c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 >> 172.21.15.17:0/1416343690 conn(0x7f258c076f80 msgr2=0x7f258c10f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 shutdown_connections 2024-02-14T15:03:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:40.415+0000 7f259373e640 1 -- 172.21.15.17:0/1416343690 wait complete. 2024-02-14T15:03:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:40 smithi076 ceph-mon[37057]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-02-14T15:03:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:40 smithi017 ceph-mon[32549]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-02-14T15:03:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:40 smithi050 ceph-mon[36512]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-02-14T15:03:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:42 smithi017 ceph-mon[32549]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-02-14T15:03:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:42 smithi050 ceph-mon[36512]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-02-14T15:03:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:42 smithi076 ceph-mon[37057]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-02-14T15:03:45.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:44 smithi017 ceph-mon[32549]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-02-14T15:03:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:44 smithi050 ceph-mon[36512]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-02-14T15:03:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:44 smithi076 ceph-mon[37057]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-02-14T15:03:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:46 smithi017 ceph-mon[32549]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:46 smithi050 ceph-mon[36512]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:47.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:03:47.218 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:03:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:46 smithi076 ceph-mon[37057]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 -- 172.21.15.17:0/2157560879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84541014b0 msgr2=0x7f8454104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 --2- 172.21.15.17:0/2157560879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84541014b0 0x7f8454104c90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8448009930 tx=0x7f844802bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 -- 172.21.15.17:0/2157560879 shutdown_connections 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 --2- 172.21.15.17:0/2157560879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84541014b0 0x7f8454104c90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 --2- 172.21.15.17:0/2157560879 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454105370 0x7f845410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 --2- 172.21.15.17:0/2157560879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8454100b00 0x7f8454100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 -- 172.21.15.17:0/2157560879 >> 172.21.15.17:0/2157560879 conn(0x7f8454076630 msgr2=0x7f84540fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 -- 172.21.15.17:0/2157560879 shutdown_connections 2024-02-14T15:03:47.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 -- 172.21.15.17:0/2157560879 wait complete. 2024-02-14T15:03:47.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.336+0000 7f8459113640 1 Processor -- start 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8459113640 1 -- start start 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8459113640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454100b00 0x7f845419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8459113640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8454105370 0x7f845419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8459113640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8454197610 0x7f8454197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8459113640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454198000 con 0x7f8454105370 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8459113640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454198170 con 0x7f8454100b00 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8459113640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454198330 con 0x7f8454197610 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8454105370 0x7f845419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8454105370 0x7f845419d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59264/0 (socket says 172.21.15.17:59264) 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 -- 172.21.15.17:0/3861420997 learned_addr learned my addr 172.21.15.17:0/3861420997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f8452d9d640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454100b00 0x7f845419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845359e640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8454197610 0x7f8454197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 -- 172.21.15.17:0/3861420997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8454197610 msgr2=0x7f8454197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8454197610 0x7f8454197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 -- 172.21.15.17:0/3861420997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454100b00 msgr2=0x7f845419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454100b00 0x7f845419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845259c640 1 -- 172.21.15.17:0/3861420997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8448009560 con 0x7f8454105370 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.337+0000 7f845359e640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8454197610 0x7f8454197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.338+0000 7f845259c640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8454105370 0x7f845419d9a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8448009230 tx=0x7f8448004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.338+0000 7f843bfff640 1 -- 172.21.15.17:0/3861420997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844802cbc0 con 0x7f8454105370 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.338+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84541a4320 con 0x7f8454105370 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.338+0000 7f843bfff640 1 -- 172.21.15.17:0/3861420997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84480052d0 con 0x7f8454105370 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.338+0000 7f843bfff640 1 -- 172.21.15.17:0/3861420997 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448003e30 con 0x7f8454105370 2024-02-14T15:03:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.338+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84541a4820 con 0x7f8454105370 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.340+0000 7f843bfff640 1 -- 172.21.15.17:0/3861420997 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f8448005440 con 0x7f8454105370 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.340+0000 7f845bdc4740 1 Processor -- start 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.340+0000 7f845bdc4740 1 -- start start 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a25aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f843bfff640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f843c0a22c0 0x7f843c0a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 0x55da3a2635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a265f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845bdc4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a2155b0 con 0x55da3a263b30 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845bdc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a1c92b0 con 0x55da3a25a630 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845bdc4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a1c67e0 con 0x55da3a25af50 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a265f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f8452d9d640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f843c0a22c0 0x7f843c0a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a265f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59274/0 (socket says 172.21.15.17:59274) 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 -- 172.21.15.17:0/3994412312 learned_addr learned my addr 172.21.15.17:0/3994412312 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a25aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f843bfff640 1 -- 172.21.15.17:0/3861420997 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f84480f08d0 con 0x7f8454105370 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845259c640 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 0x55da3a2635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 -- 172.21.15.17:0/3994412312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 msgr2=0x55da3a2635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 0x55da3a2635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 -- 172.21.15.17:0/3994412312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 msgr2=0x55da3a25aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a25aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 -- 172.21.15.17:0/3994412312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da3a184020 con 0x55da3a263b30 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f8452d9d640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f843c0a22c0 0x7f843c0a4780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8444005fd0 tx=0x7f8444009760 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.341+0000 7f845359e640 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a265f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f844c0127d0 tx=0x7f844c00d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba42da800d25753f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f84397fa640 1 -- 172.21.15.17:0/3994412312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c005220 con 0x55da3a263b30 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f84397fa640 1 -- 172.21.15.17:0/3994412312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f844c018030 con 0x55da3a263b30 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f84397fa640 1 -- 172.21.15.17:0/3994412312 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00a9d0 con 0x55da3a263b30 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 -- 172.21.15.17:0/3994412312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 msgr2=0x55da3a265f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a265f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f844c0127d0 tx=0x7f844c00d380 comp rx=0 tx=0).stop 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 -- 172.21.15.17:0/3994412312 shutdown_connections 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a265f20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 0x55da3a2635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3994412312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a25aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 -- 172.21.15.17:0/3994412312 >> 172.21.15.17:0/3994412312 conn(0x55da3a254d60 msgr2=0x55da3a255140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 -- 172.21.15.17:0/3994412312 shutdown_connections 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.342+0000 7f845bdc4740 1 -- 172.21.15.17:0/3994412312 wait complete. 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 -- start start 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 Processor -- start 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25a630 0x55da3a260b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25af50 0x55da3a2610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a261610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a1c1fc0 con 0x55da3a25a630 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da39e95430 con 0x55da3a25af50 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da39e51e00 con 0x55da3a263b30 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845bdc4740 10 client.0 _ll_register_callbacks cb 0x55da3a143870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f8452d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25a630 0x55da3a260b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f8452d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25a630 0x55da3a260b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59288/0 (socket says 172.21.15.17:59288) 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f8452d9d640 1 -- 172.21.15.17:0/4125968330 learned_addr learned my addr 172.21.15.17:0/4125968330 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845359e640 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a261610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.343+0000 7f845259c640 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25af50 0x55da3a2610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8452d9d640 1 -- 172.21.15.17:0/4125968330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 msgr2=0x55da3a261610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8452d9d640 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a261610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8452d9d640 1 -- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25af50 msgr2=0x55da3a2610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8452d9d640 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25af50 0x55da3a2610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8452d9d640 1 -- 172.21.15.17:0/4125968330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844c0124b0 con 0x55da3a25a630 2024-02-14T15:03:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8452d9d640 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25a630 0x55da3a260b90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f84440116c0 tx=0x7f8444004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739e2abedb99d70d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8436ff5640 1 -- 172.21.15.17:0/4125968330 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844401e8f0 con 0x55da3a25a630 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8436ff5640 1 -- 172.21.15.17:0/4125968330 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8444017070 con 0x55da3a25a630 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da39ec17e0 con 0x55da3a25a630 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8436ff5640 1 -- 172.21.15.17:0/4125968330 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f844401ea50 con 0x55da3a25a630 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f845bdc4740 10 client.15789 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55da3a1c0640 con 0x55da3a25a630 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8436ff5640 1 -- 172.21.15.17:0/4125968330 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da39ec17e0 con 0x55da3a25a630 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f8436ff5640 1 -- 172.21.15.17:0/4125968330 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8444024020 con 0x55da3a25a630 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f845bdc4740 10 client.15789 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f845bdc4740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.344+0000 7f845bdc4740 20 client.15789 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.345+0000 7f845bdc4740 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a274010 0x55da3a276430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.345+0000 7f845bdc4740 4 client.15789 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.345+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da39efe720 con 0x55da3a274010 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.345+0000 7f845259c640 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a274010 0x55da3a276430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.345+0000 7f8436ff5640 10 client.15789 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.345+0000 7f845259c640 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a274010 0x55da3a276430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75064d2c0d52580 server_cookie=9dffccdf91ca465d in_seq=0 out_seq=0 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f8436ff5640 1 -- 172.21.15.17:0/4125968330 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da39efe720 con 0x55da3a274010 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f8436ff5640 10 client.15789 handle_command_reply: tid=0 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 client.15789 shutdown 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25a630 msgr2=0x55da3a260b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25a630 0x55da3a260b90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f84440116c0 tx=0x7f8444004ab0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 shutdown_connections 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a274010 0x55da3a276430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a261610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25af50 0x55da3a2610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 --2- 172.21.15.17:0/4125968330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25a630 0x55da3a260b90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 >> 172.21.15.17:0/4125968330 conn(0x55da3a254d60 msgr2=0x55da3a256ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f8436ff5640 0 client.15789 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 shutdown_connections 2024-02-14T15:03:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 1 -- 172.21.15.17:0/4125968330 wait complete. 2024-02-14T15:03:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.346+0000 7f845bdc4740 20 client.15789 trim_cache size 0 max 16384 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.361+0000 7f845bdc4740 1 Processor -- start 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.361+0000 7f845bdc4740 1 -- start start 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a2632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 0x55da3a261430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a25f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845bdc4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a146fd0 con 0x55da3a263b30 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845bdc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a25c7d0 con 0x55da3a25a630 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845bdc4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a25a050 con 0x55da3a25af50 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a25f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a25f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59290/0 (socket says 172.21.15.17:59290) 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 -- 172.21.15.17:0/2351131025 learned_addr learned my addr 172.21.15.17:0/2351131025 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f8452d9d640 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a2632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 -- 172.21.15.17:0/2351131025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 msgr2=0x55da3a261430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 0x55da3a261430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 -- 172.21.15.17:0/2351131025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 msgr2=0x55da3a2632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a2632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 -- 172.21.15.17:0/2351131025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da3a15cb30 con 0x55da3a263b30 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f8452d9d640 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a2632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f845359e640 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a25f460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f844c005c30 tx=0x7f844c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c36978e17571fe1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f8438ff9640 1 -- 172.21.15.17:0/2351131025 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c022050 con 0x55da3a263b30 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f8438ff9640 1 -- 172.21.15.17:0/2351131025 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f844c023040 con 0x55da3a263b30 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.362+0000 7f8438ff9640 1 -- 172.21.15.17:0/2351131025 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c01db30 con 0x55da3a263b30 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 -- 172.21.15.17:0/2351131025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 msgr2=0x55da3a25f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a25f460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f844c005c30 tx=0x7f844c005bc0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 -- 172.21.15.17:0/2351131025 shutdown_connections 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a263b30 0x55da3a25f460 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a25af50 0x55da3a261430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 --2- 172.21.15.17:0/2351131025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a2632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 -- 172.21.15.17:0/2351131025 >> 172.21.15.17:0/2351131025 conn(0x55da3a254d60 msgr2=0x55da3a27add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 -- 172.21.15.17:0/2351131025 shutdown_connections 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 -- 172.21.15.17:0/2351131025 wait complete. 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 -- start start 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.363+0000 7f845bdc4740 1 Processor -- start 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a268a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25af50 0x55da3a268f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a2c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a269690 con 0x55da3a25af50 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a2c9a90 con 0x55da3a25a630 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3a2c9bd0 con 0x55da3a263b30 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 10 client.0 _ll_register_callbacks cb 0x7f844c00d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25af50 0x55da3a268f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25af50 0x55da3a268f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59294/0 (socket says 172.21.15.17:59294) 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 -- 172.21.15.17:0/3008727150 learned_addr learned my addr 172.21.15.17:0/3008727150 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845359e640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a2c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f8452d9d640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a268a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 -- 172.21.15.17:0/3008727150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 msgr2=0x55da3a2c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a2c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 -- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 msgr2=0x55da3a268a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a268a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 -- 172.21.15.17:0/3008727150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da3a184020 con 0x55da3a25af50 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845359e640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a2c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f8452d9d640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a268a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845259c640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25af50 0x55da3a268f90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f84480bce30 tx=0x7f8448009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f366d2a5f98af74 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f8436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f8436ff5640 1 -- 172.21.15.17:0/3008727150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84480b9510 con 0x55da3a25af50 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f8436ff5640 1 -- 172.21.15.17:0/3008727150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f84480b9670 con 0x55da3a25af50 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f844c018030 con 0x55da3a25af50 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f8436ff5640 1 -- 172.21.15.17:0/3008727150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84480bb9d0 con 0x55da3a25af50 2024-02-14T15:03:47.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 10 client.15795 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.364+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55da3a182880 con 0x55da3a25af50 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f8436ff5640 1 -- 172.21.15.17:0/3008727150 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f844c018030 con 0x55da3a25af50 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f8436ff5640 1 -- 172.21.15.17:0/3008727150 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f844803c070 con 0x55da3a25af50 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f845bdc4740 10 client.15795 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f845bdc4740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f845bdc4740 20 client.15795 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a2cb240 0x55da3a2cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f845bdc4740 4 client.15795 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da39f4d890 con 0x55da3a2cb240 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f8452d9d640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a2cb240 0x55da3a2cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f8436ff5640 10 client.15795 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.365+0000 7f8452d9d640 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a2cb240 0x55da3a2cd660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7b48555ba8722c server_cookie=98099950dc3ef78 in_seq=0 out_seq=0 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8436ff5640 1 -- 172.21.15.17:0/3008727150 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da39f4d890 con 0x55da3a2cb240 2024-02-14T15:03:47.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8436ff5640 10 client.15795 handle_command_reply: tid=0 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 client.15795 shutdown 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25af50 msgr2=0x55da3a268f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25af50 0x55da3a268f90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f84480bce30 tx=0x7f8448009260 comp rx=0 tx=0).stop 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 shutdown_connections 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55da3a2cb240 0x55da3a2cd660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da3a25af50 0x55da3a268f90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da3a263b30 0x55da3a2c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 --2- 172.21.15.17:0/3008727150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55da3a25a630 0x55da3a268a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8436ff5640 0 client.15795 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 >> 172.21.15.17:0/3008727150 conn(0x55da3a254d60 msgr2=0x55da3a25f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 shutdown_connections 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 1 -- 172.21.15.17:0/3008727150 wait complete. 2024-02-14T15:03:47.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f845bdc4740 20 client.15795 trim_cache size 0 max 16384 2024-02-14T15:03:47.374 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:47.374 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "no active scrubs running", 2024-02-14T15:03:47.374 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": {} 2024-02-14T15:03:47.374 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f843c0a22c0 msgr2=0x7f843c0a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f843c0a22c0 0x7f843c0a4780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8444005fd0 tx=0x7f8444009760 comp rx=0 tx=0).stop 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8454105370 msgr2=0x7f845419d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8454105370 0x7f845419d9a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8448009230 tx=0x7f8448004270 comp rx=0 tx=0).stop 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 shutdown_connections 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f843c0a22c0 0x7f843c0a4780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8454105370 0x7f845419d9a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8454197610 0x7f8454197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 --2- 172.21.15.17:0/3861420997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8454100b00 0x7f845419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.366+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 >> 172.21.15.17:0/3861420997 conn(0x7f8454076630 msgr2=0x7f84540ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.367+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 shutdown_connections 2024-02-14T15:03:47.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.367+0000 7f8459113640 1 -- 172.21.15.17:0/3861420997 wait complete. 2024-02-14T15:03:47.378 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-14T15:03:47.378 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/195011913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68101440 msgr2=0x7f3e68101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/195011913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68101440 0x7f3e68101820 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009930 tx=0x7f3e5802bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/195011913 shutdown_connections 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/195011913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e68106510 0x7f3e6810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/195011913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e68101df0 0x7f3e68105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/195011913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68101440 0x7f3e68101820 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/195011913 >> 172.21.15.17:0/195011913 conn(0x7f3e680fcce0 msgr2=0x7f3e680ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/195011913 shutdown_connections 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/195011913 wait complete. 2024-02-14T15:03:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 Processor -- start 2024-02-14T15:03:47.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.481+0000 7f3e6e5a1640 1 -- start start 2024-02-14T15:03:47.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6e5a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e68101440 0x7f3e6819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6e5a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68106510 0x7f3e6819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6e5a1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e68197710 0x7f3e68197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6e5a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e681981e0 con 0x7f3e68106510 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6e5a1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e68198350 con 0x7f3e68197710 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6e5a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e68198510 con 0x7f3e68101440 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68106510 0x7f3e6819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68106510 0x7f3e6819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59318/0 (socket says 172.21.15.17:59318) 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 -- 172.21.15.17:0/3288582592 learned_addr learned my addr 172.21.15.17:0/3288582592 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e67fff640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e68101440 0x7f3e6819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6cb3e640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e68197710 0x7f3e68197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 -- 172.21.15.17:0/3288582592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e68101440 msgr2=0x7f3e6819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e68101440 0x7f3e6819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 -- 172.21.15.17:0/3288582592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e68197710 msgr2=0x7f3e68197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e68197710 0x7f3e68197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 -- 172.21.15.17:0/3288582592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e58009560 con 0x7f3e68106510 2024-02-14T15:03:47.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e677fe640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68106510 0x7f3e6819db90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00e9f0 tx=0x7f3e5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e657fa640 1 -- 172.21.15.17:0/3288582592 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0096c0 con 0x7f3e68106510 2024-02-14T15:03:47.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e681a44b0 con 0x7f3e68106510 2024-02-14T15:03:47.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e657fa640 1 -- 172.21.15.17:0/3288582592 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e5c005500 con 0x7f3e68106510 2024-02-14T15:03:47.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.482+0000 7f3e657fa640 1 -- 172.21.15.17:0/3288582592 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c010430 con 0x7f3e68106510 2024-02-14T15:03:47.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.483+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e681a4940 con 0x7f3e68106510 2024-02-14T15:03:47.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.483+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e68102910 con 0x7f3e68106510 2024-02-14T15:03:47.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.485+0000 7f3e657fa640 1 -- 172.21.15.17:0/3288582592 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f3e5c005000 con 0x7f3e68106510 2024-02-14T15:03:47.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.485+0000 7f3e657fa640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3e500a22c0 0x7f3e500a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.485+0000 7f3e67fff640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3e500a22c0 0x7f3e500a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.485+0000 7f3e657fa640 1 -- 172.21.15.17:0/3288582592 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f3e5c014070 con 0x7f3e68106510 2024-02-14T15:03:47.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.486+0000 7f3e67fff640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3e500a22c0 0x7f3e500a4780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e58005ab0 tx=0x7f3e58005fd0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.489+0000 7f3e657fa640 1 -- 172.21.15.17:0/3288582592 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e5c097260 con 0x7f3e68106510 2024-02-14T15:03:47.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.709+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e68109740 con 0x7f3e68106510 2024-02-14T15:03:47.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.710+0000 7f3e657fa640 1 -- 172.21.15.17:0/3288582592 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f3e5c09a920 con 0x7f3e68106510 2024-02-14T15:03:47.711 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:03:47.711 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:03:09.607099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:03:47.711 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2024-02-14T15:03:47.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.712+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3e500a22c0 msgr2=0x7f3e500a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.712+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3e500a22c0 0x7f3e500a4780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e58005ab0 tx=0x7f3e58005fd0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.712+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68106510 msgr2=0x7f3e6819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.712+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68106510 0x7f3e6819db90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00e9f0 tx=0x7f3e5c00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.712+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 shutdown_connections 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.712+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3e500a22c0 0x7f3e500a4780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.713+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e68106510 0x7f3e6819db90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.713+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e68197710 0x7f3e68197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.713+0000 7f3e6e5a1640 1 --2- 172.21.15.17:0/3288582592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e68101440 0x7f3e6819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.713+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 >> 172.21.15.17:0/3288582592 conn(0x7f3e680fcce0 msgr2=0x7f3e680fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.713+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 shutdown_connections 2024-02-14T15:03:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.713+0000 7f3e6e5a1640 1 -- 172.21.15.17:0/3288582592 wait complete. 2024-02-14T15:03:47.725 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:03:47.725 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-14T15:03:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1705289702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 msgr2=0x7f0ed8102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1705289702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8102180 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009930 tx=0x7f0ecc02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1705289702 shutdown_connections 2024-02-14T15:03:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1705289702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8102180 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1705289702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ed81026c0 0x7f0ed810d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1705289702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed8100440 0x7f0ed8100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1705289702 >> 172.21.15.17:0/1705289702 conn(0x7f0ed8077310 msgr2=0x7f0ed8077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1705289702 shutdown_connections 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.829+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1705289702 wait complete. 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 Processor -- start 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 -- start start 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ed8100440 0x7f0ed8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed81026c0 0x7f0ed81a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8199550 con 0x7f0ed8101d20 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81a3d50 con 0x7f0ed8100440 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0edd5bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81a3e90 con 0x7f0ed81026c0 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59338/0 (socket says 172.21.15.17:59338) 2024-02-14T15:03:47.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 -- 172.21.15.17:0/1946652681 learned_addr learned my addr 172.21.15.17:0/1946652681 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ed8100440 0x7f0ed8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed81026c0 0x7f0ed81a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 -- 172.21.15.17:0/1946652681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed81026c0 msgr2=0x7f0ed81a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed81026c0 0x7f0ed81a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 -- 172.21.15.17:0/1946652681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ed8100440 msgr2=0x7f0ed8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ed8100440 0x7f0ed8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed67fc640 1 -- 172.21.15.17:0/1946652681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc009560 con 0x7f0ed8101d20 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.830+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed81026c0 0x7f0ed81a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.831+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8198cd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009430 tx=0x7f0ecc02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.831+0000 7f0ebffff640 1 -- 172.21.15.17:0/1946652681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc004350 con 0x7f0ed8101d20 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.831+0000 7f0ebffff640 1 -- 172.21.15.17:0/1946652681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ecc0044b0 con 0x7f0ed8101d20 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.831+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed81a4090 con 0x7f0ed8101d20 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.831+0000 7f0ebffff640 1 -- 172.21.15.17:0/1946652681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc003da0 con 0x7f0ed8101d20 2024-02-14T15:03:47.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.831+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed81a4540 con 0x7f0ed8101d20 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ebffff640 1 -- 172.21.15.17:0/1946652681 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f0ecc004df0 con 0x7f0ed8101d20 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 Processor -- start 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 -- start start 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a3690 0x56357f4abc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f401c80 con 0x56357f4a3690 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f411320 con 0x56357f4a2ce0 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ee0188740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f40e850 con 0x56357f4ac210 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a3690 0x56357f4abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a3690 0x56357f4abc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59348/0 (socket says 172.21.15.17:59348) 2024-02-14T15:03:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/182071899 learned_addr learned my addr 172.21.15.17:0/182071899 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ebffff640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ec00a22c0 0x7f0ec00a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ec00a22c0 0x7f0ec00a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ebffff640 1 -- 172.21.15.17:0/1946652681 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0ecc0f06d0 con 0x7f0ed8101d20 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/182071899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 msgr2=0x56357f4ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/182071899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 msgr2=0x56357f4a30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4a30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/182071899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357f3d0e30 con 0x56357f4a3690 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.834+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a3690 0x56357f4abc40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80114f0 tx=0x7f0ec80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09f525475317a3a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ec00a22c0 0x7f0ec00a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005e00 tx=0x7f0ed0005d30 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ebd7fa640 1 -- 172.21.15.17:0/182071899 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004d20 con 0x56357f4a3690 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ebd7fa640 1 -- 172.21.15.17:0/182071899 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec8004e80 con 0x56357f4a3690 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ebd7fa640 1 -- 172.21.15.17:0/182071899 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800f7c0 con 0x56357f4a3690 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 -- 172.21.15.17:0/182071899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a3690 msgr2=0x56357f4abc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a3690 0x56357f4abc40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80114f0 tx=0x7f0ec80119b0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 -- 172.21.15.17:0/182071899 shutdown_connections 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a3690 0x56357f4abc40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 --2- 172.21.15.17:0/182071899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 -- 172.21.15.17:0/182071899 >> 172.21.15.17:0/182071899 conn(0x56357f49d3b0 msgr2=0x56357f49d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.835+0000 7f0ee0188740 1 -- 172.21.15.17:0/182071899 shutdown_connections 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 -- 172.21.15.17:0/182071899 wait complete. 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 -- start start 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 Processor -- start 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f40a030 con 0x56357f4a2ce0 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f0dd430 con 0x56357f4a3690 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f099e00 con 0x56357f4ac210 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ee0188740 10 client.0 _ll_register_callbacks cb 0x56357f11abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59360/0 (socket says 172.21.15.17:59360) 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4194621952 learned_addr learned my addr 172.21.15.17:0/4194621952 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4194621952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 msgr2=0x56357f4a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4194621952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 msgr2=0x56357f4a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.836+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4194621952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec8009c80 con 0x56357f4a2ce0 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4a9160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ec801b040 tx=0x7f0ec8004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbb403469fcd36b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ebaff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ebaff5640 1 -- 172.21.15.17:0/4194621952 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80044b0 con 0x56357f4a2ce0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ebaff5640 1 -- 172.21.15.17:0/4194621952 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0ec800f560 con 0x56357f4a2ce0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56357f146720 con 0x56357f4a2ce0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ebaff5640 1 -- 172.21.15.17:0/4194621952 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec800f720 con 0x56357f4a2ce0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 10 client.15813 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56357f4086b0 con 0x56357f4a2ce0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ebaff5640 1 -- 172.21.15.17:0/4194621952 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56357f146720 con 0x56357f4a2ce0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ebaff5640 1 -- 172.21.15.17:0/4194621952 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ec8021040 con 0x56357f4a2ce0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 10 client.15813 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 10 client.15813 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 20 client.15813 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f4bc340 0x56357f4be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 4 client.15813 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.837+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56357f195890 con 0x56357f4bc340 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.838+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f4bc340 0x56357f4be730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.838+0000 7f0ebaff5640 10 client.15813 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.838+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f4bc340 0x56357f4be730 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29db798768a225e7 server_cookie=92941499255aa11f in_seq=0 out_seq=0 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ebaff5640 1 -- 172.21.15.17:0/4194621952 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56357f195890 con 0x56357f4bc340 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ebaff5640 10 client.15813 handle_command_reply: tid=0 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 client.15813 shutdown 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 msgr2=0x56357f4a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4a9160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ec801b040 tx=0x7f0ec8004c50 comp rx=0 tx=0).stop 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 shutdown_connections 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f4bc340 0x56357f4be730 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f4a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ebaff5640 0 client.15813 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 --2- 172.21.15.17:0/4194621952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4a9160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 >> 172.21.15.17:0/4194621952 conn(0x56357f49d3b0 msgr2=0x56357f49f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 shutdown_connections 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 1 -- 172.21.15.17:0/4194621952 wait complete. 2024-02-14T15:03:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.839+0000 7f0ee0188740 20 client.15813 trim_cache size 0 max 16384 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.850+0000 7f0ee0188740 1 Processor -- start 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ee0188740 1 -- start start 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4ab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4a3690 0x56357f4a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4ac210 0x56357f4a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ee0188740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f38f130 con 0x56357f4ac210 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ee0188740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f498360 con 0x56357f4a2ce0 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ee0188740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f4a4d70 con 0x56357f4a3690 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4ac210 0x56357f4a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4ac210 0x56357f4a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59362/0 (socket says 172.21.15.17:59362) 2024-02-14T15:03:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 -- 172.21.15.17:0/3135444961 learned_addr learned my addr 172.21.15.17:0/3135444961 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4a3690 0x56357f4a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 -- 172.21.15.17:0/3135444961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4a3690 msgr2=0x56357f4a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4a3690 0x56357f4a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 -- 172.21.15.17:0/3135444961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 msgr2=0x56357f4ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 -- 172.21.15.17:0/3135444961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357f3a4c90 con 0x56357f4ac210 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4ac210 0x56357f4a9bf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000f830 tx=0x7f0ed0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca78c9a4d61cd739 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ebcff9640 1 -- 172.21.15.17:0/3135444961 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0011950 con 0x56357f4ac210 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ebcff9640 1 -- 172.21.15.17:0/3135444961 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ed001f400 con 0x56357f4ac210 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.851+0000 7f0ebcff9640 1 -- 172.21.15.17:0/3135444961 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0010610 con 0x56357f4ac210 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 -- 172.21.15.17:0/3135444961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4ac210 msgr2=0x56357f4a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4ac210 0x56357f4a9bf0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000f830 tx=0x7f0ed0004730 comp rx=0 tx=0).stop 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 -- 172.21.15.17:0/3135444961 shutdown_connections 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4ac210 0x56357f4a9bf0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4a3690 0x56357f4a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 --2- 172.21.15.17:0/3135444961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a2ce0 0x56357f4ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 -- 172.21.15.17:0/3135444961 >> 172.21.15.17:0/3135444961 conn(0x56357f49d3b0 msgr2=0x56357f4c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 -- 172.21.15.17:0/3135444961 shutdown_connections 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 -- 172.21.15.17:0/3135444961 wait complete. 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 -- start start 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 Processor -- start 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.852+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f512830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f4b2ec0 con 0x56357f4a2ce0 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f512d70 con 0x56357f4a3690 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357f512eb0 con 0x56357f4ac210 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 10 client.0 _ll_register_callbacks cb 0x7f0ec8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59366/0 (socket says 172.21.15.17:59366) 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/2010808812 learned_addr learned my addr 172.21.15.17:0/2010808812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f512830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 msgr2=0x56357f512830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f512830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/2010808812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 msgr2=0x56357f4b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/2010808812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed000f510 con 0x56357f4a2ce0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed77fe640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f512830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ed6ffd640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4b20b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005b40 tx=0x7f0ec8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac58e9495154970 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ebaff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ebaff5640 1 -- 172.21.15.17:0/2010808812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004cc0 con 0x56357f4a2ce0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ebaff5640 1 -- 172.21.15.17:0/2010808812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0ec80046c0 con 0x56357f4a2ce0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56357f1097e0 con 0x56357f4a2ce0 2024-02-14T15:03:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ebaff5640 1 -- 172.21.15.17:0/2010808812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec8005000 con 0x56357f4a2ce0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 10 client.15819 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.853+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56357f3caaa0 con 0x56357f4a2ce0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ebaff5640 1 -- 172.21.15.17:0/2010808812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56357f1097e0 con 0x56357f4a2ce0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ebaff5640 1 -- 172.21.15.17:0/2010808812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ec8021030 con 0x56357f4a2ce0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ee0188740 10 client.15819 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ee0188740 10 client.15819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ee0188740 20 client.15819 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ee0188740 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f5144c0 0x56357f5168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ee0188740 4 client.15819 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56357f0b5700 con 0x56357f5144c0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f5144c0 0x56357f5168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ed67fc640 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f5144c0 0x56357f5168e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20e5e627a73b62f6 server_cookie=97ff83beb04939ab in_seq=0 out_seq=0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.854+0000 7f0ebaff5640 10 client.15819 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ebaff5640 1 -- 172.21.15.17:0/2010808812 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56357f0b5700 con 0x56357f5144c0 2024-02-14T15:03:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ebaff5640 10 client.15819 handle_command_reply: tid=0 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 client.15819 shutdown 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 msgr2=0x56357f4b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4b20b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005b40 tx=0x7f0ec8009290 comp rx=0 tx=0).stop 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 shutdown_connections 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56357f5144c0 0x56357f5168e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56357f4ac210 0x56357f512830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56357f4a3690 0x56357f4b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ebaff5640 0 client.15819 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 --2- 172.21.15.17:0/2010808812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56357f4a2ce0 0x56357f4b20b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 >> 172.21.15.17:0/2010808812 conn(0x56357f49d3b0 msgr2=0x56357f49bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 shutdown_connections 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 1 -- 172.21.15.17:0/2010808812 wait complete. 2024-02-14T15:03:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0ee0188740 20 client.15819 trim_cache size 0 max 16384 2024-02-14T15:03:47.863 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:03:47.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ec00a22c0 msgr2=0x7f0ec00a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ec00a22c0 0x7f0ec00a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005e00 tx=0x7f0ed0005d30 comp rx=0 tx=0).stop 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 msgr2=0x7f0ed8198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8198cd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009430 tx=0x7f0ecc02c6c0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 shutdown_connections 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ec00a22c0 0x7f0ec00a4780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed8101d20 0x7f0ed8198cd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed81026c0 0x7f0ed81a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 --2- 172.21.15.17:0/1946652681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ed8100440 0x7f0ed8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.855+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 >> 172.21.15.17:0/1946652681 conn(0x7f0ed8077310 msgr2=0x7f0ed8103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.856+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 shutdown_connections 2024-02-14T15:03:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.856+0000 7f0edd5bc640 1 -- 172.21.15.17:0/1946652681 wait complete. 2024-02-14T15:03:47.867 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:03:47.867 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-14T15:03:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.970+0000 7f2d30dc9640 1 -- 172.21.15.17:0/35181983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c100360 msgr2=0x7f2d2c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.970+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/35181983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c100360 0x7f2d2c105e20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009930 tx=0x7f2d1402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 -- 172.21.15.17:0/35181983 shutdown_connections 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/35181983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c100360 0x7f2d2c105e20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/35181983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c106500 0x7f2d2c10e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/35181983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2c0ff9b0 0x7f2d2c0ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 -- 172.21.15.17:0/35181983 >> 172.21.15.17:0/35181983 conn(0x7f2d2c076630 msgr2=0x7f2d2c0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 -- 172.21.15.17:0/35181983 shutdown_connections 2024-02-14T15:03:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 -- 172.21.15.17:0/35181983 wait complete. 2024-02-14T15:03:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 Processor -- start 2024-02-14T15:03:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.971+0000 7f2d30dc9640 1 -- start start 2024-02-14T15:03:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0ff9b0 0x7f2d2c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2c106500 0x7f2d2c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c1976c0 0x7f2d2c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c198190 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c198300 con 0x7f2d2c106500 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1984c0 con 0x7f2d2c1976c0 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0ff9b0 0x7f2d2c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0ff9b0 0x7f2d2c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59388/0 (socket says 172.21.15.17:59388) 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3961183624 learned_addr learned my addr 172.21.15.17:0/3961183624 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2c106500 0x7f2d2c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c1976c0 0x7f2d2c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3961183624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c1976c0 msgr2=0x7f2d2c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c1976c0 0x7f2d2c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3961183624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2c106500 msgr2=0x7f2d2c19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2c106500 0x7f2d2c19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3961183624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d14009560 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c1976c0 0x7f2d2c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0ff9b0 0x7f2d2c19d600 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e9f0 tx=0x7f2d2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d237fe640 1 -- 172.21.15.17:0/3961183624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24009700 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c1a4460 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d237fe640 1 -- 172.21.15.17:0/3961183624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d2400cd80 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d237fe640 1 -- 172.21.15.17:0/3961183624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24018920 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.972+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c1a4940 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.974+0000 7f2d237fe640 1 -- 172.21.15.17:0/3961183624 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f2d24018a80 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 Processor -- start 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 -- start start 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d237fe640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2d180a22c0 0x7f2d180a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cece0 0x5594e86cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86cf690 0x5594e86d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86d8210 0x5594e86da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e862dc80 con 0x5594e86d8210 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e863d320 con 0x5594e86cf690 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d33997740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e863a850 con 0x5594e86cece0 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d237fe640 1 -- 172.21.15.17:0/3961183624 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2d24014070 con 0x7f2d2c0ff9b0 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2d180a22c0 0x7f2d180a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d2ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86d8210 0x5594e86da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d2ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86d8210 0x5594e86da660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59404/0 (socket says 172.21.15.17:59404) 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d2ad9d640 1 -- 172.21.15.17:0/2651774799 learned_addr learned my addr 172.21.15.17:0/2651774799 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.975+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86cf690 0x5594e86d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cece0 0x5594e86cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2ad9d640 1 -- 172.21.15.17:0/2651774799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cece0 msgr2=0x5594e86cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cece0 0x5594e86cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2ad9d640 1 -- 172.21.15.17:0/2651774799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86cf690 msgr2=0x5594e86d7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86cf690 0x5594e86d7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2ad9d640 1 -- 172.21.15.17:0/2651774799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594e85fce30 con 0x5594e86d8210 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2d180a22c0 0x7f2d180a4780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c005c40 tx=0x7f2d1c009d70 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86d8210 0x5594e86da660 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c011ce0 tx=0x7f2d1c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd673e7514d4951 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d20ff9640 1 -- 172.21.15.17:0/2651774799 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c0144f0 con 0x5594e86d8210 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d20ff9640 1 -- 172.21.15.17:0/2651774799 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c01a040 con 0x5594e86d8210 2024-02-14T15:03:47.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.976+0000 7f2d20ff9640 1 -- 172.21.15.17:0/2651774799 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c005360 con 0x5594e86d8210 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 -- 172.21.15.17:0/2651774799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86d8210 msgr2=0x5594e86da660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86d8210 0x5594e86da660 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c011ce0 tx=0x7f2d1c01b000 comp rx=0 tx=0).stop 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 -- 172.21.15.17:0/2651774799 shutdown_connections 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86d8210 0x5594e86da660 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86cf690 0x5594e86d7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 --2- 172.21.15.17:0/2651774799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cece0 0x5594e86cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 -- 172.21.15.17:0/2651774799 >> 172.21.15.17:0/2651774799 conn(0x5594e86c93b0 msgr2=0x5594e86c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 -- 172.21.15.17:0/2651774799 shutdown_connections 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 -- 172.21.15.17:0/2651774799 wait complete. 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 -- start start 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.977+0000 7f2d33997740 1 Processor -- start 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e8636030 con 0x5594e86cece0 2024-02-14T15:03:47.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e8309430 con 0x5594e86d8210 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e82c5e00 con 0x5594e86cf690 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 10 client.0 _ll_register_callbacks cb 0x5594e8346be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59406/0 (socket says 172.21.15.17:59406) 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2930548928 learned_addr learned my addr 172.21.15.17:0/2930548928 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 msgr2=0x5594e86d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2930548928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 msgr2=0x5594e86d5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2930548928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c011aa0 con 0x5594e86cece0 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d5180 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2d240978a0 tx=0x7f2d2409a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770b477a2ed81b44 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:47.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2930548928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2409b5d0 con 0x5594e86cece0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2930548928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2d240781a0 con 0x5594e86cece0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594e83357e0 con 0x5594e86cece0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2930548928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d2409aed0 con 0x5594e86cece0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 10 client.15831 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.978+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5594e86346b0 con 0x5594e86cece0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2930548928 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5594e83357e0 con 0x5594e86cece0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2930548928 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d2409a320 con 0x5594e86cece0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d33997740 10 client.15831 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d33997740 10 client.15831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d33997740 20 client.15831 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d33997740 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e86e8360 0x5594e86ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d33997740 4 client.15831 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594e8372720 con 0x5594e86e8360 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e86e8360 0x5594e86ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d11ffb640 10 client.15831 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.979+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e86e8360 0x5594e86ea750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2426c43fc9a1e13c server_cookie=277936a0effed954 in_seq=0 out_seq=0 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2930548928 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594e8372720 con 0x5594e86e8360 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d11ffb640 10 client.15831 handle_command_reply: tid=0 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 client.15831 shutdown 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 msgr2=0x5594e86d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d5180 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2d240978a0 tx=0x7f2d2409a920 comp rx=0 tx=0).stop 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 shutdown_connections 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e86e8360 0x5594e86ea750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d11ffb640 0 client.15831 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 --2- 172.21.15.17:0/2930548928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d5180 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 >> 172.21.15.17:0/2930548928 conn(0x5594e86c93b0 msgr2=0x5594e86cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 shutdown_connections 2024-02-14T15:03:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 1 -- 172.21.15.17:0/2930548928 wait complete. 2024-02-14T15:03:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.980+0000 7f2d33997740 20 client.15831 trim_cache size 0 max 16384 2024-02-14T15:03:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 Processor -- start 2024-02-14T15:03:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 -- start start 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e85bb130 con 0x5594e86cece0 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e86c8f10 con 0x5594e86d8210 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d33997740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e86e81b0 con 0x5594e86cf690 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59414/0 (socket says 172.21.15.17:59414) 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3089238096 learned_addr learned my addr 172.21.15.17:0/3089238096 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3089238096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 msgr2=0x5594e86d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.992+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3089238096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 msgr2=0x5594e86d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d2a59c640 1 -- 172.21.15.17:0/3089238096 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594e85d0c90 con 0x5594e86cece0 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d78b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2d2409a870 tx=0x7f2d240993e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaf23475db02712 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d13fff640 1 -- 172.21.15.17:0/3089238096 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240cbc20 con 0x5594e86cece0 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d13fff640 1 -- 172.21.15.17:0/3089238096 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d2409b5d0 con 0x5594e86cece0 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d13fff640 1 -- 172.21.15.17:0/3089238096 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240cbf90 con 0x5594e86cece0 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 -- 172.21.15.17:0/3089238096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 msgr2=0x5594e86d78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d78b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2d2409a870 tx=0x7f2d240993e0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 -- 172.21.15.17:0/3089238096 shutdown_connections 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 --2- 172.21.15.17:0/3089238096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86d78b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 -- 172.21.15.17:0/3089238096 >> 172.21.15.17:0/3089238096 conn(0x5594e86c93b0 msgr2=0x5594e86d3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 -- 172.21.15.17:0/3089238096 shutdown_connections 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.993+0000 7f2d33997740 1 -- 172.21.15.17:0/3089238096 wait complete. 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 -- start start 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 Processor -- start 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86de150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e86dfb10 con 0x5594e86cece0 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e873e7d0 con 0x5594e86d8210 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e873e960 con 0x5594e86cf690 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d33997740 10 client.0 _ll_register_callbacks cb 0x7f2d1c014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86ddc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59424/0 (socket says 172.21.15.17:59424) 2024-02-14T15:03:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2859630149 learned_addr learned my addr 172.21.15.17:0/2859630149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86de150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 msgr2=0x5594e86de150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86de150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2859630149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 msgr2=0x5594e86de690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86de690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 -- 172.21.15.17:0/2859630149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594e86346b0 con 0x5594e86cece0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2ad9d640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d2a59c640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86ddc10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d240a3140 tx=0x7f2d240a3670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e16462f5880070 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.994+0000 7f2d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2859630149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240cc190 con 0x5594e86cece0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2859630149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2d2409aed0 con 0x5594e86cece0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d1c01a040 con 0x5594e86cece0 2024-02-14T15:03:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2859630149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d240cbc20 con 0x5594e86cece0 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 10 client.15837 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5594e85f6aa0 con 0x5594e86cece0 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2859630149 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c01a040 con 0x5594e86cece0 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2859630149 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d24099f60 con 0x5594e86cece0 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 10 client.15837 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 10 client.15837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 20 client.15837 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e873fc30 0x5594e8742020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 4 client.15837 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5594e83c1890 con 0x5594e873fc30 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.995+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e873fc30 0x5594e8742020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d11ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d29d9b640 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e873fc30 0x5594e8742020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6242414c4ec4728 server_cookie=d215b4ded5d7cd72 in_seq=0 out_seq=0 2024-02-14T15:03:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d11ffb640 1 -- 172.21.15.17:0/2859630149 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5594e83c1890 con 0x5594e873fc30 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d11ffb640 10 client.15837 handle_command_reply: tid=0 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 client.15837 shutdown 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 msgr2=0x5594e86ddc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86ddc10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d240a3140 tx=0x7f2d240a3670 comp rx=0 tx=0).stop 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 shutdown_connections 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x5594e873fc30 0x5594e8742020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5594e86d8210 0x5594e86de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5594e86cf690 0x5594e86de150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d11ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 --2- 172.21.15.17:0/2859630149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5594e86cece0 0x5594e86ddc10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 >> 172.21.15.17:0/2859630149 conn(0x5594e86c93b0 msgr2=0x5594e86c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 shutdown_connections 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 1 -- 172.21.15.17:0/2859630149 wait complete. 2024-02-14T15:03:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.996+0000 7f2d33997740 20 client.15837 trim_cache size 0 max 16384 2024-02-14T15:03:48.005 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:03:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2d180a22c0 msgr2=0x7f2d180a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2d180a22c0 0x7f2d180a4780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c005c40 tx=0x7f2d1c009d70 comp rx=0 tx=0).stop 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0ff9b0 msgr2=0x7f2d2c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0ff9b0 0x7f2d2c19d600 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e9f0 tx=0x7f2d2400eeb0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 shutdown_connections 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2d180a22c0 0x7f2d180a4780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c1976c0 0x7f2d2c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d2c106500 0x7f2d2c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 --2- 172.21.15.17:0/3961183624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0ff9b0 0x7f2d2c19d600 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 >> 172.21.15.17:0/3961183624 conn(0x7f2d2c076630 msgr2=0x7f2d2c104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 shutdown_connections 2024-02-14T15:03:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:47.997+0000 7f2d30dc9640 1 -- 172.21.15.17:0/3961183624 wait complete. 2024-02-14T15:03:48.007 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:47 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3288582592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:48.008 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:03:48.008 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 -- 172.21.15.17:0/2193009506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058105430 msgr2=0x7fc05810d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 --2- 172.21.15.17:0/2193009506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058105430 0x7fc05810d000 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc050009cd0 tx=0x7fc05002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 -- 172.21.15.17:0/2193009506 shutdown_connections 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 --2- 172.21.15.17:0/2193009506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058105430 0x7fc05810d000 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 --2- 172.21.15.17:0/2193009506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc058100d10 0x7fc058104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 --2- 172.21.15.17:0/2193009506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc058100360 0x7fc058100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 -- 172.21.15.17:0/2193009506 >> 172.21.15.17:0/2193009506 conn(0x7fc0580772a0 msgr2=0x7fc0580776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 -- 172.21.15.17:0/2193009506 shutdown_connections 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 -- 172.21.15.17:0/2193009506 wait complete. 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 Processor -- start 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.112+0000 7fc05dafa640 1 -- start start 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc05dafa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058100360 0x7fc05819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc05dafa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc058105430 0x7fc05819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc05dafa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc058197630 0x7fc058197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc05dafa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058198100 con 0x7fc058100360 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc05dafa640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058198270 con 0x7fc058105430 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc05dafa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058198430 con 0x7fc058197630 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058100360 0x7fc05819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058100360 0x7fc05819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59430/0 (socket says 172.21.15.17:59430) 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 -- 172.21.15.17:0/1721954257 learned_addr learned my addr 172.21.15.17:0/1721954257 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc04effd640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc058105430 0x7fc05819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc057fff640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc058197630 0x7fc058197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 -- 172.21.15.17:0/1721954257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc058197630 msgr2=0x7fc058197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc058197630 0x7fc058197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 -- 172.21.15.17:0/1721954257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc058105430 msgr2=0x7fc05819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc058105430 0x7fc05819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 -- 172.21.15.17:0/1721954257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc050009900 con 0x7fc058100360 2024-02-14T15:03:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0577fe640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058100360 0x7fc05819d570 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc04800d950 tx=0x7fc04800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.113+0000 7fc0557fa640 1 -- 172.21.15.17:0/1721954257 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048009c30 con 0x7fc058100360 2024-02-14T15:03:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.114+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0581a43d0 con 0x7fc058100360 2024-02-14T15:03:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.114+0000 7fc0557fa640 1 -- 172.21.15.17:0/1721954257 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc048009d90 con 0x7fc058100360 2024-02-14T15:03:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.114+0000 7fc0557fa640 1 -- 172.21.15.17:0/1721954257 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800f7e0 con 0x7fc058100360 2024-02-14T15:03:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.114+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0581a4900 con 0x7fc058100360 2024-02-14T15:03:48.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc0557fa640 1 -- 172.21.15.17:0/1721954257 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fc048010ad0 con 0x7fc058100360 2024-02-14T15:03:48.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc0557fa640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc0440a22c0 0x7fc0440a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 Processor -- start 2024-02-14T15:03:48.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 -- start start 2024-02-14T15:03:48.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc0557fa640 1 -- 172.21.15.17:0/1721954257 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc0480ce170 con 0x7fc058100360 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc04effd640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc0440a22c0 0x7fc0440a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8dde3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde4570 0x55dd8dded0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddedad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8dd42c80 con 0x55dd8dded670 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8dd52320 con 0x55dd8dde3bc0 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.116+0000 7fc06079c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8dd4f850 con 0x55dd8dde4570 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddedad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddedad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59432/0 (socket says 172.21.15.17:59432) 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 -- 172.21.15.17:0/2902266828 learned_addr learned my addr 172.21.15.17:0/2902266828 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc04effd640 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde4570 0x55dd8dded0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc04effd640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc0440a22c0 0x7fc0440a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc058198e80 tx=0x7fc04000b390 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc0577fe640 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8dde3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 -- 172.21.15.17:0/2902266828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde4570 msgr2=0x55dd8dded0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde4570 0x55dd8dded0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 -- 172.21.15.17:0/2902266828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 msgr2=0x55dd8dde3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8dde3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 -- 172.21.15.17:0/2902266828 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd8dd11e30 con 0x55dd8dded670 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc057fff640 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddedad0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc05002c700 tx=0x7fc050032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c3bc107dd396ee server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc04dffb640 1 -- 172.21.15.17:0/2902266828 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050034440 con 0x55dd8dded670 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc04dffb640 1 -- 172.21.15.17:0/2902266828 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0500345a0 con 0x55dd8dded670 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.117+0000 7fc04dffb640 1 -- 172.21.15.17:0/2902266828 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050003bb0 con 0x55dd8dded670 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 -- 172.21.15.17:0/2902266828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 msgr2=0x55dd8ddedad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddedad0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc05002c700 tx=0x7fc050032d70 comp rx=0 tx=0).stop 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 -- 172.21.15.17:0/2902266828 shutdown_connections 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddedad0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde4570 0x55dd8dded0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 --2- 172.21.15.17:0/2902266828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8dde3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 -- 172.21.15.17:0/2902266828 >> 172.21.15.17:0/2902266828 conn(0x55dd8ddde3b0 msgr2=0x55dd8ddde790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 -- 172.21.15.17:0/2902266828 shutdown_connections 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 -- 172.21.15.17:0/2902266828 wait complete. 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 -- start start 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.118+0000 7fc06079c740 1 Processor -- start 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde4570 0x55dd8ddf15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8ddf1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8dd4b030 con 0x55dd8dde4570 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8da1e430 con 0x55dd8dde3bc0 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8d9dae00 con 0x55dd8dded670 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 10 client.0 _ll_register_callbacks cb 0x55dd8da5bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde4570 0x55dd8ddf15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde4570 0x55dd8ddf15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59448/0 (socket says 172.21.15.17:59448) 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 -- 172.21.15.17:0/811018302 learned_addr learned my addr 172.21.15.17:0/811018302 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc057fff640 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8ddf1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc0577fe640 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 -- 172.21.15.17:0/811018302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 msgr2=0x55dd8ddf1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8ddf1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 -- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 msgr2=0x55dd8ddf10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 -- 172.21.15.17:0/811018302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0580698d0 con 0x55dd8dde4570 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc04effd640 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde4570 0x55dd8ddf15f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc040003e80 tx=0x7fc040004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e11b46f6073e949 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc036ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc036ffd640 1 -- 172.21.15.17:0/811018302 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04001f840 con 0x55dd8dde4570 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc036ffd640 1 -- 172.21.15.17:0/811018302 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc040017070 con 0x55dd8dde4570 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.119+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd8da4a7e0 con 0x55dd8dde4570 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc036ffd640 1 -- 172.21.15.17:0/811018302 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc040011e50 con 0x55dd8dde4570 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 10 client.15849 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dd8dd496b0 con 0x55dd8dde4570 2024-02-14T15:03:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc036ffd640 1 -- 172.21.15.17:0/811018302 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dd8da4a7e0 con 0x55dd8dde4570 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc036ffd640 1 -- 172.21.15.17:0/811018302 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc040028c60 con 0x55dd8dde4570 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 10 client.15849 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 20 client.15849 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de017b0 0x55dd8de03bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 4 client.15849 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd8da87720 con 0x55dd8de017b0 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.120+0000 7fc0577fe640 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de017b0 0x55dd8de03bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.121+0000 7fc0577fe640 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de017b0 0x55dd8de03bd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44fdf94e67fe47e9 server_cookie=563f379747b9f75c in_seq=0 out_seq=0 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.121+0000 7fc036ffd640 10 client.15849 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc036ffd640 1 -- 172.21.15.17:0/811018302 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd8da87720 con 0x55dd8de017b0 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc036ffd640 10 client.15849 handle_command_reply: tid=0 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 client.15849 shutdown 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde4570 msgr2=0x55dd8ddf15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde4570 0x55dd8ddf15f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc040003e80 tx=0x7fc040004730 comp rx=0 tx=0).stop 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 shutdown_connections 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de017b0 0x55dd8de03bd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde4570 0x55dd8ddf15f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc036ffd640 0 client.15849 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8ddf1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 --2- 172.21.15.17:0/811018302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 >> 172.21.15.17:0/811018302 conn(0x55dd8ddde3b0 msgr2=0x55dd8dde0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 shutdown_connections 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 1 -- 172.21.15.17:0/811018302 wait complete. 2024-02-14T15:03:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.122+0000 7fc06079c740 20 client.15849 trim_cache size 0 max 16384 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.133+0000 7fc06079c740 1 Processor -- start 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.133+0000 7fc06079c740 1 -- start start 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc06079c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc06079c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddeea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc06079c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddf00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc06079c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8dcd0130 con 0x55dd8dded670 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc06079c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8ddeb260 con 0x55dd8dde4570 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc06079c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8dde3040 con 0x55dd8dde3bc0 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddf00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddf00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59464/0 (socket says 172.21.15.17:59464) 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 -- 172.21.15.17:0/1359729607 learned_addr learned my addr 172.21.15.17:0/1359729607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc0577fe640 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc04effd640 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddeea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 -- 172.21.15.17:0/1359729607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde3bc0 msgr2=0x55dd8ddf3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 -- 172.21.15.17:0/1359729607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 msgr2=0x55dd8ddeea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddeea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 -- 172.21.15.17:0/1359729607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd8dce5c90 con 0x55dd8dded670 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc057fff640 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddf00d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc05002c700 tx=0x7fc05002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5e3a4bc11332b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc04d7fa640 1 -- 172.21.15.17:0/1359729607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050032340 con 0x55dd8dded670 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc04d7fa640 1 -- 172.21.15.17:0/1359729607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0500324a0 con 0x55dd8dded670 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.134+0000 7fc04d7fa640 1 -- 172.21.15.17:0/1359729607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050003cf0 con 0x55dd8dded670 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 -- 172.21.15.17:0/1359729607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 msgr2=0x55dd8ddf00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddf00d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc05002c700 tx=0x7fc05002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 -- 172.21.15.17:0/1359729607 shutdown_connections 2024-02-14T15:03:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dded670 0x55dd8ddf00d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddeea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 --2- 172.21.15.17:0/1359729607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 -- 172.21.15.17:0/1359729607 >> 172.21.15.17:0/1359729607 conn(0x55dd8ddde3b0 msgr2=0x55dd8ddef9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 -- 172.21.15.17:0/1359729607 shutdown_connections 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 -- 172.21.15.17:0/1359729607 wait complete. 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 -- start start 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.135+0000 7fc06079c740 1 Processor -- start 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddf7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8de57ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8ddf80d0 con 0x55dd8dde3bc0 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8ddf8240 con 0x55dd8dde4570 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd8de58220 con 0x55dd8dded670 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 10 client.0 _ll_register_callbacks cb 0x7fc050037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59476/0 (socket says 172.21.15.17:59476) 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 -- 172.21.15.17:0/610208046 learned_addr learned my addr 172.21.15.17:0/610208046 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc04effd640 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddf7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc057fff640 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8de57ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 -- 172.21.15.17:0/610208046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 msgr2=0x55dd8de57ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8de57ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 -- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 msgr2=0x55dd8ddf7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddf7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 -- 172.21.15.17:0/610208046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd8dd11e30 con 0x55dd8dde3bc0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc0577fe640 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf7430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc048097110 tx=0x7fc048010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752744a1757a3b99 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc036ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc036ffd640 1 -- 172.21.15.17:0/610208046 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0480a2e30 con 0x55dd8dde3bc0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc036ffd640 1 -- 172.21.15.17:0/610208046 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc048098a40 con 0x55dd8dde3bc0 2024-02-14T15:03:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.136+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0500345a0 con 0x55dd8dde3bc0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc036ffd640 1 -- 172.21.15.17:0/610208046 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc048096db0 con 0x55dd8dde3bc0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 10 client.15855 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dd8dd0baa0 con 0x55dd8dde3bc0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc036ffd640 1 -- 172.21.15.17:0/610208046 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0500345a0 con 0x55dd8dde3bc0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc036ffd640 1 -- 172.21.15.17:0/610208046 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc048096040 con 0x55dd8dde3bc0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 10 client.15855 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 20 client.15855 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de598a0 0x55dd8de5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 4 client.15855 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd8dad6890 con 0x55dd8de598a0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc04effd640 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de598a0 0x55dd8de5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc04effd640 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de598a0 0x55dd8de5bcc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64361bec27ef8541 server_cookie=a4d6efa4a2c52953 in_seq=0 out_seq=0 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.137+0000 7fc036ffd640 10 client.15855 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc036ffd640 1 -- 172.21.15.17:0/610208046 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd8dad6890 con 0x55dd8de598a0 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc036ffd640 10 client.15855 handle_command_reply: tid=0 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 client.15855 shutdown 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde3bc0 msgr2=0x55dd8ddf7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf7430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc048097110 tx=0x7fc048010e70 comp rx=0 tx=0).stop 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 shutdown_connections 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55dd8de598a0 0x55dd8de5bcc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd8dded670 0x55dd8de57ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd8dde4570 0x55dd8ddf7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc036ffd640 0 client.15855 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 --2- 172.21.15.17:0/610208046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dd8dde3bc0 0x55dd8ddf7430 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 >> 172.21.15.17:0/610208046 conn(0x55dd8ddde3b0 msgr2=0x55dd8dddcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 shutdown_connections 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 1 -- 172.21.15.17:0/610208046 wait complete. 2024-02-14T15:03:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.138+0000 7fc06079c740 20 client.15855 trim_cache size 0 max 16384 2024-02-14T15:03:48.146 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:03:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc0440a22c0 msgr2=0x7fc0440a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc0440a22c0 0x7fc0440a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc058198e80 tx=0x7fc04000b390 comp rx=0 tx=0).stop 2024-02-14T15:03:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058100360 msgr2=0x7fc05819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058100360 0x7fc05819d570 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc04800d950 tx=0x7fc04800de10 comp rx=0 tx=0).stop 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 shutdown_connections 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc0440a22c0 0x7fc0440a4780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc058197630 0x7fc058197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc058105430 0x7fc05819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 --2- 172.21.15.17:0/1721954257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc058100360 0x7fc05819d570 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 >> 172.21.15.17:0/1721954257 conn(0x7fc0580772a0 msgr2=0x7fc0580fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 shutdown_connections 2024-02-14T15:03:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:48.139+0000 7fc05dafa640 1 -- 172.21.15.17:0/1721954257 wait complete. 2024-02-14T15:03:48.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:47 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3288582592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:48.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:47 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3288582592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:03:49.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:48 smithi017 ceph-mon[32549]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-02-14T15:03:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:48 smithi050 ceph-mon[36512]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-02-14T15:03:49.151 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-14T15:03:49.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:03:49.151 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-14T15:03:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:48 smithi076 ceph-mon[37057]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-02-14T15:03:49.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.268+0000 7fdc5106b640 1 -- 172.21.15.17:0/4209743278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 msgr2=0x7fdc4c103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.268+0000 7fdc5106b640 1 --2- 172.21.15.17:0/4209743278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c103260 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009930 tx=0x7fdc3402bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 -- 172.21.15.17:0/4209743278 shutdown_connections 2024-02-14T15:03:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 --2- 172.21.15.17:0/4209743278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c103260 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 --2- 172.21.15.17:0/4209743278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4c1037a0 0x7fdc4c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 --2- 172.21.15.17:0/4209743278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc4c101520 0x7fdc4c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 -- 172.21.15.17:0/4209743278 >> 172.21.15.17:0/4209743278 conn(0x7fdc4c0fccc0 msgr2=0x7fdc4c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 -- 172.21.15.17:0/4209743278 shutdown_connections 2024-02-14T15:03:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 -- 172.21.15.17:0/4209743278 wait complete. 2024-02-14T15:03:49.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.269+0000 7fdc5106b640 1 Processor -- start 2024-02-14T15:03:49.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc5106b640 1 -- start start 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc5106b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4c101520 0x7fdc4c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc5106b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc5106b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc4c1037a0 0x7fdc4c197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc5106b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c197ec0 con 0x7fdc4c102e00 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc5106b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c198030 con 0x7fdc4c101520 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc5106b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1981f0 con 0x7fdc4c1037a0 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c19dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59488/0 (socket says 172.21.15.17:59488) 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1952371982 learned_addr learned my addr 172.21.15.17:0/1952371982 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4c101520 0x7fdc4c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4b59e640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc4c1037a0 0x7fdc4c197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1952371982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc4c1037a0 msgr2=0x7fdc4c197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc4c1037a0 0x7fdc4c197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1952371982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4c101520 msgr2=0x7fdc4c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4c101520 0x7fdc4c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1952371982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34009560 con 0x7fdc4c102e00 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.270+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c19dbb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009230 tx=0x7fdc3402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.271+0000 7fdc43fff640 1 -- 172.21.15.17:0/1952371982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34004350 con 0x7fdc4c102e00 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.271+0000 7fdc43fff640 1 -- 172.21.15.17:0/1952371982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc340044b0 con 0x7fdc4c102e00 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.271+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c198470 con 0x7fdc4c102e00 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.271+0000 7fdc43fff640 1 -- 172.21.15.17:0/1952371982 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34003d00 con 0x7fdc4c102e00 2024-02-14T15:03:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.271+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c1a45f0 con 0x7fdc4c102e00 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 Processor -- start 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc43fff640 1 -- 172.21.15.17:0/1952371982 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fdc34004df0 con 0x7fdc4c102e00 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 -- start start 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d94c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94ca10 0x56068d9550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d9555f0 0x56068d9579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d907bd0 con 0x56068d9555f0 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d8ba030 con 0x56068d94ca10 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d8b7560 con 0x56068d94c0f0 2024-02-14T15:03:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d9555f0 0x56068d9579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d9555f0 0x56068d9579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59496/0 (socket says 172.21.15.17:59496) 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3293205770 learned_addr learned my addr 172.21.15.17:0/3293205770 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94ca10 0x56068d9550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.274+0000 7fdc4b59e640 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d94c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc43fff640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fdc380a2340 0x7fdc380a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3293205770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 msgr2=0x56068d94c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d94c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3293205770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94ca10 msgr2=0x56068d9550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94ca10 0x56068d9550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3293205770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56068d875290 con 0x56068d9555f0 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4b59e640 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d94c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4b59e640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fdc380a2340 0x7fdc380a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc43fff640 1 -- 172.21.15.17:0/1952371982 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fdc340f0450 con 0x7fdc4c102e00 2024-02-14T15:03:49.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d9555f0 0x56068d9579e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdc340ed830 tx=0x7fdc340b2150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b66178e29bd340 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc417fa640 1 -- 172.21.15.17:0/3293205770 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340bc5c0 con 0x56068d9555f0 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc4b59e640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fdc380a2340 0x7fdc380a4800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c005fd0 tx=0x7fdc3c004a40 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc417fa640 1 -- 172.21.15.17:0/3293205770 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc340bc3e0 con 0x56068d9555f0 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.275+0000 7fdc417fa640 1 -- 172.21.15.17:0/3293205770 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340035f0 con 0x56068d9555f0 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3293205770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d9555f0 msgr2=0x56068d9579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d9555f0 0x56068d9579e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdc340ed830 tx=0x7fdc340b2150 comp rx=0 tx=0).stop 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3293205770 shutdown_connections 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d9555f0 0x56068d9579e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94ca10 0x56068d9550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3293205770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d94c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3293205770 >> 172.21.15.17:0/3293205770 conn(0x56068d946820 msgr2=0x56068d946c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3293205770 shutdown_connections 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3293205770 wait complete. 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.276+0000 7fdc53c3c740 1 -- start start 2024-02-14T15:03:49.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 1 Processor -- start 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d952520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d952fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d8b2d40 con 0x56068d94ca10 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d857e60 con 0x56068d94c0f0 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d5864b0 con 0x56068d9555f0 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc53c3c740 10 client.0 _ll_register_callbacks cb 0x56068d86e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59512/0 (socket says 172.21.15.17:59512) 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1301708466 learned_addr learned my addr 172.21.15.17:0/1301708466 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d952520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4b59e640 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d952fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1301708466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 msgr2=0x56068d952fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d952fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 msgr2=0x56068d952520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d952520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1301708466 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c1a45f0 con 0x56068d94ca10 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d952520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952a60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b9450 tx=0x7fdc340037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93164c986da69000 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.277+0000 7fdc327fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc327fc640 1 -- 172.21.15.17:0/1301708466 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340035f0 con 0x56068d94ca10 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc327fc640 1 -- 172.21.15.17:0/1301708466 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fdc340bd180 con 0x56068d94ca10 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56068d8d8dd0 con 0x56068d94ca10 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc327fc640 1 -- 172.21.15.17:0/1301708466 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc340bca80 con 0x56068d94ca10 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 10 client.15867 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56068d8b13c0 con 0x56068d94ca10 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc327fc640 1 -- 172.21.15.17:0/1301708466 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56068d8d8dd0 con 0x56068d94ca10 2024-02-14T15:03:49.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc327fc640 1 -- 172.21.15.17:0/1301708466 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc3403c070 con 0x56068d94ca10 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 10 client.15867 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 20 client.15867 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d964e60 0x56068d967280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 4 client.15867 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56068d885cb0 con 0x56068d964e60 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.278+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d964e60 0x56068d967280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.279+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d964e60 0x56068d967280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab7d6b20c01e4fb server_cookie=613799738d9a21da in_seq=0 out_seq=0 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.279+0000 7fdc327fc640 10 client.15867 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.279+0000 7fdc327fc640 1 -- 172.21.15.17:0/1301708466 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56068d885cb0 con 0x56068d964e60 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.279+0000 7fdc327fc640 10 client.15867 handle_command_reply: tid=0 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.279+0000 7fdc53c3c740 1 client.15867 shutdown 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 msgr2=0x56068d952a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952a60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b9450 tx=0x7fdc340037d0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 shutdown_connections 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d964e60 0x56068d967280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952a60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc327fc640 0 client.15867 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d952fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1301708466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d952520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 >> 172.21.15.17:0/1301708466 conn(0x56068d946820 msgr2=0x56068d948570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 shutdown_connections 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1301708466 wait complete. 2024-02-14T15:03:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.280+0000 7fdc53c3c740 20 client.15867 trim_cache size 0 max 16384 2024-02-14T15:03:49.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 Processor -- start 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 -- start start 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d9516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d9555f0 0x56068d945900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d838670 con 0x56068d94ca10 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d885ce0 con 0x56068d9555f0 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d951be0 con 0x56068d94c0f0 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.295+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59520/0 (socket says 172.21.15.17:59520) 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1907204458 learned_addr learned my addr 172.21.15.17:0/1907204458 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1907204458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 msgr2=0x56068d9516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1907204458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d9516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1907204458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d9555f0 msgr2=0x56068d945900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1907204458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d9555f0 0x56068d945900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 -- 172.21.15.17:0/1907204458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56068d84dc70 con 0x56068d94ca10 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/1907204458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952dc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdc340bc550 tx=0x7fdc3402c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22857314c4776822 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc40ff9640 1 -- 172.21.15.17:0/1907204458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340bc5c0 con 0x56068d94ca10 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc40ff9640 1 -- 172.21.15.17:0/1907204458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc34003e60 con 0x56068d94ca10 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.296+0000 7fdc40ff9640 1 -- 172.21.15.17:0/1907204458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340c0bd0 con 0x56068d94ca10 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1907204458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 msgr2=0x56068d952dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1907204458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952dc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdc340bc550 tx=0x7fdc3402c450 comp rx=0 tx=0).stop 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1907204458 shutdown_connections 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1907204458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d952dc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1907204458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d9555f0 0x56068d945900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/1907204458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d94c0f0 0x56068d9516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1907204458 >> 172.21.15.17:0/1907204458 conn(0x56068d946820 msgr2=0x56068d9451f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1907204458 shutdown_connections 2024-02-14T15:03:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- 172.21.15.17:0/1907204458 wait complete. 2024-02-14T15:03:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- start start 2024-02-14T15:03:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 Processor -- start 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d95aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d95b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d95b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d95cb40 con 0x56068d94ca10 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.297+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d9bb330 con 0x56068d94c0f0 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc53c3c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068d9bb470 con 0x56068d9555f0 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc53c3c740 10 client.0 _ll_register_callbacks cb 0x7fdc44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d95b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d95b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59524/0 (socket says 172.21.15.17:59524) 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3676658984 learned_addr learned my addr 172.21.15.17:0/3676658984 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d95aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4b59e640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d95b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3676658984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 msgr2=0x56068d95b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d95b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 msgr2=0x56068d95aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d95aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 -- 172.21.15.17:0/3676658984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56068d8b13c0 con 0x56068d94ca10 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d95aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc4a59c640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d95b0e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b8ee0 tx=0x7fdc34039fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41edf91dd5822bf server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc327fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc327fc640 1 -- 172.21.15.17:0/3676658984 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340bc5c0 con 0x56068d94ca10 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc327fc640 1 -- 172.21.15.17:0/3676658984 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fdc340bca80 con 0x56068d94ca10 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc340bc3e0 con 0x56068d94ca10 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc327fc640 1 -- 172.21.15.17:0/3676658984 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc340035f0 con 0x56068d94ca10 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc53c3c740 10 client.15873 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.298+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56068d873af0 con 0x56068d94ca10 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc327fc640 1 -- 172.21.15.17:0/3676658984 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc340bc3e0 con 0x56068d94ca10 2024-02-14T15:03:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc327fc640 1 -- 172.21.15.17:0/3676658984 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc3403c070 con 0x56068d94ca10 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc53c3c740 10 client.15873 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc53c3c740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc53c3c740 20 client.15873 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d9bc770 0x56068d9beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc53c3c740 4 client.15873 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56068d5b2860 con 0x56068d9bc770 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d9bc770 0x56068d9beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc4ad9d640 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d9bc770 0x56068d9beb60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906864dca9cc5ce8 server_cookie=400b90e1ee7c8df9 in_seq=0 out_seq=0 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.299+0000 7fdc327fc640 10 client.15873 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.300+0000 7fdc327fc640 1 -- 172.21.15.17:0/3676658984 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56068d5b2860 con 0x56068d9bc770 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.300+0000 7fdc327fc640 10 client.15873 handle_command_reply: tid=0 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.300+0000 7fdc53c3c740 1 client.15873 shutdown 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 msgr2=0x56068d95b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d95b0e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b8ee0 tx=0x7fdc34039fd0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 shutdown_connections 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56068d9bc770 0x56068d9beb60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56068d94ca10 0x56068d95b0e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56068d9555f0 0x56068d95b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 --2- 172.21.15.17:0/3676658984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56068d94c0f0 0x56068d95aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 >> 172.21.15.17:0/3676658984 conn(0x56068d946820 msgr2=0x56068d945a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc327fc640 0 client.15873 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 shutdown_connections 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 1 -- 172.21.15.17:0/3676658984 wait complete. 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc53c3c740 20 client.15873 trim_cache size 0 max 16384 2024-02-14T15:03:49.309 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_tag": "8d8f27a5-f6e2-4efc-9da0-46a119b8f827", 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": "asynchronous" 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fdc380a2340 msgr2=0x7fdc380a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fdc380a2340 0x7fdc380a4800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c005fd0 tx=0x7fdc3c004a40 comp rx=0 tx=0).stop 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 msgr2=0x7fdc4c19dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c19dbb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009230 tx=0x7fdc3402c650 comp rx=0 tx=0).stop 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 shutdown_connections 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fdc380a2340 0x7fdc380a4800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4c102e00 0x7fdc4c19dbb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc4c1037a0 0x7fdc4c197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.301+0000 7fdc5106b640 1 --2- 172.21.15.17:0/1952371982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdc4c101520 0x7fdc4c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.302+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 >> 172.21.15.17:0/1952371982 conn(0x7fdc4c0fccc0 msgr2=0x7fdc4c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.302+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 shutdown_connections 2024-02-14T15:03:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.302+0000 7fdc5106b640 1 -- 172.21.15.17:0/1952371982 wait complete. 2024-02-14T15:03:49.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:03:49.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/148725862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100ce0 msgr2=0x7fa9a4104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/148725862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100ce0 0x7fa9a4104d10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009930 tx=0x7fa98c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/148725862 shutdown_connections 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/148725862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100ce0 0x7fa9a4104d10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/148725862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9a41053f0 0x7fa9a410cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/148725862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9a4100330 0x7fa9a4100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/148725862 >> 172.21.15.17:0/148725862 conn(0x7fa9a4077260 msgr2=0x7fa9a4077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/148725862 shutdown_connections 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.418+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/148725862 wait complete. 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 Processor -- start 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 -- start start 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100330 0x7fa9a4101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9a41053f0 0x7fa9a41022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9a4102830 0x7fa9a41a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41a6030 con 0x7fa9a4100330 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41a61a0 con 0x7fa9a41053f0 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a8d7a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41a6360 con 0x7fa9a4102830 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100330 0x7fa9a4101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100330 0x7fa9a4101db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59544/0 (socket says 172.21.15.17:59544) 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a259c640 1 -- 172.21.15.17:0/2881090094 learned_addr learned my addr 172.21.15.17:0/2881090094 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa99bfff640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9a41053f0 0x7fa9a41022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a2d9d640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9a4102830 0x7fa9a41a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a259c640 1 -- 172.21.15.17:0/2881090094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9a4102830 msgr2=0x7fa9a41a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a259c640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9a4102830 0x7fa9a41a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a259c640 1 -- 172.21.15.17:0/2881090094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9a41053f0 msgr2=0x7fa9a41022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.419+0000 7fa9a259c640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9a41053f0 0x7fa9a41022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa9a259c640 1 -- 172.21.15.17:0/2881090094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009560 con 0x7fa9a4100330 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa9a2d9d640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9a4102830 0x7fa9a41a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa9a259c640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100330 0x7fa9a4101db0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d770 tx=0x7fa99400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa99b7fe640 1 -- 172.21.15.17:0/2881090094 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994009c30 con 0x7fa9a4100330 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa99b7fe640 1 -- 172.21.15.17:0/2881090094 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa99400de80 con 0x7fa9a4100330 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa99b7fe640 1 -- 172.21.15.17:0/2881090094 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994004e80 con 0x7fa9a4100330 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a41a6640 con 0x7fa9a4100330 2024-02-14T15:03:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.420+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a41a6ad0 con 0x7fa9a4100330 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.422+0000 7fa99b7fe640 1 -- 172.21.15.17:0/2881090094 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fa994005020 con 0x7fa9a4100330 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.422+0000 7fa9aba26740 1 Processor -- start 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9aba26740 1 -- start start 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e3003ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003ef50 0x556e300475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e30047b30 0x556e30049f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99b7fe640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa9900a22f0 0x7fa9900a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9aba26740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2fff95b0 con 0x556e30047b30 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9aba26740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2ffad2b0 con 0x556e3003ef50 2024-02-14T15:03:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9aba26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2ffaa7e0 con 0x556e3003e630 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e30047b30 0x556e30049f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e30047b30 0x556e30049f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59556/0 (socket says 172.21.15.17:59556) 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99bfff640 1 -- 172.21.15.17:0/102993837 learned_addr learned my addr 172.21.15.17:0/102993837 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9a2d9d640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa9900a22f0 0x7fa9900a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9a259c640 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003ef50 0x556e300475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa9a2d9d640 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e3003ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99b7fe640 1 -- 172.21.15.17:0/2881090094 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa9940cdfe0 con 0x7fa9a4100330 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99bfff640 1 -- 172.21.15.17:0/102993837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 msgr2=0x556e3003ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99bfff640 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e3003ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99bfff640 1 -- 172.21.15.17:0/102993837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003ef50 msgr2=0x556e300475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.423+0000 7fa99bfff640 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003ef50 0x556e300475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.424+0000 7fa99bfff640 1 -- 172.21.15.17:0/102993837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e2ff68020 con 0x556e30047b30 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.424+0000 7fa9a2d9d640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa9900a22f0 0x7fa9900a47b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa99c005fd0 tx=0x7fa99c009e40 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.424+0000 7fa99bfff640 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e30047b30 0x556e30049f20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa98c005ab0 tx=0x7fa98c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70548a51bb396b86 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.424+0000 7fa998ff9640 1 -- 172.21.15.17:0/102993837 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c004880 con 0x556e30047b30 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.424+0000 7fa998ff9640 1 -- 172.21.15.17:0/102993837 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa98c0049e0 con 0x556e30047b30 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.424+0000 7fa998ff9640 1 -- 172.21.15.17:0/102993837 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c005010 con 0x556e30047b30 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- 172.21.15.17:0/102993837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e30047b30 msgr2=0x556e30049f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e30047b30 0x556e30049f20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa98c005ab0 tx=0x7fa98c02c680 comp rx=0 tx=0).stop 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- 172.21.15.17:0/102993837 shutdown_connections 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e30047b30 0x556e30049f20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003ef50 0x556e300475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 --2- 172.21.15.17:0/102993837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e3003ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- 172.21.15.17:0/102993837 >> 172.21.15.17:0/102993837 conn(0x556e30038d60 msgr2=0x556e30039140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- 172.21.15.17:0/102993837 shutdown_connections 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- 172.21.15.17:0/102993837 wait complete. 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- start start 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 Processor -- start 2024-02-14T15:03:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e30044af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30045030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 0x556e30045570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2ffa5fc0 con 0x556e3003ef50 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2fc79430 con 0x556e30047b30 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2fc35e00 con 0x556e3003e630 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa9aba26740 10 client.0 _ll_register_callbacks cb 0x556e2ff27870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30045030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.425+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30045030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59570/0 (socket says 172.21.15.17:59570) 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa99bfff640 1 -- 172.21.15.17:0/1218951558 learned_addr learned my addr 172.21.15.17:0/1218951558 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa9a259c640 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e30044af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa99bfff640 1 -- 172.21.15.17:0/1218951558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 msgr2=0x556e30044af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa9a2d9d640 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 0x556e30045570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa99bfff640 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e30044af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa99bfff640 1 -- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 msgr2=0x556e30045570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa99bfff640 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 0x556e30045570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa99bfff640 1 -- 172.21.15.17:0/1218951558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a410d7d0 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa99bfff640 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30045030 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa98c036040 tx=0x7fa98c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae240048428faca0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa989ffb640 1 -- 172.21.15.17:0/1218951558 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c004880 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa989ffb640 1 -- 172.21.15.17:0/1218951558 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa98c04d070 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e2fca57e0 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa989ffb640 1 -- 172.21.15.17:0/1218951558 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa98c005200 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa9aba26740 10 client.15885 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556e2ffa4640 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.426+0000 7fa989ffb640 1 -- 172.21.15.17:0/1218951558 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556e2fca57e0 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa989ffb640 1 -- 172.21.15.17:0/1218951558 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa98c03c050 con 0x556e3003ef50 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9aba26740 10 client.15885 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9aba26740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9aba26740 20 client.15885 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e30057ae0 0x556e30059f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9aba26740 4 client.15885 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e2fce2720 con 0x556e30057ae0 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9a259c640 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e30057ae0 0x556e30059f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa9a259c640 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e30057ae0 0x556e30059f00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2d8b5b937cd8c0 server_cookie=222bbe86c3248cba in_seq=0 out_seq=0 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.427+0000 7fa989ffb640 10 client.15885 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa989ffb640 1 -- 172.21.15.17:0/1218951558 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e2fce2720 con 0x556e30057ae0 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa989ffb640 10 client.15885 handle_command_reply: tid=0 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 client.15885 shutdown 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 msgr2=0x556e30045030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30045030 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa98c036040 tx=0x7fa98c0058d0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 shutdown_connections 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e30057ae0 0x556e30059f00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30045030 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 0x556e30045570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1218951558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003e630 0x556e30044af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa989ffb640 0 client.15885 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 >> 172.21.15.17:0/1218951558 conn(0x556e30038d60 msgr2=0x556e3003aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 shutdown_connections 2024-02-14T15:03:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 1 -- 172.21.15.17:0/1218951558 wait complete. 2024-02-14T15:03:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.428+0000 7fa9aba26740 20 client.15885 trim_cache size 0 max 16384 2024-02-14T15:03:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.443+0000 7fa9aba26740 1 Processor -- start 2024-02-14T15:03:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.443+0000 7fa9aba26740 1 -- start start 2024-02-14T15:03:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003e630 0x556e30047220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30043ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e30047b30 0x556e30045390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa9aba26740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e300344d0 con 0x556e3003ef50 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa9aba26740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2ff2afd0 con 0x556e3003e630 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa9aba26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3003e290 con 0x556e30047b30 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30043ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30043ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59576/0 (socket says 172.21.15.17:59576) 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 -- 172.21.15.17:0/1932431514 learned_addr learned my addr 172.21.15.17:0/1932431514 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 -- 172.21.15.17:0/1932431514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e30047b30 msgr2=0x556e30045390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 --2- 172.21.15.17:0/1932431514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e30047b30 0x556e30045390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 -- 172.21.15.17:0/1932431514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003e630 msgr2=0x556e30047220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 --2- 172.21.15.17:0/1932431514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003e630 0x556e30047220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 -- 172.21.15.17:0/1932431514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e2ff40b30 con 0x556e3003ef50 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa99bfff640 1 --2- 172.21.15.17:0/1932431514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30043ad0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa98c005ab0 tx=0x7fa98c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4a26c53f46b4ac server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa98bfff640 1 -- 172.21.15.17:0/1932431514 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0046c0 con 0x556e3003ef50 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa98bfff640 1 -- 172.21.15.17:0/1932431514 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa98c004820 con 0x556e3003ef50 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.444+0000 7fa98bfff640 1 -- 172.21.15.17:0/1932431514 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0056f0 con 0x556e3003ef50 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- 172.21.15.17:0/1932431514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 msgr2=0x556e30043ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1932431514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30043ad0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa98c005ab0 tx=0x7fa98c005fd0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- 172.21.15.17:0/1932431514 shutdown_connections 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1932431514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003ef50 0x556e30043ad0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1932431514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e30047b30 0x556e30045390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 --2- 172.21.15.17:0/1932431514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e3003e630 0x556e30047220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- 172.21.15.17:0/1932431514 >> 172.21.15.17:0/1932431514 conn(0x556e30038d60 msgr2=0x556e300433c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- 172.21.15.17:0/1932431514 shutdown_connections 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- 172.21.15.17:0/1932431514 wait complete. 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- start start 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 Processor -- start 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003e630 0x556e3004e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003ef50 0x556e3004ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 0x556e3004d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3004f2a0 con 0x556e3003e630 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3004d930 con 0x556e30047b30 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3004daf0 con 0x556e3003ef50 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9aba26740 10 client.0 _ll_register_callbacks cb 0x7fa98c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003e630 0x556e3004e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003e630 0x556e3004e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59590/0 (socket says 172.21.15.17:59590) 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.445+0000 7fa9a259c640 1 -- 172.21.15.17:0/3149432104 learned_addr learned my addr 172.21.15.17:0/3149432104 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa99bfff640 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003ef50 0x556e3004ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9a259c640 1 -- 172.21.15.17:0/3149432104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003ef50 msgr2=0x556e3004ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9a259c640 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003ef50 0x556e3004ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9a259c640 1 -- 172.21.15.17:0/3149432104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 msgr2=0x556e3004d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9a259c640 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 0x556e3004d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9a259c640 1 -- 172.21.15.17:0/3149432104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e2ffa4640 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa99bfff640 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003ef50 0x556e3004ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9a259c640 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003e630 0x556e3004e9d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa99409a1e0 tx=0x7fa994017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2566d4d0eea12114 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa989ffb640 1 -- 172.21.15.17:0/3149432104 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99409a4e0 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa989ffb640 1 -- 172.21.15.17:0/3149432104 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa99409e630 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa98c0049e0 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa989ffb640 1 -- 172.21.15.17:0/3149432104 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9940a2c80 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 10 client.15891 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556e2ff66880 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa989ffb640 1 -- 172.21.15.17:0/3149432104 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa98c0049e0 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa989ffb640 1 -- 172.21.15.17:0/3149432104 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa99409e2d0 con 0x556e3003e630 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 10 client.15891 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 20 client.15891 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e300af290 0x556e300b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 4 client.15891 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.446+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556e2fd31890 con 0x556e300af290 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa99bfff640 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e300af290 0x556e300b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa989ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa99bfff640 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e300af290 0x556e300b16b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e910b958ef8350 server_cookie=77c5e26ce212fd9c in_seq=0 out_seq=0 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa989ffb640 1 -- 172.21.15.17:0/3149432104 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556e2fd31890 con 0x556e300af290 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa989ffb640 10 client.15891 handle_command_reply: tid=0 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 client.15891 shutdown 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003e630 msgr2=0x556e3004e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003e630 0x556e3004e9d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa99409a1e0 tx=0x7fa994017f90 comp rx=0 tx=0).stop 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 shutdown_connections 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556e300af290 0x556e300b16b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e30047b30 0x556e3004d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e3003ef50 0x556e3004ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 --2- 172.21.15.17:0/3149432104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556e3003e630 0x556e3004e9d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa989ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.447+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 >> 172.21.15.17:0/3149432104 conn(0x556e30038d60 msgr2=0x556e30037e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 shutdown_connections 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9aba26740 1 -- 172.21.15.17:0/3149432104 wait complete. 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9aba26740 20 client.15891 trim_cache size 0 max 16384 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": { 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: "8d8f27a5-f6e2-4efc-9da0-46a119b8f827": { 2024-02-14T15:03:49.455 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/", 2024-02-14T15:03:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: "tag": "8d8f27a5-f6e2-4efc-9da0-46a119b8f827", 2024-02-14T15:03:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: "options": "recursive,force" 2024-02-14T15:03:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:03:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:03:49.456 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa9900a22f0 msgr2=0x7fa9900a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa9900a22f0 0x7fa9900a47b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa99c005fd0 tx=0x7fa99c009e40 comp rx=0 tx=0).stop 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100330 msgr2=0x7fa9a4101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100330 0x7fa9a4101db0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d770 tx=0x7fa99400dc30 comp rx=0 tx=0).stop 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 shutdown_connections 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa9900a22f0 0x7fa9900a47b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9a4102830 0x7fa9a41a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9a41053f0 0x7fa9a41022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 --2- 172.21.15.17:0/2881090094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a4100330 0x7fa9a4101db0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 >> 172.21.15.17:0/2881090094 conn(0x7fa9a4077260 msgr2=0x7fa9a40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 shutdown_connections 2024-02-14T15:03:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:03:49.448+0000 7fa9a8d7a640 1 -- 172.21.15.17:0/2881090094 wait complete. 2024-02-14T15:03:49.459 INFO:tasks.cephfs.filesystem:scrub status for tag:8d8f27a5-f6e2-4efc-9da0-46a119b8f827 - {'path': '/', 'tag': '8d8f27a5-f6e2-4efc-9da0-46a119b8f827', 'options': 'recursive,force'} 2024-02-14T15:03:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:50 smithi017 ceph-mon[32549]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:50 smithi017 ceph-mon[32549]: scrub queued for path: / 2024-02-14T15:03:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:50 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:51.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:50 smithi017 ceph-mon[32549]: scrub summary: active paths [/] 2024-02-14T15:03:51.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:50 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:50 smithi050 ceph-mon[36512]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:50 smithi050 ceph-mon[36512]: scrub queued for path: / 2024-02-14T15:03:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:50 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:50 smithi050 ceph-mon[36512]: scrub summary: active paths [/] 2024-02-14T15:03:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:50 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:50 smithi076 ceph-mon[37057]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:50 smithi076 ceph-mon[37057]: scrub queued for path: / 2024-02-14T15:03:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:50 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:50 smithi076 ceph-mon[37057]: scrub summary: active paths [/] 2024-02-14T15:03:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:50 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:03:53.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:52 smithi017 ceph-mon[32549]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:53.036 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:52 smithi076 ceph-mon[37057]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:52 smithi050 ceph-mon[36512]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-02-14T15:03:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.513+0000 7f5341bc9640 1 -- 172.21.15.17:0/4276196106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c06a1e0 msgr2=0x7f533c10e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.513+0000 7f5341bc9640 1 --2- 172.21.15.17:0/4276196106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c06a1e0 0x7f533c10e0c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5334009cd0 tx=0x7f533402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 -- 172.21.15.17:0/4276196106 shutdown_connections 2024-02-14T15:03:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 --2- 172.21.15.17:0/4276196106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c06a1e0 0x7f533c10e0c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 --2- 172.21.15.17:0/4276196106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f533c069840 0x7f533c069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 --2- 172.21.15.17:0/4276196106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f533c107320 0x7f533c107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 -- 172.21.15.17:0/4276196106 >> 172.21.15.17:0/4276196106 conn(0x7f533c076f80 msgr2=0x7f533c077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 -- 172.21.15.17:0/4276196106 shutdown_connections 2024-02-14T15:03:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 -- 172.21.15.17:0/4276196106 wait complete. 2024-02-14T15:03:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 Processor -- start 2024-02-14T15:03:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.514+0000 7f5341bc9640 1 -- start start 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f5341bc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f533c069840 0x7f533c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f5341bc9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f533c06a1e0 0x7f533c19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f5341bc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c107320 0x7f533c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f5341bc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c199e70 con 0x7f533c107320 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f5341bc9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c199fe0 con 0x7f533c06a1e0 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f5341bc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c19a1a0 con 0x7f533c069840 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c107320 0x7f533c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c107320 0x7f533c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59958/0 (socket says 172.21.15.17:59958) 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f533c069840 0x7f533c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533bfff640 1 -- 172.21.15.17:0/2430130980 learned_addr learned my addr 172.21.15.17:0/2430130980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f533c06a1e0 0x7f533c19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533affd640 1 -- 172.21.15.17:0/2430130980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f533c069840 msgr2=0x7f533c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533affd640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f533c069840 0x7f533c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533affd640 1 -- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c107320 msgr2=0x7f533c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533affd640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c107320 0x7f533c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533affd640 1 -- 172.21.15.17:0/2430130980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334009900 con 0x7f533c06a1e0 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533bfff640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c107320 0x7f533c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.515+0000 7f533affd640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f533c06a1e0 0x7f533c19fba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f533000d6d0 tx=0x7f533000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.516+0000 7f5338ff9640 1 -- 172.21.15.17:0/2430130980 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330005220 con 0x7f533c06a1e0 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.516+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c19a480 con 0x7f533c06a1e0 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.516+0000 7f5338ff9640 1 -- 172.21.15.17:0/2430130980 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5330009d00 con 0x7f533c06a1e0 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.516+0000 7f5338ff9640 1 -- 172.21.15.17:0/2430130980 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330004de0 con 0x7f533c06a1e0 2024-02-14T15:03:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.516+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1a6690 con 0x7f533c06a1e0 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.518+0000 7f5338ff9640 1 -- 172.21.15.17:0/2430130980 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f5330009820 con 0x7f533c06a1e0 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.518+0000 7f534478a740 1 Processor -- start 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.518+0000 7f534478a740 1 -- start start 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.518+0000 7f534478a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.518+0000 7f534478a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f534478a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c23c20 0x557a68c26010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f534478a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68b889e0 con 0x557a68c23c20 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f534478a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68b85f10 con 0x557a68c1a720 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f534478a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68b816f0 con 0x557a68c1b040 2024-02-14T15:03:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c23c20 0x557a68c26010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c23c20 0x557a68c26010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59972/0 (socket says 172.21.15.17:59972) 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 -- 172.21.15.17:0/1552239041 learned_addr learned my addr 172.21.15.17:0/1552239041 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f5338ff9640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53240a22b0 0x7f53240a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533bfff640 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533b7fe640 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 -- 172.21.15.17:0/1552239041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 msgr2=0x557a68c236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 -- 172.21.15.17:0/1552239041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 msgr2=0x557a68c1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 -- 172.21.15.17:0/1552239041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a68892920 con 0x557a68c23c20 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533b7fe640 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533bfff640 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533bfff640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53240a22b0 0x7f53240a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f5338ff9640 1 -- 172.21.15.17:0/2430130980 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f53300ce090 con 0x7f533c06a1e0 2024-02-14T15:03:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f533affd640 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c23c20 0x557a68c26010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f53300cb360 tx=0x7f5330022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1af19f996eb85c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.519+0000 7f5321ffb640 1 -- 172.21.15.17:0/1552239041 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330005380 con 0x557a68c23c20 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f5321ffb640 1 -- 172.21.15.17:0/1552239041 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5330020ec0 con 0x557a68c23c20 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f5321ffb640 1 -- 172.21.15.17:0/1552239041 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330099fc0 con 0x557a68c23c20 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 -- 172.21.15.17:0/1552239041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c23c20 msgr2=0x557a68c26010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f533bfff640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53240a22b0 0x7f53240a4770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5334006010 tx=0x7f53340095c0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c23c20 0x557a68c26010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f53300cb360 tx=0x7f5330022000 comp rx=0 tx=0).stop 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 -- 172.21.15.17:0/1552239041 shutdown_connections 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c23c20 0x557a68c26010 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 --2- 172.21.15.17:0/1552239041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 -- 172.21.15.17:0/1552239041 >> 172.21.15.17:0/1552239041 conn(0x557a68c14e50 msgr2=0x557a68c15230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 -- 172.21.15.17:0/1552239041 shutdown_connections 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.520+0000 7f534478a740 1 -- 172.21.15.17:0/1552239041 wait complete. 2024-02-14T15:03:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 -- start start 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 Processor -- start 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1a720 0x557a68c235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c20940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c20ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68b1dfa0 con 0x557a68c1a720 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68855250 con 0x557a68c23c20 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68b07200 con 0x557a68c1b040 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f534478a740 10 client.0 _ll_register_callbacks cb 0x557a68b76b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f533b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1a720 0x557a68c235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f533b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1a720 0x557a68c235b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59984/0 (socket says 172.21.15.17:59984) 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f533b7fe640 1 -- 172.21.15.17:0/927065314 learned_addr learned my addr 172.21.15.17:0/927065314 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.521+0000 7f533affd640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c20940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533bfff640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c20ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533b7fe640 1 -- 172.21.15.17:0/927065314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 msgr2=0x557a68c20940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533b7fe640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c20940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533b7fe640 1 -- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 msgr2=0x557a68c20ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533b7fe640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c20ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533b7fe640 1 -- 172.21.15.17:0/927065314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1a6690 con 0x557a68c1a720 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533bfff640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c20ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f533b7fe640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1a720 0x557a68c235b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f532c012700 tx=0x7f532c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558bcd9a44a0dc82 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f531affd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f531affd640 1 -- 172.21.15.17:0/927065314 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c017070 con 0x557a68c1a720 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f531affd640 1 -- 172.21.15.17:0/927065314 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f532c004050 con 0x557a68c1a720 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a68881600 con 0x557a68c1a720 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f531affd640 1 -- 172.21.15.17:0/927065314 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f532c004210 con 0x557a68c1a720 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f534478a740 10 client.15903 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a68b7fd70 con 0x557a68c1a720 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.522+0000 7f531affd640 1 -- 172.21.15.17:0/927065314 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557a68881600 con 0x557a68c1a720 2024-02-14T15:03:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f531affd640 1 -- 172.21.15.17:0/927065314 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f532c003a60 con 0x557a68c1a720 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f534478a740 10 client.15903 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f534478a740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f534478a740 20 client.15903 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f534478a740 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c33990 0x557a68c35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f534478a740 4 client.15903 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a688d21b0 con 0x557a68c33990 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f533affd640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c33990 0x557a68c35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f533affd640 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c33990 0x557a68c35db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ca74db30537993 server_cookie=c314a2f8735f5c47 in_seq=0 out_seq=0 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.523+0000 7f531affd640 10 client.15903 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f531affd640 1 -- 172.21.15.17:0/927065314 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a688d21b0 con 0x557a68c33990 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f531affd640 10 client.15903 handle_command_reply: tid=0 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 client.15903 shutdown 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1a720 msgr2=0x557a68c235b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1a720 0x557a68c235b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f532c012700 tx=0x7f532c012bc0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 shutdown_connections 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c33990 0x557a68c35db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c20ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1b040 0x557a68c20940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 --2- 172.21.15.17:0/927065314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1a720 0x557a68c235b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f531affd640 0 client.15903 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 >> 172.21.15.17:0/927065314 conn(0x557a68c14e50 msgr2=0x557a68c16ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 shutdown_connections 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 1 -- 172.21.15.17:0/927065314 wait complete. 2024-02-14T15:03:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.524+0000 7f534478a740 20 client.15903 trim_cache size 0 max 16384 2024-02-14T15:03:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 Processor -- start 2024-02-14T15:03:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 -- start start 2024-02-14T15:03:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c23070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c23c20 0x557a68c201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68c10060 con 0x557a68c1b040 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68c3afa0 con 0x557a68c1a720 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f534478a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68c15fc0 con 0x557a68c23c20 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c235b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59986/0 (socket says 172.21.15.17:59986) 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 -- 172.21.15.17:0/3752035586 learned_addr learned my addr 172.21.15.17:0/3752035586 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 -- 172.21.15.17:0/3752035586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c23c20 msgr2=0x557a68c201d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c23c20 0x557a68c201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533b7fe640 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c23070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 -- 172.21.15.17:0/3752035586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 msgr2=0x557a68c23070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c23070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.538+0000 7f533affd640 1 -- 172.21.15.17:0/3752035586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a68b1c820 con 0x557a68c1b040 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f533affd640 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c235b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f53300cb4a0 tx=0x7f53300d1ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe1db6a8a95068a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f53217fa640 1 -- 172.21.15.17:0/3752035586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330005380 con 0x557a68c1b040 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f53217fa640 1 -- 172.21.15.17:0/3752035586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f533009a430 con 0x557a68c1b040 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f53217fa640 1 -- 172.21.15.17:0/3752035586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300a2d50 con 0x557a68c1b040 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 -- 172.21.15.17:0/3752035586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 msgr2=0x557a68c235b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c235b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f53300cb4a0 tx=0x7f53300d1ad0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 -- 172.21.15.17:0/3752035586 shutdown_connections 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c235b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c23c20 0x557a68c201d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 --2- 172.21.15.17:0/3752035586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c1a720 0x557a68c23070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 -- 172.21.15.17:0/3752035586 >> 172.21.15.17:0/3752035586 conn(0x557a68c14e50 msgr2=0x557a68c1f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 -- 172.21.15.17:0/3752035586 shutdown_connections 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.539+0000 7f534478a740 1 -- 172.21.15.17:0/3752035586 wait complete. 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 -- start start 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 Processor -- start 2024-02-14T15:03:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1a720 0x557a68c28e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c29350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68c893d0 con 0x557a68c1b040 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68c89510 con 0x557a68c23c20 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a68c896d0 con 0x557a68c1a720 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f534478a740 10 client.0 _ll_register_callbacks cb 0x7f53300cb250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f533affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c29350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f533affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c29350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60000/0 (socket says 172.21.15.17:60000) 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.540+0000 7f533affd640 1 -- 172.21.15.17:0/1100972935 learned_addr learned my addr 172.21.15.17:0/1100972935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f533b7fe640 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1a720 0x557a68c28e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f533affd640 1 -- 172.21.15.17:0/1100972935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1a720 msgr2=0x557a68c28e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f533affd640 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1a720 0x557a68c28e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f533affd640 1 -- 172.21.15.17:0/1100972935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 msgr2=0x557a68c88e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f533affd640 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f533affd640 1 -- 172.21.15.17:0/1100972935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a68892920 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f533affd640 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c29350 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5330097030 tx=0x7f53300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fb58d4ad1e1dd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f531affd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f531affd640 1 -- 172.21.15.17:0/1100972935 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533009e770 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f531affd640 1 -- 172.21.15.17:0/1100972935 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f53300a2d50 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5330020ec0 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f531affd640 1 -- 172.21.15.17:0/1100972935 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5330099fc0 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f534478a740 10 client.15909 fetch_fsmap learned FSMap version 24 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a68b42720 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f531affd640 1 -- 172.21.15.17:0/1100972935 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5330020ec0 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.541+0000 7f531affd640 1 -- 172.21.15.17:0/1100972935 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53300b8d20 con 0x557a68c1b040 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f534478a740 10 client.15909 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f534478a740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f534478a740 20 client.15909 populate_metadata read hostname 'smithi017' 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f534478a740 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c8a9d0 0x557a68c8cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:03:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f534478a740 4 client.15909 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:03:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a688be550 con 0x557a68c8a9d0 2024-02-14T15:03:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f533b7fe640 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c8a9d0 0x557a68c8cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:03:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f533b7fe640 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c8a9d0 0x557a68c8cdf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd298b3c3045a856 server_cookie=9f595ba049ba0ce4 in_seq=0 out_seq=0 2024-02-14T15:03:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.542+0000 7f531affd640 10 client.15909 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f531affd640 1 -- 172.21.15.17:0/1100972935 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a688be550 con 0x557a68c8a9d0 2024-02-14T15:03:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f531affd640 10 client.15909 handle_command_reply: tid=0 2024-02-14T15:03:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 client.15909 shutdown 2024-02-14T15:03:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 msgr2=0x557a68c29350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c29350 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5330097030 tx=0x7f53300057f0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 shutdown_connections 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557a68c8a9d0 0x557a68c8cdf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557a68c1b040 0x557a68c29350 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a68c23c20 0x557a68c88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 --2- 172.21.15.17:0/1100972935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a68c1a720 0x557a68c28e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f531affd640 0 client.15909 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 >> 172.21.15.17:0/1100972935 conn(0x557a68c14e50 msgr2=0x557a68c201a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 shutdown_connections 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 1 -- 172.21.15.17:0/1100972935 wait complete. 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.546+0000 7f534478a740 20 client.15909 trim_cache size 0 max 16384 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:03:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:03:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:03:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53240a22b0 msgr2=0x7f53240a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53240a22b0 0x7f53240a4770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5334006010 tx=0x7f53340095c0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f533c06a1e0 msgr2=0x7f533c19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:03:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f533c06a1e0 0x7f533c19fba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f533000d6d0 tx=0x7f533000db90 comp rx=0 tx=0).stop 2024-02-14T15:03:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 shutdown_connections 2024-02-14T15:03:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53240a22b0 0x7f53240a4770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c107320 0x7f533c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f533c06a1e0 0x7f533c19fba0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 --2- 172.21.15.17:0/2430130980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f533c069840 0x7f533c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:03:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 >> 172.21.15.17:0/2430130980 conn(0x7f533c076f80 msgr2=0x7f533c10c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:03:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 shutdown_connections 2024-02-14T15:03:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:03:53.547+0000 7f5341bc9640 1 -- 172.21.15.17:0/2430130980 wait complete. 2024-02-14T15:03:54.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:53 smithi017 ceph-mon[32549]: scrub summary: idle 2024-02-14T15:03:54.043 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:53 smithi050 ceph-mon[36512]: scrub summary: idle 2024-02-14T15:03:54.049 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:53 smithi076 ceph-mon[37057]: scrub summary: idle 2024-02-14T15:03:55.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:54 smithi017 ceph-mon[32549]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-14T15:03:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:54 smithi050 ceph-mon[36512]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-14T15:03:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:54 smithi076 ceph-mon[37057]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-02-14T15:03:57.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:56 smithi017 ceph-mon[32549]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:03:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:56 smithi050 ceph-mon[36512]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:03:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:56 smithi076 ceph-mon[37057]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:03:59.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:03:58 smithi017 ceph-mon[32549]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:03:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:03:58 smithi050 ceph-mon[36512]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:03:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:03:58 smithi076 ceph-mon[37057]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:00.767 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:04:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:04:00.423+0000 7f908357c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:04:00.767 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:04:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:04:00.423+0000 7f908357c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:04:00.767 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:04:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:04:00.423+0000 7f908357c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:04:00.767 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:04:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:04:00.423+0000 7f908357c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:04:00.767 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:04:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:04:00.423+0000 7f908357c700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-14T15:04:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:00 smithi050 ceph-mon[36512]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-14T15:04:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:00 smithi076 ceph-mon[37057]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-14T15:04:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:00 smithi017 ceph-mon[32549]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-14T15:04:03.068 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:02 smithi017 ceph-mon[32549]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-02-14T15:04:03.069 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:02 smithi017 ceph-mon[32549]: mgrmap e26: x(active, since 36s), standbys: y, z 2024-02-14T15:04:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:02 smithi050 ceph-mon[36512]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-02-14T15:04:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:02 smithi050 ceph-mon[36512]: mgrmap e26: x(active, since 36s), standbys: y, z 2024-02-14T15:04:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:02 smithi076 ceph-mon[37057]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-02-14T15:04:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:02 smithi076 ceph-mon[37057]: mgrmap e26: x(active, since 36s), standbys: y, z 2024-02-14T15:04:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:04 smithi050 ceph-mon[36512]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-02-14T15:04:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:04 smithi076 ceph-mon[37057]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-02-14T15:04:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:04 smithi017 ceph-mon[32549]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-02-14T15:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 -- 172.21.15.17:0/4166532034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100cf0 msgr2=0x7fa38c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 --2- 172.21.15.17:0/4166532034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100cf0 0x7fa38c104d30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa380009930 tx=0x7fa38002bc30 comp rx=0 tx=0).stop 2024-02-14T15:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 -- 172.21.15.17:0/4166532034 shutdown_connections 2024-02-14T15:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 --2- 172.21.15.17:0/4166532034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100cf0 0x7fa38c104d30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 --2- 172.21.15.17:0/4166532034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa38c105410 0x7fa38c10cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 --2- 172.21.15.17:0/4166532034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38c100340 0x7fa38c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 -- 172.21.15.17:0/4166532034 >> 172.21.15.17:0/4166532034 conn(0x7fa38c077220 msgr2=0x7fa38c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 -- 172.21.15.17:0/4166532034 shutdown_connections 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 -- 172.21.15.17:0/4166532034 wait complete. 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 Processor -- start 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.646+0000 7fa392980640 1 -- start start 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa392980640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100340 0x7fa38c101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa392980640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38c105410 0x7fa38c102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa392980640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa38c102850 0x7fa38c1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa392980640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1a5fa0 con 0x7fa38c100340 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa392980640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1a6110 con 0x7fa38c105410 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa392980640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1a62d0 con 0x7fa38c102850 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100340 0x7fa38c101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100340 0x7fa38c101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50276/0 (socket says 172.21.15.17:50276) 2024-02-14T15:04:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 -- 172.21.15.17:0/1600271760 learned_addr learned my addr 172.21.15.17:0/1600271760 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa390f1d640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa38c102850 0x7fa38c1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38c105410 0x7fa38c102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 -- 172.21.15.17:0/1600271760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa38c102850 msgr2=0x7fa38c1a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa38c102850 0x7fa38c1a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 -- 172.21.15.17:0/1600271760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38c105410 msgr2=0x7fa38c102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38c105410 0x7fa38c102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 -- 172.21.15.17:0/1600271760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa380009560 con 0x7fa38c100340 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.647+0000 7fa38bfff640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100340 0x7fa38c101dd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00d950 tx=0x7fa37c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.648+0000 7fa3897fa640 1 -- 172.21.15.17:0/1600271760 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c009c30 con 0x7fa38c100340 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.648+0000 7fa3897fa640 1 -- 172.21.15.17:0/1600271760 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa37c009d90 con 0x7fa38c100340 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.648+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38c1a65b0 con 0x7fa38c100340 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.648+0000 7fa3897fa640 1 -- 172.21.15.17:0/1600271760 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00f7e0 con 0x7fa38c100340 2024-02-14T15:04:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.648+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c1a6a90 con 0x7fa38c100340 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa3897fa640 1 -- 172.21.15.17:0/1600271760 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fa37c010ad0 con 0x7fa38c100340 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 Processor -- start 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 -- start start 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fd0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090f329e0 con 0x55b090fcdc20 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090f2ff10 con 0x55b090fc4720 2024-02-14T15:04:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa395620740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090f2b6f0 con 0x55b090fc5040 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa3897fa640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa3740a2340 0x7fa3740a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa38b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fd0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa390f1d640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa3740a2340 0x7fa3740a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa38b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fd0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50282/0 (socket says 172.21.15.17:50282) 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa38b7fe640 1 -- 172.21.15.17:0/2472908323 learned_addr learned my addr 172.21.15.17:0/2472908323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa3897fa640 1 -- 172.21.15.17:0/1600271760 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa37c0ce340 con 0x7fa38c100340 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa38bfff640 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.651+0000 7fa390f1d640 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa38b7fe640 1 -- 172.21.15.17:0/2472908323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 msgr2=0x55b090fcd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa38b7fe640 1 -- 172.21.15.17:0/2472908323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 msgr2=0x55b090fc4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa38b7fe640 1 -- 172.21.15.17:0/2472908323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b090c3c920 con 0x55b090fcdc20 2024-02-14T15:04:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa390f1d640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa3740a2340 0x7fa3740a4800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa384007fd0 tx=0x7fa384007d60 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fd0010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa380005ab0 tx=0x7fa380004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5469815ae3b2527d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa3727fc640 1 -- 172.21.15.17:0/2472908323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38002c7e0 con 0x55b090fcdc20 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa3727fc640 1 -- 172.21.15.17:0/2472908323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38002c940 con 0x55b090fcdc20 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.652+0000 7fa3727fc640 1 -- 172.21.15.17:0/2472908323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380005350 con 0x55b090fcdc20 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 -- 172.21.15.17:0/2472908323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 msgr2=0x55b090fd0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fd0010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa380005ab0 tx=0x7fa380004740 comp rx=0 tx=0).stop 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 -- 172.21.15.17:0/2472908323 shutdown_connections 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fd0010 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 --2- 172.21.15.17:0/2472908323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 -- 172.21.15.17:0/2472908323 >> 172.21.15.17:0/2472908323 conn(0x55b090fbee50 msgr2=0x55b090fbf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 -- 172.21.15.17:0/2472908323 shutdown_connections 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 -- 172.21.15.17:0/2472908323 wait complete. 2024-02-14T15:04:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 -- start start 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.653+0000 7fa395620740 1 Processor -- start 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc4720 0x55b090fcd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fcdc20 0x55b090fcaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090ec7fa0 con 0x55b090fc4720 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090bff250 con 0x55b090fcdc20 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090eb1200 con 0x55b090fc5040 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 10 client.0 _ll_register_callbacks cb 0x55b090f20b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc4720 0x55b090fcd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc4720 0x55b090fcd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50294/0 (socket says 172.21.15.17:50294) 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 -- 172.21.15.17:0/1868218535 learned_addr learned my addr 172.21.15.17:0/1868218535 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa390f1d640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fcdc20 0x55b090fcaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 -- 172.21.15.17:0/1868218535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 msgr2=0x55b090fca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 -- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fcdc20 msgr2=0x55b090fcaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fcdc20 0x55b090fcaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 -- 172.21.15.17:0/1868218535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c10d7f0 con 0x55b090fc4720 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa390f1d640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fcdc20 0x55b090fcaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa38bfff640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc4720 0x55b090fcd5a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa37c096be0 tx=0x7fa37c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870baf9e0440f3bd server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa36b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa36b7fe640 1 -- 172.21.15.17:0/1868218535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0ce1b0 con 0x55b090fc4720 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa36b7fe640 1 -- 172.21.15.17:0/1868218535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa37c09ae80 con 0x55b090fc4720 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.654+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b090c2b600 con 0x55b090fc4720 2024-02-14T15:04:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa36b7fe640 1 -- 172.21.15.17:0/1868218535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa37c099bd0 con 0x55b090fc4720 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 10 client.15927 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b090f29d70 con 0x55b090fc4720 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa36b7fe640 1 -- 172.21.15.17:0/1868218535 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b090c2b600 con 0x55b090fc4720 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa36b7fe640 1 -- 172.21.15.17:0/1868218535 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa37c096df0 con 0x55b090fc4720 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 10 client.15927 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 20 client.15927 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b090fdd7f0 0x55b090fdfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 4 client.15927 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b090c7c1b0 con 0x55b090fdd7f0 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.655+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b090fdd7f0 0x55b090fdfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.656+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b090fdd7f0 0x55b090fdfc00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899c65aacf3dfb7b server_cookie=9264a3c7d0e91185 in_seq=0 out_seq=0 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.656+0000 7fa36b7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa36b7fe640 1 -- 172.21.15.17:0/1868218535 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b090c7c1b0 con 0x55b090fdd7f0 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa36b7fe640 10 client.15927 handle_command_reply: tid=0 2024-02-14T15:04:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 client.15927 shutdown 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc4720 msgr2=0x55b090fcd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc4720 0x55b090fcd5a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa37c096be0 tx=0x7fa37c005880 comp rx=0 tx=0).stop 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 shutdown_connections 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b090fdd7f0 0x55b090fdfc00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fcdc20 0x55b090fcaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa36b7fe640 0 client.15927 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 --2- 172.21.15.17:0/1868218535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc4720 0x55b090fcd5a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 >> 172.21.15.17:0/1868218535 conn(0x55b090fbee50 msgr2=0x55b090fc0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 shutdown_connections 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 1 -- 172.21.15.17:0/1868218535 wait complete. 2024-02-14T15:04:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.657+0000 7fa395620740 20 client.15927 trim_cache size 0 max 16384 2024-02-14T15:04:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.670+0000 7fa395620740 1 Processor -- start 2024-02-14T15:04:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa395620740 1 -- start start 2024-02-14T15:04:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa395620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa395620740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa395620740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fcd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa395620740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090fc4160 con 0x55b090fcdc20 2024-02-14T15:04:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa395620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090fc6570 con 0x55b090fc4720 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa395620740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b090fe4e50 con 0x55b090fc5040 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fcd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fcd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50298/0 (socket says 172.21.15.17:50298) 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 -- 172.21.15.17:0/3726588790 learned_addr learned my addr 172.21.15.17:0/3726588790 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa38bfff640 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 -- 172.21.15.17:0/3726588790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 msgr2=0x55b090fcd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 -- 172.21.15.17:0/3726588790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 msgr2=0x55b090fc9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 -- 172.21.15.17:0/3726588790 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b090ec6820 con 0x55b090fcdc20 2024-02-14T15:04:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa38bfff640 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa390f1d640 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fcd5a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa3840143a0 tx=0x7fa384004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a63684bbf43471 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.671+0000 7fa371ffb640 1 -- 172.21.15.17:0/3726588790 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384012d60 con 0x55b090fcdc20 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa371ffb640 1 -- 172.21.15.17:0/3726588790 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38401a040 con 0x55b090fcdc20 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa371ffb640 1 -- 172.21.15.17:0/3726588790 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38400f630 con 0x55b090fcdc20 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 -- 172.21.15.17:0/3726588790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 msgr2=0x55b090fcd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fcd5a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa3840143a0 tx=0x7fa384004ab0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 -- 172.21.15.17:0/3726588790 shutdown_connections 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fcdc20 0x55b090fcd5a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fc5040 0x55b090fcd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 --2- 172.21.15.17:0/3726588790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fc9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 -- 172.21.15.17:0/3726588790 >> 172.21.15.17:0/3726588790 conn(0x55b090fbee50 msgr2=0x55b090fc0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 -- 172.21.15.17:0/3726588790 shutdown_connections 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 -- 172.21.15.17:0/3726588790 wait complete. 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.672+0000 7fa395620740 1 -- start start 2024-02-14T15:04:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 1 Processor -- start 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fd3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc5040 0x55b090fd3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fcdc20 0x55b090fd40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091033d30 con 0x55b090fc5040 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091033e70 con 0x55b090fc4720 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091034000 con 0x55b090fcdc20 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa395620740 10 client.0 _ll_register_callbacks cb 0x7fa380009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc5040 0x55b090fd3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc5040 0x55b090fd3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50302/0 (socket says 172.21.15.17:50302) 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 -- 172.21.15.17:0/47822724 learned_addr learned my addr 172.21.15.17:0/47822724 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa390f1d640 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fcdc20 0x55b090fd40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38bfff640 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fd3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 -- 172.21.15.17:0/47822724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fcdc20 msgr2=0x55b090fd40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fcdc20 0x55b090fd40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 -- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 msgr2=0x55b090fd3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fd3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.673+0000 7fa38b7fe640 1 -- 172.21.15.17:0/47822724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3840140b0 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa38b7fe640 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc5040 0x55b090fd3ba0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa380004f00 tx=0x7fa380004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503e14917679bf77 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa36b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa36b7fe640 1 -- 172.21.15.17:0/47822724 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38004f070 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa36b7fe640 1 -- 172.21.15.17:0/47822724 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa3800048b0 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa38002c940 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa36b7fe640 1 -- 172.21.15.17:0/47822724 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3800046c0 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 10 client.15933 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b090eec720 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa36b7fe640 1 -- 172.21.15.17:0/47822724 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38002c940 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa36b7fe640 1 -- 172.21.15.17:0/47822724 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa38003c040 con 0x55b090fc5040 2024-02-14T15:04:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 10 client.15933 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 20 client.15933 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0910354c0 0x55b0910378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 4 client.15933 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.674+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b090c68550 con 0x55b0910354c0 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.675+0000 7fa38bfff640 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0910354c0 0x55b0910378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.675+0000 7fa38bfff640 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0910354c0 0x55b0910378e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15656eb096f6a81e server_cookie=4a719de828a6b005 in_seq=0 out_seq=0 2024-02-14T15:04:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.675+0000 7fa36b7fe640 10 client.15933 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.683+0000 7fa36b7fe640 1 -- 172.21.15.17:0/47822724 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b090c68550 con 0x55b0910354c0 2024-02-14T15:04:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.683+0000 7fa36b7fe640 10 client.15933 handle_command_reply: tid=0 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.683+0000 7fa395620740 1 client.15933 shutdown 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.683+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc5040 msgr2=0x55b090fd3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.683+0000 7fa395620740 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc5040 0x55b090fd3ba0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa380004f00 tx=0x7fa380004e90 comp rx=0 tx=0).stop 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.683+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 shutdown_connections 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.683+0000 7fa395620740 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0910354c0 0x55b0910378e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa395620740 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b090fc5040 0x55b090fd3ba0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa395620740 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b090fcdc20 0x55b090fd40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa36b7fe640 0 client.15933 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa395620740 1 --2- 172.21.15.17:0/47822724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b090fc4720 0x55b090fd3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 >> 172.21.15.17:0/47822724 conn(0x55b090fbee50 msgr2=0x55b090fc01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 shutdown_connections 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa395620740 1 -- 172.21.15.17:0/47822724 wait complete. 2024-02-14T15:04:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa395620740 20 client.15933 trim_cache size 0 max 16384 2024-02-14T15:04:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:04:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:04:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:04:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa3740a2340 msgr2=0x7fa3740a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa3740a2340 0x7fa3740a4800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa384007fd0 tx=0x7fa384007d60 comp rx=0 tx=0).stop 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100340 msgr2=0x7fa38c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100340 0x7fa38c101dd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00d950 tx=0x7fa37c00de10 comp rx=0 tx=0).stop 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 shutdown_connections 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa3740a2340 0x7fa3740a4800 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa38c102850 0x7fa38c1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa38c105410 0x7fa38c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 --2- 172.21.15.17:0/1600271760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c100340 0x7fa38c101dd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 >> 172.21.15.17:0/1600271760 conn(0x7fa38c077220 msgr2=0x7fa38c068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.684+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 shutdown_connections 2024-02-14T15:04:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:06.685+0000 7fa392980640 1 -- 172.21.15.17:0/1600271760 wait complete. 2024-02-14T15:04:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:06 smithi050 ceph-mon[36512]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-02-14T15:04:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:06 smithi076 ceph-mon[37057]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-02-14T15:04:07.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:06 smithi017 ceph-mon[32549]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-02-14T15:04:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:08 smithi050 ceph-mon[36512]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-02-14T15:04:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:08 smithi076 ceph-mon[37057]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-02-14T15:04:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:08 smithi017 ceph-mon[32549]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-02-14T15:04:11.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:10 smithi050 ceph-mon[36512]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-02-14T15:04:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:10 smithi076 ceph-mon[37057]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-02-14T15:04:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:10 smithi017 ceph-mon[32549]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-02-14T15:04:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:12 smithi050 ceph-mon[36512]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-02-14T15:04:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:12 smithi076 ceph-mon[37057]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-02-14T15:04:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:12 smithi017 ceph-mon[32549]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-02-14T15:04:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:14 smithi050 ceph-mon[36512]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-02-14T15:04:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:14 smithi076 ceph-mon[37057]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-02-14T15:04:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:14 smithi017 ceph-mon[32549]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-02-14T15:04:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:16 smithi050 ceph-mon[36512]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-02-14T15:04:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:16 smithi076 ceph-mon[37057]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-02-14T15:04:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:16 smithi017 ceph-mon[32549]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-02-14T15:04:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:18 smithi050 ceph-mon[36512]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-02-14T15:04:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:18 smithi076 ceph-mon[37057]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-02-14T15:04:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:18 smithi017 ceph-mon[32549]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-02-14T15:04:19.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:04:19.461 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:04:19.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 -- 172.21.15.17:0/1586752608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef810b820 msgr2=0x7f0ef810dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 --2- 172.21.15.17:0/1586752608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef810b820 0x7f0ef810dc50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009ca0 tx=0x7f0ef002c180 comp rx=0 tx=0).stop 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 -- 172.21.15.17:0/1586752608 shutdown_connections 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 --2- 172.21.15.17:0/1586752608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef810b820 0x7f0ef810dc50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 --2- 172.21.15.17:0/1586752608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef8102d70 0x7f0ef810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 --2- 172.21.15.17:0/1586752608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8102450 0x7f0ef8102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 -- 172.21.15.17:0/1586752608 >> 172.21.15.17:0/1586752608 conn(0x7f0ef80772e0 msgr2=0x7f0ef80776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 -- 172.21.15.17:0/1586752608 shutdown_connections 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.568+0000 7f0efda39640 1 -- 172.21.15.17:0/1586752608 wait complete. 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 Processor -- start 2024-02-14T15:04:19.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 -- start start 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef8102450 0x7f0ef819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8102d70 0x7f0ef819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef810b820 0x7f0ef81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef8199e80 con 0x7f0ef8102450 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef8199ff0 con 0x7f0ef8102d70 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0efda39640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef819a1b0 con 0x7f0ef810b820 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef8102450 0x7f0ef819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef8102450 0x7f0ef819f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38052/0 (socket says 172.21.15.17:38052) 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/2791708814 learned_addr learned my addr 172.21.15.17:0/2791708814 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef810b820 0x7f0ef81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8102d70 0x7f0ef819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/2791708814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef810b820 msgr2=0x7f0ef81997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef810b820 0x7f0ef81997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/2791708814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8102d70 msgr2=0x7f0ef819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8102d70 0x7f0ef819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/2791708814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef0009900 con 0x7f0ef8102450 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.569+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8102d70 0x7f0ef819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.570+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef8102450 0x7f0ef819f620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800d950 tx=0x7f0ee800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.570+0000 7f0efca37640 1 -- 172.21.15.17:0/2791708814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8009c30 con 0x7f0ef8102450 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.570+0000 7f0efca37640 1 -- 172.21.15.17:0/2791708814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8009d90 con 0x7f0ef8102450 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.570+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef819a490 con 0x7f0ef8102450 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.570+0000 7f0efca37640 1 -- 172.21.15.17:0/2791708814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee800f7e0 con 0x7f0ef8102450 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.570+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef81a6650 con 0x7f0ef8102450 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.572+0000 7f0efca37640 1 -- 172.21.15.17:0/2791708814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f0ee8010ad0 con 0x7f0ef8102450 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.572+0000 7f0f00609740 1 Processor -- start 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.572+0000 7f0f00609740 1 -- start start 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0f00609740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996630 0x561651996a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0f00609740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x56165199f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0f00609740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x5616519a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0f00609740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616519515b0 con 0x56165199fb30 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0f00609740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616519052b0 con 0x561651996f50 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0f00609740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616519027e0 con 0x561651996630 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0efca37640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ee00a2350 0x7f0ee00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x5616519a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x5616519a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38066/0 (socket says 172.21.15.17:38066) 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 -- 172.21.15.17:0/3079915281 learned_addr learned my addr 172.21.15.17:0/3079915281 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ee00a2350 0x7f0ee00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x56165199f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996630 0x561651996a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 -- 172.21.15.17:0/3079915281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996630 msgr2=0x561651996a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996630 0x561651996a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 -- 172.21.15.17:0/3079915281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 msgr2=0x56165199f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x56165199f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 -- 172.21.15.17:0/3079915281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616518c0020 con 0x56165199fb30 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0efca37640 1 -- 172.21.15.17:0/2791708814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0ee80ce360 con 0x7f0ef8102450 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ee00a2350 0x7f0ee00a4810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0004af0 tx=0x7f0ef0032510 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.573+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x5616519a1f20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009b70 tx=0x7f0eec00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b808459b6e2bb11 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0eddffb640 1 -- 172.21.15.17:0/3079915281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec009e60 con 0x56165199fb30 2024-02-14T15:04:19.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0eddffb640 1 -- 172.21.15.17:0/3079915281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eec005500 con 0x56165199fb30 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0eddffb640 1 -- 172.21.15.17:0/3079915281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec011470 con 0x56165199fb30 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 -- 172.21.15.17:0/3079915281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 msgr2=0x5616519a1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x5616519a1f20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009b70 tx=0x7f0eec00e6c0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 -- 172.21.15.17:0/3079915281 shutdown_connections 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x5616519a1f20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x56165199f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 --2- 172.21.15.17:0/3079915281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996630 0x561651996a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 -- 172.21.15.17:0/3079915281 >> 172.21.15.17:0/3079915281 conn(0x561651990d60 msgr2=0x561651991140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 -- 172.21.15.17:0/3079915281 shutdown_connections 2024-02-14T15:04:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.574+0000 7f0f00609740 1 -- 172.21.15.17:0/3079915281 wait complete. 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 -- start start 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 Processor -- start 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996f50 0x56165199d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x56165199d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616518fdfc0 con 0x56165199fb30 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616515d1430 con 0x561651996630 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165158de00 con 0x561651996f50 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0f00609740 10 client.0 _ll_register_callbacks cb 0x56165187f870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:19.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0ef77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x56165199d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0ef77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x56165199d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38068/0 (socket says 172.21.15.17:38068) 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.575+0000 7f0ef77fe640 1 -- 172.21.15.17:0/3569347501 learned_addr learned my addr 172.21.15.17:0/3569347501 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996f50 0x56165199d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef77fe640 1 -- 172.21.15.17:0/3569347501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996f50 msgr2=0x56165199d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996f50 0x56165199d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef77fe640 1 -- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 msgr2=0x56165199cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef77fe640 1 -- 172.21.15.17:0/3569347501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec0097a0 con 0x56165199fb30 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996f50 0x56165199d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x56165199d5c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0036000 tx=0x7f0ef0048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2146ca2957ea1048 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/3569347501 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0048070 con 0x56165199fb30 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/3569347501 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0ef0043700 con 0x56165199fb30 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616515fd7e0 con 0x56165199fb30 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/3569347501 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ef00438c0 con 0x56165199fb30 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0f00609740 10 client.15945 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5616518fc640 con 0x56165199fb30 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.576+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/3569347501 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5616515fd7e0 con 0x56165199fb30 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/3569347501 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ef0050c80 con 0x56165199fb30 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0f00609740 10 client.15945 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0f00609740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0f00609740 20 client.15945 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0f00609740 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x5616519b2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0f00609740 4 client.15945 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56165163a720 con 0x5616519b0000 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x5616519b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x5616519b2420 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d5521884b0422 server_cookie=4f07bcee4dec001b in_seq=0 out_seq=0 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.577+0000 7f0ed6ffd640 10 client.15945 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/3569347501 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56165163a720 con 0x5616519b0000 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0ed6ffd640 10 client.15945 handle_command_reply: tid=0 2024-02-14T15:04:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 client.15945 shutdown 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 msgr2=0x56165199d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x56165199d5c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0036000 tx=0x7f0ef0048000 comp rx=0 tx=0).stop 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 shutdown_connections 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x5616519b2420 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56165199fb30 0x56165199d5c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561651996f50 0x56165199d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0ed6ffd640 0 client.15945 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 --2- 172.21.15.17:0/3569347501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 >> 172.21.15.17:0/3569347501 conn(0x561651990d60 msgr2=0x561651992ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 shutdown_connections 2024-02-14T15:04:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 1 -- 172.21.15.17:0/3569347501 wait complete. 2024-02-14T15:04:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.578+0000 7f0f00609740 20 client.15945 trim_cache size 0 max 16384 2024-02-14T15:04:19.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 Processor -- start 2024-02-14T15:04:19.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 -- start start 2024-02-14T15:04:19.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996f50 0x56165199b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x56165199b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165198c4d0 con 0x561651996f50 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561651882fd0 con 0x561651996630 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0f00609740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165163a750 con 0x56165199fb30 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996f50 0x56165199b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996f50 0x56165199b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38076/0 (socket says 172.21.15.17:38076) 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 -- 172.21.15.17:0/1594354693 learned_addr learned my addr 172.21.15.17:0/1594354693 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x56165199b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 -- 172.21.15.17:0/1594354693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 msgr2=0x56165199b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x56165199b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 -- 172.21.15.17:0/1594354693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 msgr2=0x56165199f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef67fc640 1 -- 172.21.15.17:0/1594354693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561651898b30 con 0x561651996f50 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.594+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x56165199b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.595+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996f50 0x56165199b410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00ebc0 tx=0x7f0eec009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635c78cd8a8dc211 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.595+0000 7f0edd7fa640 1 -- 172.21.15.17:0/1594354693 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec021070 con 0x561651996f50 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.595+0000 7f0edd7fa640 1 -- 172.21.15.17:0/1594354693 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eec005670 con 0x561651996f50 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.595+0000 7f0edd7fa640 1 -- 172.21.15.17:0/1594354693 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec004cc0 con 0x561651996f50 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.595+0000 7f0f00609740 1 -- 172.21.15.17:0/1594354693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996f50 msgr2=0x56165199b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.595+0000 7f0f00609740 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996f50 0x56165199b410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00ebc0 tx=0x7f0eec009680 comp rx=0 tx=0).stop 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- 172.21.15.17:0/1594354693 shutdown_connections 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996f50 0x56165199b410 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x56165199b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 --2- 172.21.15.17:0/1594354693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996630 0x56165199f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- 172.21.15.17:0/1594354693 >> 172.21.15.17:0/1594354693 conn(0x561651990d60 msgr2=0x561651991140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- 172.21.15.17:0/1594354693 shutdown_connections 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- 172.21.15.17:0/1594354693 wait complete. 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- start start 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 Processor -- start 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996630 0x5616519a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x5616519a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x5616519a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616519a5cd0 con 0x561651996630 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616519a5e40 con 0x561651996f50 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.596+0000 7f0f00609740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561651a06550 con 0x56165199fb30 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0f00609740 10 client.0 _ll_register_callbacks cb 0x7f0eec009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996630 0x5616519a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996630 0x5616519a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38080/0 (socket says 172.21.15.17:38080) 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/4120921864 learned_addr learned my addr 172.21.15.17:0/4120921864 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef77fe640 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x5616519a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x5616519a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/4120921864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 msgr2=0x5616519a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x5616519a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 msgr2=0x5616519a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x5616519a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 -- 172.21.15.17:0/4120921864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616518c0020 con 0x561651996630 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ef6ffd640 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996630 0x5616519a6d20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8097170 tx=0x7f0ee80cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b2ffdff6af539f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4120921864 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80cbea0 con 0x561651996630 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4120921864 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0ee80a2f40 con 0x561651996630 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561651689890 con 0x561651996630 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4120921864 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee80b3250 con 0x561651996630 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0f00609740 10 client.15951 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:19.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.597+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5616518be880 con 0x561651996630 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4120921864 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561651689890 con 0x561651996630 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4120921864 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ee80cb0b0 con 0x561651996630 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0f00609740 10 client.15951 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0f00609740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0f00609740 20 client.15951 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0f00609740 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x561651a09b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0f00609740 4 client.15951 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5616515a9700 con 0x5616519b0000 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x561651a09b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0ef67fc640 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x561651a09b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252174dada7a8ef server_cookie=ec237352664651b4 in_seq=0 out_seq=0 2024-02-14T15:04:19.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.598+0000 7f0ed6ffd640 10 client.15951 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0ed6ffd640 1 -- 172.21.15.17:0/4120921864 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5616515a9700 con 0x5616519b0000 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0ed6ffd640 10 client.15951 handle_command_reply: tid=0 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 client.15951 shutdown 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996630 msgr2=0x5616519a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996630 0x5616519a6d20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8097170 tx=0x7f0ee80cb040 comp rx=0 tx=0).stop 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 shutdown_connections 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5616519b0000 0x561651a09b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165199fb30 0x5616519a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561651996f50 0x5616519a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 --2- 172.21.15.17:0/4120921864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561651996630 0x5616519a6d20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 >> 172.21.15.17:0/4120921864 conn(0x561651990d60 msgr2=0x56165199b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0ed6ffd640 0 client.15951 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 shutdown_connections 2024-02-14T15:04:19.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 1 -- 172.21.15.17:0/4120921864 wait complete. 2024-02-14T15:04:19.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.599+0000 7f0f00609740 20 client.15951 trim_cache size 0 max 16384 2024-02-14T15:04:19.610 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:04:19.610 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "no active scrubs running", 2024-02-14T15:04:19.610 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": {} 2024-02-14T15:04:19.610 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:04:19.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ee00a2350 msgr2=0x7f0ee00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ee00a2350 0x7f0ee00a4810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0004af0 tx=0x7f0ef0032510 comp rx=0 tx=0).stop 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef8102450 msgr2=0x7f0ef819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef8102450 0x7f0ef819f620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800d950 tx=0x7f0ee800de10 comp rx=0 tx=0).stop 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 shutdown_connections 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0ee00a2350 0x7f0ee00a4810 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ef810b820 0x7f0ef81997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8102d70 0x7f0ef819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 --2- 172.21.15.17:0/2791708814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ef8102450 0x7f0ef819f620 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 >> 172.21.15.17:0/2791708814 conn(0x7f0ef80772e0 msgr2=0x7f0ef8107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 shutdown_connections 2024-02-14T15:04:19.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.600+0000 7f0efda39640 1 -- 172.21.15.17:0/2791708814 wait complete. 2024-02-14T15:04:19.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-14T15:04:19.614 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:04:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 -- 172.21.15.17:0/1719114663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102d70 msgr2=0x7fc77410b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/1719114663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102d70 0x7fc77410b250 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc764009900 tx=0x7fc76402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 -- 172.21.15.17:0/1719114663 shutdown_connections 2024-02-14T15:04:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/1719114663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102d70 0x7fc77410b250 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/1719114663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc77410b820 0x7fc77410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/1719114663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc774102450 0x7fc774102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 -- 172.21.15.17:0/1719114663 >> 172.21.15.17:0/1719114663 conn(0x7fc7740772b0 msgr2=0x7fc7740776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 -- 172.21.15.17:0/1719114663 shutdown_connections 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 -- 172.21.15.17:0/1719114663 wait complete. 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.719+0000 7fc77bc9f640 1 Processor -- start 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77bc9f640 1 -- start start 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77bc9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102450 0x7fc7741000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77bc9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc774102d70 0x7fc7741005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77bc9f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc77410b820 0x7fc774100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77bc9f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7741a5bc0 con 0x7fc774102450 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77bc9f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7741a5d30 con 0x7fc774102d70 2024-02-14T15:04:19.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77bc9f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7741a5ef0 con 0x7fc77410b820 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102450 0x7fc7741000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102450 0x7fc7741000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38094/0 (socket says 172.21.15.17:38094) 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 -- 172.21.15.17:0/3300862140 learned_addr learned my addr 172.21.15.17:0/3300862140 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77a23c640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc77410b820 0x7fc774100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77923a640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc774102d70 0x7fc7741005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 -- 172.21.15.17:0/3300862140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc77410b820 msgr2=0x7fc774100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc77410b820 0x7fc774100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 -- 172.21.15.17:0/3300862140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc774102d70 msgr2=0x7fc7741005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc774102d70 0x7fc7741005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 -- 172.21.15.17:0/3300862140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764009560 con 0x7fc774102450 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc77923a640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc774102d70 0x7fc7741005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.720+0000 7fc779a3b640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102450 0x7fc7741000a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc77000d950 tx=0x7fc77000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.721+0000 7fc76affd640 1 -- 172.21.15.17:0/3300862140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770009c30 con 0x7fc774102450 2024-02-14T15:04:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.721+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7741a61d0 con 0x7fc774102450 2024-02-14T15:04:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.721+0000 7fc76affd640 1 -- 172.21.15.17:0/3300862140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc770009d90 con 0x7fc774102450 2024-02-14T15:04:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.721+0000 7fc76affd640 1 -- 172.21.15.17:0/3300862140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000f7e0 con 0x7fc774102450 2024-02-14T15:04:19.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.721+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7741a6730 con 0x7fc774102450 2024-02-14T15:04:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.722+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7740680b0 con 0x7fc774102450 2024-02-14T15:04:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.723+0000 7fc76affd640 1 -- 172.21.15.17:0/3300862140 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fc770010ad0 con 0x7fc774102450 2024-02-14T15:04:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.724+0000 7fc76affd640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc7600a2350 0x7fc7600a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.724+0000 7fc76affd640 1 -- 172.21.15.17:0/3300862140 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc7700ce370 con 0x7fc774102450 2024-02-14T15:04:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.724+0000 7fc77923a640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc7600a2350 0x7fc7600a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.725+0000 7fc77923a640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc7600a2350 0x7fc7600a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc774101b30 tx=0x7fc764005a20 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.729+0000 7fc76affd640 1 -- 172.21.15.17:0/3300862140 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc770096a50 con 0x7fc774102450 2024-02-14T15:04:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 -- 172.21.15.17:0/2536454911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078068d40 msgr2=0x7f0078069120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 --2- 172.21.15.17:0/2536454911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078068d40 0x7f0078069120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0068009890 tx=0x7f006802bb70 comp rx=0 tx=0).stop 2024-02-14T15:04:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 -- 172.21.15.17:0/2536454911 shutdown_connections 2024-02-14T15:04:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 --2- 172.21.15.17:0/2536454911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0078106510 0x7f007810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 --2- 172.21.15.17:0/2536454911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00780696f0 0x7f0078105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 --2- 172.21.15.17:0/2536454911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0078068d40 0x7f0078069120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 -- 172.21.15.17:0/2536454911 >> 172.21.15.17:0/2536454911 conn(0x7f0078077220 msgr2=0x7f0078077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 -- 172.21.15.17:0/2536454911 shutdown_connections 2024-02-14T15:04:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.778+0000 7f007e450640 1 -- 172.21.15.17:0/2536454911 wait complete. 2024-02-14T15:04:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 Processor -- start 2024-02-14T15:04:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 -- start start 2024-02-14T15:04:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0078068d40 0x7f007819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0078106510 0x7f007819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00781976d0 0x7f0078197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781981a0 con 0x7f0078106510 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0078198310 con 0x7f00781976d0 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f007e450640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781984d0 con 0x7f0078068d40 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f00777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0078106510 0x7f007819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.779+0000 7f0077fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0078068d40 0x7f007819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0078068d40 0x7f007819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:56840/0 (socket says 172.21.15.17:56840) 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f00777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0078106510 0x7f007819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38112/0 (socket says 172.21.15.17:38112) 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 -- 172.21.15.17:0/3292405065 learned_addr learned my addr 172.21.15.17:0/3292405065 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f007c9ed640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00781976d0 0x7f0078197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 -- 172.21.15.17:0/3292405065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00781976d0 msgr2=0x7f0078197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00781976d0 0x7f0078197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 -- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0078106510 msgr2=0x7f007819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0078106510 0x7f007819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 -- 172.21.15.17:0/3292405065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068009490 con 0x7f0078068d40 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f007c9ed640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00781976d0 0x7f0078197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f0077fff640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0078068d40 0x7f007819d610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f00757fa640 1 -- 172.21.15.17:0/3292405065 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068034950 con 0x7f0078068d40 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00781a4410 con 0x7f0078068d40 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.780+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00781a47d0 con 0x7f0078068d40 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.781+0000 7f00757fa640 1 -- 172.21.15.17:0/3292405065 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f006802c9f0 con 0x7f0078068d40 2024-02-14T15:04:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.781+0000 7f00757fa640 1 -- 172.21.15.17:0/3292405065 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068032600 con 0x7f0078068d40 2024-02-14T15:04:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.783+0000 7f00757fa640 1 -- 172.21.15.17:0/3292405065 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f006803b040 con 0x7f0078068d40 2024-02-14T15:04:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f00757fa640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f00600a2340 0x7f00600a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 Processor -- start 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f00777fe640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f00600a2340 0x7f00600a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 -- start start 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f00757fa640 1 -- 172.21.15.17:0/3292405065 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f00680efdf0 con 0x7f0078068d40 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff90ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff9136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff916010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff8789e0 con 0x55d5ff90a720 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff875f10 con 0x55d5ff90b040 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f008101c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff8716f0 con 0x55d5ff913c20 2024-02-14T15:04:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f0077fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff90ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f0077fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff90ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38116/0 (socket says 172.21.15.17:38116) 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f0077fff640 1 -- 172.21.15.17:0/1791479418 learned_addr learned my addr 172.21.15.17:0/1791479418 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.784+0000 7f00777fe640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f00600a2340 0x7f00600a4800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0078198f20 tx=0x7f006c005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f007c9ed640 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff916010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f0077fff640 1 -- 172.21.15.17:0/1791479418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 msgr2=0x55d5ff916010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f0077fff640 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff916010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f0077fff640 1 -- 172.21.15.17:0/1791479418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 msgr2=0x55d5ff9136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f0077fff640 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff9136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f0077fff640 1 -- 172.21.15.17:0/1791479418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5ff582920 con 0x55d5ff90a720 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f0077fff640 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff90ab00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0068009860 tx=0x7f0068005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61220a9f7f485c7c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f005e7fc640 1 -- 172.21.15.17:0/1791479418 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680bc2e0 con 0x55d5ff90a720 2024-02-14T15:04:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f005e7fc640 1 -- 172.21.15.17:0/1791479418 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00680c0430 con 0x55d5ff90a720 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.785+0000 7f005e7fc640 1 -- 172.21.15.17:0/1791479418 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680bc7a0 con 0x55d5ff90a720 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 -- 172.21.15.17:0/1791479418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 msgr2=0x55d5ff90ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff90ab00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0068009860 tx=0x7f0068005ab0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 -- 172.21.15.17:0/1791479418 shutdown_connections 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff916010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff9136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 --2- 172.21.15.17:0/1791479418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff90ab00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 -- 172.21.15.17:0/1791479418 >> 172.21.15.17:0/1791479418 conn(0x55d5ff904e50 msgr2=0x55d5ff905230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 -- 172.21.15.17:0/1791479418 shutdown_connections 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 -- 172.21.15.17:0/1791479418 wait complete. 2024-02-14T15:04:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.786+0000 7f008101c740 1 -- start start 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 1 Processor -- start 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff9134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff910840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff910d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff80dfa0 con 0x55d5ff90a720 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff545250 con 0x55d5ff90b040 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff7f7200 con 0x55d5ff913c20 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f008101c740 10 client.0 _ll_register_callbacks cb 0x55d5ff866b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff9134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff9134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38120/0 (socket says 172.21.15.17:38120) 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 -- 172.21.15.17:0/3241800036 learned_addr learned my addr 172.21.15.17:0/3241800036 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f00777fe640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff910840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f007c9ed640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff910d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 -- 172.21.15.17:0/3241800036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 msgr2=0x55d5ff910d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff910d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 -- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 msgr2=0x55d5ff910840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff910840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f0077fff640 1 -- 172.21.15.17:0/3241800036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00781a47d0 con 0x55d5ff90a720 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f007c9ed640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff910d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.787+0000 7f00777fe640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff910840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f0077fff640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff9134b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f00680c4550 tx=0x7f00680b8950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8300ec5957382ea5 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f00577fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f00577fe640 1 -- 172.21.15.17:0/3241800036 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680c47f0 con 0x55d5ff90a720 2024-02-14T15:04:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f00577fe640 1 -- 172.21.15.17:0/3241800036 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f00680bc2e0 con 0x55d5ff90a720 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5ff571600 con 0x55d5ff90a720 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f00577fe640 1 -- 172.21.15.17:0/3241800036 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00680ed2e0 con 0x55d5ff90a720 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f008101c740 10 client.15969 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d5ff86fd70 con 0x55d5ff90a720 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f00577fe640 1 -- 172.21.15.17:0/3241800036 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d5ff571600 con 0x55d5ff90a720 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f00577fe640 1 -- 172.21.15.17:0/3241800036 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f00680b8030 con 0x55d5ff90a720 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f008101c740 10 client.15969 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f008101c740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f008101c740 20 client.15969 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.788+0000 7f008101c740 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff923190 0x55d5ff9255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.789+0000 7f008101c740 4 client.15969 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.789+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5ff5c21b0 con 0x55d5ff923190 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.789+0000 7f00777fe640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff923190 0x55d5ff9255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.789+0000 7f00577fe640 10 client.15969 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.789+0000 7f00777fe640 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff923190 0x55d5ff9255b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d61c868c7d37235 server_cookie=a1e155a519f8e3ef in_seq=0 out_seq=0 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f00577fe640 1 -- 172.21.15.17:0/3241800036 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5ff5c21b0 con 0x55d5ff923190 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f00577fe640 10 client.15969 handle_command_reply: tid=0 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 client.15969 shutdown 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 msgr2=0x55d5ff9134b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff9134b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f00680c4550 tx=0x7f00680b8950 comp rx=0 tx=0).stop 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 shutdown_connections 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff923190 0x55d5ff9255b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff913c20 0x55d5ff910d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff90b040 0x55d5ff910840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f00577fe640 0 client.15969 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 --2- 172.21.15.17:0/3241800036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90a720 0x55d5ff9134b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 >> 172.21.15.17:0/3241800036 conn(0x55d5ff904e50 msgr2=0x55d5ff906ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 shutdown_connections 2024-02-14T15:04:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 1 -- 172.21.15.17:0/3241800036 wait complete. 2024-02-14T15:04:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.790+0000 7f008101c740 20 client.15969 trim_cache size 0 max 16384 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.804+0000 7f008101c740 1 Processor -- start 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.804+0000 7f008101c740 1 -- start start 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.804+0000 7f008101c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff90bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f008101c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff912f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f008101c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff9134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f008101c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff920080 con 0x55d5ff90b040 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f008101c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff905e50 con 0x55d5ff913c20 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f008101c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff916010 con 0x55d5ff90a720 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff912f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff912f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38122/0 (socket says 172.21.15.17:38122) 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 -- 172.21.15.17:0/3635732905 learned_addr learned my addr 172.21.15.17:0/3635732905 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f007c9ed640 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff9134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f0077fff640 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff90bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 -- 172.21.15.17:0/3635732905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 msgr2=0x55d5ff90bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff90bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 -- 172.21.15.17:0/3635732905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 msgr2=0x55d5ff9134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff9134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 -- 172.21.15.17:0/3635732905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5ff80c820 con 0x55d5ff90b040 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f007c9ed640 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff9134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f00777fe640 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff912f70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55d5ff9100a0 tx=0x7f006c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67695872ccefc25 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.805+0000 7f005dffb640 1 -- 172.21.15.17:0/3635732905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c027de0 con 0x55d5ff90b040 2024-02-14T15:04:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.806+0000 7f005dffb640 1 -- 172.21.15.17:0/3635732905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f006c027800 con 0x55d5ff90b040 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.806+0000 7f005dffb640 1 -- 172.21.15.17:0/3635732905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c004a00 con 0x55d5ff90b040 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.806+0000 7f008101c740 1 -- 172.21.15.17:0/3635732905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 msgr2=0x55d5ff912f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.806+0000 7f008101c740 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff912f70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55d5ff9100a0 tx=0x7f006c014000 comp rx=0 tx=0).stop 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 -- 172.21.15.17:0/3635732905 shutdown_connections 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff912f70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff9134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 --2- 172.21.15.17:0/3635732905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff90bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 -- 172.21.15.17:0/3635732905 >> 172.21.15.17:0/3635732905 conn(0x55d5ff904e50 msgr2=0x55d5ff90b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 -- 172.21.15.17:0/3635732905 shutdown_connections 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 -- 172.21.15.17:0/3635732905 wait complete. 2024-02-14T15:04:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 -- start start 2024-02-14T15:04:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.807+0000 7f008101c740 1 Processor -- start 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f008101c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff9192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f008101c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff9197e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f008101c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff919d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f008101c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff91b100 con 0x55d5ff90b040 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f008101c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff9796a0 con 0x55d5ff913c20 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f008101c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5ff979860 con 0x55d5ff90a720 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff9197e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff9197e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38138/0 (socket says 172.21.15.17:38138) 2024-02-14T15:04:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 -- 172.21.15.17:0/3931598391 learned_addr learned my addr 172.21.15.17:0/3931598391 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f008101c740 10 client.0 _ll_register_callbacks cb 0x7f00680bc0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f0077fff640 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff9192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f007c9ed640 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff919d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 -- 172.21.15.17:0/3931598391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 msgr2=0x55d5ff9192a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff9192a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 -- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 msgr2=0x55d5ff919d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff919d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 -- 172.21.15.17:0/3931598391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c017070 con 0x55d5ff90b040 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00777fe640 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff9197e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f006c009220 tx=0x7f006c005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563ee7989eb35aa5 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00577fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.808+0000 7f00577fe640 1 -- 172.21.15.17:0/3931598391 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c027df0 con 0x55d5ff90b040 2024-02-14T15:04:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f00577fe640 1 -- 172.21.15.17:0/3931598391 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f006c024070 con 0x55d5ff90b040 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00680c0430 con 0x55d5ff90b040 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f00577fe640 1 -- 172.21.15.17:0/3931598391 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006c027320 con 0x55d5ff90b040 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 10 client.15975 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d5ff832720 con 0x55d5ff90b040 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f00577fe640 1 -- 172.21.15.17:0/3931598391 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00680c0430 con 0x55d5ff90b040 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f00577fe640 1 -- 172.21.15.17:0/3931598391 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f006c02d020 con 0x55d5ff90b040 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 10 client.15975 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 20 client.15975 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff97b0e0 0x55d5ff97d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 4 client.15975 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.809+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5ff5ae550 con 0x55d5ff97b0e0 2024-02-14T15:04:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.810+0000 7f0077fff640 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff97b0e0 0x55d5ff97d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.810+0000 7f00577fe640 10 client.15975 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.810+0000 7f0077fff640 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff97b0e0 0x55d5ff97d500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ee063339fb310b server_cookie=f3af733ab8b24c27 in_seq=0 out_seq=0 2024-02-14T15:04:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f00577fe640 1 -- 172.21.15.17:0/3931598391 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5ff5ae550 con 0x55d5ff97b0e0 2024-02-14T15:04:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f00577fe640 10 client.15975 handle_command_reply: tid=0 2024-02-14T15:04:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 client.15975 shutdown 2024-02-14T15:04:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 msgr2=0x55d5ff9197e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff9197e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f006c009220 tx=0x7f006c005600 comp rx=0 tx=0).stop 2024-02-14T15:04:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 shutdown_connections 2024-02-14T15:04:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5ff97b0e0 0x55d5ff97d500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5ff90b040 0x55d5ff9197e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5ff913c20 0x55d5ff919d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 --2- 172.21.15.17:0/3931598391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5ff90a720 0x55d5ff9192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 >> 172.21.15.17:0/3931598391 conn(0x55d5ff904e50 msgr2=0x55d5ff90bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f00577fe640 0 client.15975 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 shutdown_connections 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 1 -- 172.21.15.17:0/3931598391 wait complete. 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.814+0000 7f008101c740 20 client.15975 trim_cache size 0 max 16384 2024-02-14T15:04:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:04:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:04:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:04:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:04:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f00600a2340 msgr2=0x7f00600a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f00600a2340 0x7f00600a4800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0078198f20 tx=0x7f006c005f90 comp rx=0 tx=0).stop 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0078068d40 msgr2=0x7f007819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0078068d40 0x7f007819d610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068004530 comp rx=0 tx=0).stop 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 shutdown_connections 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f00600a2340 0x7f00600a4800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0078106510 0x7f007819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f00781976d0 0x7f0078197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 --2- 172.21.15.17:0/3292405065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0078068d40 0x7f007819d610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 >> 172.21.15.17:0/3292405065 conn(0x7f0078077220 msgr2=0x7f00781076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 shutdown_connections 2024-02-14T15:04:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:19.815+0000 7f007e450640 1 -- 172.21.15.17:0/3292405065 wait complete. 2024-02-14T15:04:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.995+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc774106910 con 0x7fc774102450 2024-02-14T15:04:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.996+0000 7fc76affd640 1 -- 172.21.15.17:0/3300862140 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11317 (secure 0 0 0) 0x7fc77009a110 con 0x7fc774102450 2024-02-14T15:04:19.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:04:19.998 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:03:09.607099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:04:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2024-02-14T15:04:20.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc7600a2350 msgr2=0x7fc7600a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc7600a2350 0x7fc7600a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc774101b30 tx=0x7fc764005a20 comp rx=0 tx=0).stop 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102450 msgr2=0x7fc7741000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102450 0x7fc7741000a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc77000d950 tx=0x7fc77000de10 comp rx=0 tx=0).stop 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 shutdown_connections 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc7600a2350 0x7fc7600a4810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc77410b820 0x7fc774100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc774102d70 0x7fc7741005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 --2- 172.21.15.17:0/3300862140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc774102450 0x7fc7741000a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 >> 172.21.15.17:0/3300862140 conn(0x7fc7740772b0 msgr2=0x7fc7740fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 shutdown_connections 2024-02-14T15:04:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:19.998+0000 7fc77bc9f640 1 -- 172.21.15.17:0/3300862140 wait complete. 2024-02-14T15:04:20.011 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:04:20.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-14T15:04:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 -- 172.21.15.17:0/3797113451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98105410 msgr2=0x7f0c9810cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 --2- 172.21.15.17:0/3797113451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98105410 0x7f0c9810cfe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009cd0 tx=0x7f0c9002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 -- 172.21.15.17:0/3797113451 shutdown_connections 2024-02-14T15:04:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 --2- 172.21.15.17:0/3797113451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98105410 0x7f0c9810cfe0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 --2- 172.21.15.17:0/3797113451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c98100cf0 0x7f0c98104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 --2- 172.21.15.17:0/3797113451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c98100340 0x7f0c98100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 -- 172.21.15.17:0/3797113451 >> 172.21.15.17:0/3797113451 conn(0x7f0c980772e0 msgr2=0x7f0c980776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 -- 172.21.15.17:0/3797113451 shutdown_connections 2024-02-14T15:04:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 -- 172.21.15.17:0/3797113451 wait complete. 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 Processor -- start 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.116+0000 7f0ca028f640 1 -- start start 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98100340 0x7f0c9819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c98105410 0x7f0c9819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c98199880 0x7f0c98199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9819a270 con 0x7f0c98100340 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9819a3e0 con 0x7f0c98105410 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9819a5a0 con 0x7f0c98199880 2024-02-14T15:04:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98100340 0x7f0c9819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98100340 0x7f0c9819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38166/0 (socket says 172.21.15.17:38166) 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 -- 172.21.15.17:0/2191115323 learned_addr learned my addr 172.21.15.17:0/2191115323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c98199880 0x7f0c98199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c98105410 0x7f0c9819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 -- 172.21.15.17:0/2191115323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c98199880 msgr2=0x7f0c98199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c98199880 0x7f0c98199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 -- 172.21.15.17:0/2191115323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c98105410 msgr2=0x7f0c9819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c98105410 0x7f0c9819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 -- 172.21.15.17:0/2191115323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009900 con 0x7f0c98100340 2024-02-14T15:04:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98100340 0x7f0c9819f6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e9f0 tx=0x7f0c9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c8f7fe640 1 -- 172.21.15.17:0/2191115323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94009720 con 0x7f0c98100340 2024-02-14T15:04:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c8f7fe640 1 -- 172.21.15.17:0/2191115323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c9400cd80 con 0x7f0c98100340 2024-02-14T15:04:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c981a6590 con 0x7f0c98100340 2024-02-14T15:04:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0c8f7fe640 1 -- 172.21.15.17:0/2191115323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94010690 con 0x7f0c98100340 2024-02-14T15:04:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.117+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c981a6aa0 con 0x7f0c98100340 2024-02-14T15:04:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.120+0000 7f0c8f7fe640 1 -- 172.21.15.17:0/2191115323 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f0c940046c0 con 0x7f0c98100340 2024-02-14T15:04:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.120+0000 7f0c8f7fe640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c840a23d0 0x7f0c840a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.120+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c840a23d0 0x7f0c840a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.120+0000 7f0c8f7fe640 1 -- 172.21.15.17:0/2191115323 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0c94014070 con 0x7f0c98100340 2024-02-14T15:04:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.121+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c840a23d0 0x7f0c840a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005e00 tx=0x7f0c88005c50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.121+0000 7f0ca2e65740 1 Processor -- start 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.121+0000 7f0ca2e65740 1 -- start start 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.121+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.121+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.121+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f6248c80 con 0x55e5f62f3210 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f6258320 con 0x55e5f62e9ce0 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f6255850 con 0x55e5f62ea690 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38172/0 (socket says 172.21.15.17:38172) 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2319277127 learned_addr learned my addr 172.21.15.17:0/2319277127 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2319277127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 msgr2=0x55e5f62f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2319277127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 msgr2=0x55e5f62ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62ea0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2319277127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f6217e30 con 0x55e5f62f3210 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f5660 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0c9002c700 tx=0x7f0c90032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118c78ac745b7b04 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c8cbfe640 1 -- 172.21.15.17:0/2319277127 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90034440 con 0x55e5f62f3210 2024-02-14T15:04:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c8cbfe640 1 -- 172.21.15.17:0/2319277127 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c900345a0 con 0x55e5f62f3210 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.122+0000 7f0c8cbfe640 1 -- 172.21.15.17:0/2319277127 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90003b90 con 0x55e5f62f3210 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2319277127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 msgr2=0x55e5f62f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f5660 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0c9002c700 tx=0x7f0c90032d70 comp rx=0 tx=0).stop 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2319277127 shutdown_connections 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f5660 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2319277127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2319277127 >> 172.21.15.17:0/2319277127 conn(0x55e5f62e43b0 msgr2=0x55e5f62e4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2319277127 shutdown_connections 2024-02-14T15:04:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.123+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2319277127 wait complete. 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 -- start start 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 Processor -- start 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f6251030 con 0x55e5f62f3210 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5f24430 con 0x55e5f62ea690 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5ee0e00 con 0x55e5f62e9ce0 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0ca2e65740 10 client.0 _ll_register_callbacks cb 0x55e5f5f61be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e82c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e82c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38174/0 (socket says 172.21.15.17:38174) 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e82c640 1 -- 172.21.15.17:0/1110611417 learned_addr learned my addr 172.21.15.17:0/1110611417 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e82c640 1 -- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62e9ce0 msgr2=0x55e5f62f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e82c640 1 -- 172.21.15.17:0/1110611417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 msgr2=0x55e5f62f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.124+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c9e82c640 1 -- 172.21.15.17:0/1110611417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9810d7f0 con 0x55e5f62f3210 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f0c60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009bf0 tx=0x7f0c90009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d867441b65b730ea server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c81ffb640 1 -- 172.21.15.17:0/1110611417 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90034ea0 con 0x55e5f62f3210 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c81ffb640 1 -- 172.21.15.17:0/1110611417 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0c90034710 con 0x55e5f62f3210 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5f5f507e0 con 0x55e5f62f3210 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c81ffb640 1 -- 172.21.15.17:0/1110611417 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c90003970 con 0x55e5f62f3210 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 10 client.15987 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e5f624f6b0 con 0x55e5f62f3210 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c81ffb640 1 -- 172.21.15.17:0/1110611417 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e5f5f507e0 con 0x55e5f62f3210 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0c81ffb640 1 -- 172.21.15.17:0/1110611417 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c9004a450 con 0x55e5f62f3210 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 10 client.15987 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 10 client.15987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:04:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 20 client.15987 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f6303030 0x55e5f6305420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 4 client.15987 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.125+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5f5f8d720 con 0x55e5f6303030 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.126+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f6303030 0x55e5f6305420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.126+0000 7f0c81ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.126+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f6303030 0x55e5f6305420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7d815ac5790a532 server_cookie=afd421f160b325db in_seq=0 out_seq=0 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0c81ffb640 1 -- 172.21.15.17:0/1110611417 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5f5f8d720 con 0x55e5f6303030 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0c81ffb640 10 client.15987 handle_command_reply: tid=0 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 client.15987 shutdown 2024-02-14T15:04:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 msgr2=0x55e5f62f0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f0c60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009bf0 tx=0x7f0c90009360 comp rx=0 tx=0).stop 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 shutdown_connections 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f6303030 0x55e5f6305420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62f0c60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/1110611417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 >> 172.21.15.17:0/1110611417 conn(0x55e5f62e43b0 msgr2=0x55e5f62e6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0c81ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 shutdown_connections 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 1 -- 172.21.15.17:0/1110611417 wait complete. 2024-02-14T15:04:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.127+0000 7f0ca2e65740 20 client.15987 trim_cache size 0 max 16384 2024-02-14T15:04:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 Processor -- start 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 -- start start 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f61d6130 con 0x55e5f62f3210 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f62f5cf0 con 0x55e5f62e9ce0 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f62e3c50 con 0x55e5f62ea690 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62eeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38182/0 (socket says 172.21.15.17:38182) 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2286118232 learned_addr learned my addr 172.21.15.17:0/2286118232 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2286118232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 msgr2=0x55e5f62eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2286118232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 msgr2=0x55e5f62f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e82c640 1 -- 172.21.15.17:0/2286118232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f61ebc90 con 0x55e5f62f3210 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.139+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62eeff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0c90002230 tx=0x7f0c90005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5d829e76e42ce server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0c83fff640 1 -- 172.21.15.17:0/2286118232 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90034440 con 0x55e5f62f3210 2024-02-14T15:04:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0c83fff640 1 -- 172.21.15.17:0/2286118232 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c9004ae80 con 0x55e5f62f3210 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0c83fff640 1 -- 172.21.15.17:0/2286118232 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90004290 con 0x55e5f62f3210 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2286118232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 msgr2=0x55e5f62eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62eeff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0c90002230 tx=0x7f0c90005bd0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2286118232 shutdown_connections 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62f3210 0x55e5f62eeff0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62ea690 0x55e5f62eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/2286118232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62e9ce0 0x55e5f62f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.140+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2286118232 >> 172.21.15.17:0/2286118232 conn(0x55e5f62e43b0 msgr2=0x55e5f62eb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2286118232 shutdown_connections 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 -- 172.21.15.17:0/2286118232 wait complete. 2024-02-14T15:04:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 -- start start 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 Processor -- start 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62e9ce0 0x55e5f62fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62f3210 0x55e5f62f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f62f9830 con 0x55e5f62e9ce0 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f62f99a0 con 0x55e5f62ea690 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f63594e0 con 0x55e5f62f3210 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0ca2e65740 10 client.0 _ll_register_callbacks cb 0x7f0c90037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0c9e02b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62e9ce0 0x55e5f62fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0c9e02b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62e9ce0 0x55e5f62fa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38186/0 (socket says 172.21.15.17:38186) 2024-02-14T15:04:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0c9e02b640 1 -- 172.21.15.17:0/3769025096 learned_addr learned my addr 172.21.15.17:0/3769025096 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0c9e82c640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62f3210 0x55e5f62f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.141+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c9e02b640 1 -- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62f3210 msgr2=0x55e5f62f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62f3210 0x55e5f62f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c9e02b640 1 -- 172.21.15.17:0/3769025096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 msgr2=0x55e5f62f8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c9e02b640 1 -- 172.21.15.17:0/3769025096 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5f624f6b0 con 0x55e5f62e9ce0 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c9e02b640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62e9ce0 0x55e5f62fa650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0c940cbe00 tx=0x7f0c9409aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d31a4baf7a600b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c81ffb640 1 -- 172.21.15.17:0/3769025096 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c940a3660 con 0x55e5f62e9ce0 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c81ffb640 1 -- 172.21.15.17:0/3769025096 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0c94097530 con 0x55e5f62e9ce0 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5f5fdc890 con 0x55e5f62e9ce0 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c81ffb640 1 -- 172.21.15.17:0/3769025096 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c940976f0 con 0x55e5f62e9ce0 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 10 client.15993 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e5f6211aa0 con 0x55e5f62e9ce0 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c81ffb640 1 -- 172.21.15.17:0/3769025096 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c940ac030 con 0x55e5f62e9ce0 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0c81ffb640 1 -- 172.21.15.17:0/3769025096 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c9401f060 con 0x55e5f62e9ce0 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 10 client.15993 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 10 client.15993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 20 client.15993 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f635ad40 0x55e5f635d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 4 client.15993 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.142+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5f5efc700 con 0x55e5f635ad40 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f635ad40 0x55e5f635d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0c9d82a640 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f635ad40 0x55e5f635d130 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e1c84deac3d9a89 server_cookie=ecfba343454c3f96 in_seq=0 out_seq=0 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0c81ffb640 10 client.15993 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0c81ffb640 1 -- 172.21.15.17:0/3769025096 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5f5efc700 con 0x55e5f635ad40 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0c81ffb640 10 client.15993 handle_command_reply: tid=0 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0ca2e65740 1 client.15993 shutdown 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62e9ce0 msgr2=0x55e5f62fa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.143+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62e9ce0 0x55e5f62fa650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0c940cbe00 tx=0x7f0c9409aac0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 shutdown_connections 2024-02-14T15:04:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55e5f635ad40 0x55e5f635d130 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5f62f3210 0x55e5f62f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e5f62ea690 0x55e5f62f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0c81ffb640 0 client.15993 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 --2- 172.21.15.17:0/3769025096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e5f62e9ce0 0x55e5f62fa650 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 >> 172.21.15.17:0/3769025096 conn(0x55e5f62e43b0 msgr2=0x55e5f62eea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 shutdown_connections 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 1 -- 172.21.15.17:0/3769025096 wait complete. 2024-02-14T15:04:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca2e65740 20 client.15993 trim_cache size 0 max 16384 2024-02-14T15:04:20.158 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:04:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c840a23d0 msgr2=0x7f0c840a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c840a23d0 0x7f0c840a4890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005e00 tx=0x7f0c88005c50 comp rx=0 tx=0).stop 2024-02-14T15:04:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98100340 msgr2=0x7f0c9819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98100340 0x7f0c9819f6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e9f0 tx=0x7f0c9400edf0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 shutdown_connections 2024-02-14T15:04:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f0c840a23d0 0x7f0c840a4890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c98199880 0x7f0c98199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c98105410 0x7f0c9819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 --2- 172.21.15.17:0/2191115323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c98100340 0x7f0c9819f6d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 >> 172.21.15.17:0/2191115323 conn(0x7f0c980772e0 msgr2=0x7f0c98068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 shutdown_connections 2024-02-14T15:04:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.144+0000 7f0ca028f640 1 -- 172.21.15.17:0/2191115323 wait complete. 2024-02-14T15:04:20.160 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:04:20.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-14T15:04:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.265+0000 7feac506f640 1 -- 172.21.15.17:0/3981334764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102d70 msgr2=0x7feac010b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.265+0000 7feac506f640 1 --2- 172.21.15.17:0/3981334764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102d70 0x7feac010b250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feab4009900 tx=0x7feab402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.265+0000 7feac506f640 1 -- 172.21.15.17:0/3981334764 shutdown_connections 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.265+0000 7feac506f640 1 --2- 172.21.15.17:0/3981334764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102d70 0x7feac010b250 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.265+0000 7feac506f640 1 --2- 172.21.15.17:0/3981334764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feac010b820 0x7feac010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.265+0000 7feac506f640 1 --2- 172.21.15.17:0/3981334764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac0102450 0x7feac0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.265+0000 7feac506f640 1 -- 172.21.15.17:0/3981334764 >> 172.21.15.17:0/3981334764 conn(0x7feac0077250 msgr2=0x7feac0077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.266+0000 7feac506f640 1 -- 172.21.15.17:0/3981334764 shutdown_connections 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.266+0000 7feac506f640 1 -- 172.21.15.17:0/3981334764 wait complete. 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.266+0000 7feac506f640 1 Processor -- start 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.266+0000 7feac506f640 1 -- start start 2024-02-14T15:04:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feac506f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102450 0x7feac019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feac506f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feac0102d70 0x7feac019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feac506f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac010b820 0x7feac0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feac506f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0197c80 con 0x7feac0102450 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feac506f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0197df0 con 0x7feac0102d70 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feac506f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0197fb0 con 0x7feac010b820 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102450 0x7feac019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102450 0x7feac019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38204/0 (socket says 172.21.15.17:38204) 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 -- 172.21.15.17:0/3832027264 learned_addr learned my addr 172.21.15.17:0/3832027264 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabf59e640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac010b820 0x7feac0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabe59c640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feac0102d70 0x7feac019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 -- 172.21.15.17:0/3832027264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac010b820 msgr2=0x7feac0197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac010b820 0x7feac0197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 -- 172.21.15.17:0/3832027264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feac0102d70 msgr2=0x7feac019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feac0102d70 0x7feac019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 -- 172.21.15.17:0/3832027264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4009560 con 0x7feac0102450 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feabed9d640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102450 0x7feac019d430 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feab000d770 tx=0x7feab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.267+0000 7feaa7fff640 1 -- 172.21.15.17:0/3832027264 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0009c30 con 0x7feac0102450 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.268+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac0198290 con 0x7feac0102450 2024-02-14T15:04:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.268+0000 7feaa7fff640 1 -- 172.21.15.17:0/3832027264 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feab000de80 con 0x7feac0102450 2024-02-14T15:04:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.268+0000 7feaa7fff640 1 -- 172.21.15.17:0/3832027264 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0004e80 con 0x7feac0102450 2024-02-14T15:04:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.268+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01a44d0 con 0x7feac0102450 2024-02-14T15:04:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.270+0000 7feaa7fff640 1 -- 172.21.15.17:0/3832027264 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7feab0005020 con 0x7feac0102450 2024-02-14T15:04:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 Processor -- start 2024-02-14T15:04:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 -- start start 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22eefce0 0x55eb22ef00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22ef0690 0x55eb22ef8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22efb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22e4ec80 con 0x55eb22eefce0 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22e5e320 con 0x55eb22ef0690 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feac7c41740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22e5b850 con 0x55eb22ef9210 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22eefce0 0x55eb22ef00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22eefce0 0x55eb22ef00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38214/0 (socket says 172.21.15.17:38214) 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 -- 172.21.15.17:0/2314483598 learned_addr learned my addr 172.21.15.17:0/2314483598 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabed9d640 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22ef0690 0x55eb22ef8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feaa7fff640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feaa80a2350 0x7feaa80a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabe59c640 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22efb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feaa80a2350 0x7feaa80a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feaa7fff640 1 -- 172.21.15.17:0/3832027264 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7feab00ce060 con 0x7feac0102450 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 -- 172.21.15.17:0/2314483598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 msgr2=0x55eb22efb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22efb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 -- 172.21.15.17:0/2314483598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22ef0690 msgr2=0x55eb22ef8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22ef0690 0x55eb22ef8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.271+0000 7feabf59e640 1 -- 172.21.15.17:0/2314483598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb22e1de30 con 0x55eb22eefce0 2024-02-14T15:04:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.272+0000 7feabf59e640 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22eefce0 0x55eb22ef00c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feab80139d0 tx=0x7feab8013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a34f97caf02e18 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.272+0000 7feaa57fa640 1 -- 172.21.15.17:0/2314483598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8013050 con 0x55eb22eefce0 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.272+0000 7feaa57fa640 1 -- 172.21.15.17:0/2314483598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feab8019070 con 0x55eb22eefce0 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.272+0000 7feaa57fa640 1 -- 172.21.15.17:0/2314483598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab80126b0 con 0x55eb22eefce0 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.272+0000 7feabf59e640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feaa80a2350 0x7feaa80a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feab8021000 tx=0x7feab801f000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- 172.21.15.17:0/2314483598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22eefce0 msgr2=0x55eb22ef00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22eefce0 0x55eb22ef00c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feab80139d0 tx=0x7feab8013e90 comp rx=0 tx=0).stop 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- 172.21.15.17:0/2314483598 shutdown_connections 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22efb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22ef0690 0x55eb22ef8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 --2- 172.21.15.17:0/2314483598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22eefce0 0x55eb22ef00c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- 172.21.15.17:0/2314483598 >> 172.21.15.17:0/2314483598 conn(0x55eb22eea3b0 msgr2=0x55eb22eea790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- 172.21.15.17:0/2314483598 shutdown_connections 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- 172.21.15.17:0/2314483598 wait complete. 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- start start 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 Processor -- start 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ef6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22ef6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22e57030 con 0x55eb22ef0690 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22b2a430 con 0x55eb22eefce0 2024-02-14T15:04:20.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22ae6e00 con 0x55eb22ef9210 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feac7c41740 10 client.0 _ll_register_callbacks cb 0x55eb22b67be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feabe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ef6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feabe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ef6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38222/0 (socket says 172.21.15.17:38222) 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feabe59c640 1 -- 172.21.15.17:0/3109191110 learned_addr learned my addr 172.21.15.17:0/3109191110 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.273+0000 7feabf59e640 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22ef6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feabed9d640 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feabe59c640 1 -- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 msgr2=0x55eb22ef6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feabe59c640 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22ef6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feabe59c640 1 -- 172.21.15.17:0/3109191110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 msgr2=0x55eb22ef6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feabe59c640 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feabe59c640 1 -- 172.21.15.17:0/3109191110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab8013640 con 0x55eb22ef0690 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feabe59c640 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ef6740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feab4005ec0 tx=0x7feab402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eb9f7940baad70 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feaa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feaa2ff5640 1 -- 172.21.15.17:0/3109191110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab402cce0 con 0x55eb22ef0690 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feaa2ff5640 1 -- 172.21.15.17:0/3109191110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7feab4045070 con 0x55eb22ef0690 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb22b567e0 con 0x55eb22ef0690 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feaa2ff5640 1 -- 172.21.15.17:0/3109191110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab4004250 con 0x55eb22ef0690 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feac7c41740 10 client.16005 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.274+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55eb22e556b0 con 0x55eb22ef0690 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feaa2ff5640 1 -- 172.21.15.17:0/3109191110 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55eb22b567e0 con 0x55eb22ef0690 2024-02-14T15:04:20.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feaa2ff5640 1 -- 172.21.15.17:0/3109191110 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feab403c040 con 0x55eb22ef0690 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feac7c41740 10 client.16005 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feac7c41740 10 client.16005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feac7c41740 20 client.16005 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feac7c41740 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f09470 0x55eb22f0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feac7c41740 4 client.16005 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb22b93720 con 0x55eb22f09470 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.275+0000 7feabed9d640 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f09470 0x55eb22f0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feaa2ff5640 10 client.16005 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feabed9d640 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f09470 0x55eb22f0b890 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1c2d47d75111cd7 server_cookie=fa3b329ba00647e8 in_seq=0 out_seq=0 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feaa2ff5640 1 -- 172.21.15.17:0/3109191110 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb22b93720 con 0x55eb22f09470 2024-02-14T15:04:20.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feaa2ff5640 10 client.16005 handle_command_reply: tid=0 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 client.16005 shutdown 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 msgr2=0x55eb22ef6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ef6740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feab4005ec0 tx=0x7feab402cbf0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 shutdown_connections 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f09470 0x55eb22f0b890 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ef6740 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22ef6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feaa2ff5640 0 client.16005 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 --2- 172.21.15.17:0/3109191110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 >> 172.21.15.17:0/3109191110 conn(0x55eb22eea3b0 msgr2=0x55eb22eec100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 shutdown_connections 2024-02-14T15:04:20.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.276+0000 7feac7c41740 1 -- 172.21.15.17:0/3109191110 wait complete. 2024-02-14T15:04:20.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.277+0000 7feac7c41740 20 client.16005 trim_cache size 0 max 16384 2024-02-14T15:04:20.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.288+0000 7feac7c41740 1 Processor -- start 2024-02-14T15:04:20.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.288+0000 7feac7c41740 1 -- start start 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.288+0000 7feac7c41740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.288+0000 7feac7c41740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ee9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feac7c41740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22f00240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feac7c41740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22ddc130 con 0x55eb22ef0690 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feac7c41740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22f10ae0 con 0x55eb22eefce0 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feac7c41740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22eeb140 con 0x55eb22ef9210 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ee9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ee9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38230/0 (socket says 172.21.15.17:38230) 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 -- 172.21.15.17:0/222056611 learned_addr learned my addr 172.21.15.17:0/222056611 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabf59e640 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22f00240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabed9d640 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 -- 172.21.15.17:0/222056611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 msgr2=0x55eb22f00240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22f00240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 -- 172.21.15.17:0/222056611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 msgr2=0x55eb22ef6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 -- 172.21.15.17:0/222056611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb22df1c90 con 0x55eb22ef0690 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feabe59c640 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ee9350 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feab4009850 tx=0x7feab402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d41c8bc3ee5b23 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feaa4ff9640 1 -- 172.21.15.17:0/222056611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab402c6b0 con 0x55eb22ef0690 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feaa4ff9640 1 -- 172.21.15.17:0/222056611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feab4043070 con 0x55eb22ef0690 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.289+0000 7feaa4ff9640 1 -- 172.21.15.17:0/222056611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40052d0 con 0x55eb22ef0690 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 -- 172.21.15.17:0/222056611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 msgr2=0x55eb22ee9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ee9350 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feab4009850 tx=0x7feab402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 -- 172.21.15.17:0/222056611 shutdown_connections 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef0690 0x55eb22ee9350 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef9210 0x55eb22f00240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 --2- 172.21.15.17:0/222056611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22ef6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 -- 172.21.15.17:0/222056611 >> 172.21.15.17:0/222056611 conn(0x55eb22eea3b0 msgr2=0x55eb22ee8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 -- 172.21.15.17:0/222056611 shutdown_connections 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.290+0000 7feac7c41740 1 -- 172.21.15.17:0/222056611 wait complete. 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 -- start start 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 Processor -- start 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22effa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef0690 0x55eb22efff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef9210 0x55eb22f58c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22f59190 con 0x55eb22ef9210 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22f59300 con 0x55eb22eefce0 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb22f594c0 con 0x55eb22ef0690 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feac7c41740 10 client.0 _ll_register_callbacks cb 0x7feab8007930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feabf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef9210 0x55eb22f58c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feabf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef9210 0x55eb22f58c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38244/0 (socket says 172.21.15.17:38244) 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feabf59e640 1 -- 172.21.15.17:0/2791179558 learned_addr learned my addr 172.21.15.17:0/2791179558 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feabe59c640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef0690 0x55eb22efff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.291+0000 7feabed9d640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22effa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feabf59e640 1 -- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef0690 msgr2=0x55eb22efff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feabf59e640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef0690 0x55eb22efff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feabf59e640 1 -- 172.21.15.17:0/2791179558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 msgr2=0x55eb22effa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feabf59e640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22effa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feabf59e640 1 -- 172.21.15.17:0/2791179558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb22e556b0 con 0x55eb22ef9210 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feabed9d640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22effa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feabf59e640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef9210 0x55eb22f58c50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feab8007f50 tx=0x7feab8007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223806bc0fa4ccbe server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feaa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feaa2ff5640 1 -- 172.21.15.17:0/2791179558 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8039070 con 0x55eb22ef9210 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feaa2ff5640 1 -- 172.21.15.17:0/2791179558 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7feab8003b80 con 0x55eb22ef9210 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feab8019070 con 0x55eb22ef9210 2024-02-14T15:04:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feaa2ff5640 1 -- 172.21.15.17:0/2791179558 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab802d040 con 0x55eb22ef9210 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feac7c41740 10 client.16011 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55eb22e17aa0 con 0x55eb22ef9210 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feaa2ff5640 1 -- 172.21.15.17:0/2791179558 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feab8019070 con 0x55eb22ef9210 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.292+0000 7feaa2ff5640 1 -- 172.21.15.17:0/2791179558 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feab803c020 con 0x55eb22ef9210 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feac7c41740 10 client.16011 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feac7c41740 10 client.16011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feac7c41740 20 client.16011 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feac7c41740 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f62040 0x55eb22f64460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feac7c41740 4 client.16011 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb22be2890 con 0x55eb22f62040 2024-02-14T15:04:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feabed9d640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f62040 0x55eb22f64460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feaa2ff5640 10 client.16011 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feabed9d640 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f62040 0x55eb22f64460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48cd6567a64497b1 server_cookie=20b865695132b036 in_seq=0 out_seq=0 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feaa2ff5640 1 -- 172.21.15.17:0/2791179558 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb22be2890 con 0x55eb22f62040 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feaa2ff5640 10 client.16011 handle_command_reply: tid=0 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.293+0000 7feac7c41740 1 client.16011 shutdown 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef9210 msgr2=0x55eb22f58c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef9210 0x55eb22f58c50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feab8007f50 tx=0x7feab8007b30 comp rx=0 tx=0).stop 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 shutdown_connections 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55eb22f62040 0x55eb22f64460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eb22ef9210 0x55eb22f58c50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb22ef0690 0x55eb22efff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feaa2ff5640 0 client.16011 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:20.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 --2- 172.21.15.17:0/2791179558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55eb22eefce0 0x55eb22effa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 >> 172.21.15.17:0/2791179558 conn(0x55eb22eea3b0 msgr2=0x55eb22eebf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 shutdown_connections 2024-02-14T15:04:20.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 1 -- 172.21.15.17:0/2791179558 wait complete. 2024-02-14T15:04:20.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac7c41740 20 client.16011 trim_cache size 0 max 16384 2024-02-14T15:04:20.306 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:04:20.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feaa80a2350 msgr2=0x7feaa80a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feaa80a2350 0x7feaa80a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feab8021000 tx=0x7feab801f000 comp rx=0 tx=0).stop 2024-02-14T15:04:20.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102450 msgr2=0x7feac019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102450 0x7feac019d430 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feab000d770 tx=0x7feab000dc30 comp rx=0 tx=0).stop 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 shutdown_connections 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feaa80a2350 0x7feaa80a4810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac010b820 0x7feac0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feac0102d70 0x7feac019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 --2- 172.21.15.17:0/3832027264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feac0102450 0x7feac019d430 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.294+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 >> 172.21.15.17:0/3832027264 conn(0x7feac0077250 msgr2=0x7feac00fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.295+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 shutdown_connections 2024-02-14T15:04:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.295+0000 7feac506f640 1 -- 172.21.15.17:0/3832027264 wait complete. 2024-02-14T15:04:20.310 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:04:20.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-14T15:04:20.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 -- 172.21.15.17:0/1317244935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec105410 msgr2=0x7f53ec10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 --2- 172.21.15.17:0/1317244935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec105410 0x7f53ec10cfe0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009cd0 tx=0x7f53e402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 -- 172.21.15.17:0/1317244935 shutdown_connections 2024-02-14T15:04:20.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 --2- 172.21.15.17:0/1317244935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec105410 0x7f53ec10cfe0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 --2- 172.21.15.17:0/1317244935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec100cf0 0x7f53ec104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 --2- 172.21.15.17:0/1317244935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ec100340 0x7f53ec100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 -- 172.21.15.17:0/1317244935 >> 172.21.15.17:0/1317244935 conn(0x7f53ec0772e0 msgr2=0x7f53ec0776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 -- 172.21.15.17:0/1317244935 shutdown_connections 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 -- 172.21.15.17:0/1317244935 wait complete. 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 Processor -- start 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.422+0000 7f53f1edd640 1 -- start start 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53f1edd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec100340 0x7f53ec19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53f1edd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ec105410 0x7f53ec19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53f1edd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec197630 0x7f53ec197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53f1edd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec198100 con 0x7f53ec197630 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53f1edd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec198270 con 0x7f53ec100340 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53f1edd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec198430 con 0x7f53ec105410 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec197630 0x7f53ec197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec197630 0x7f53ec197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38262/0 (socket says 172.21.15.17:38262) 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 -- 172.21.15.17:0/2547711833 learned_addr learned my addr 172.21.15.17:0/2547711833 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec100340 0x7f53ec19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ec105410 0x7f53ec19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 -- 172.21.15.17:0/2547711833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ec105410 msgr2=0x7f53ec19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ec105410 0x7f53ec19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 -- 172.21.15.17:0/2547711833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec100340 msgr2=0x7f53ec19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec100340 0x7f53ec19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 -- 172.21.15.17:0/2547711833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e4009900 con 0x7f53ec197630 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ec105410 0x7f53ec19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53ebfff640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec197630 0x7f53ec197ab0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f53e4004af0 tx=0x7f53e4032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.423+0000 7f53e8ff9640 1 -- 172.21.15.17:0/2547711833 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40328b0 con 0x7f53ec197630 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.424+0000 7f53e8ff9640 1 -- 172.21.15.17:0/2547711833 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53e4003e50 con 0x7f53ec197630 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.424+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec1a4370 con 0x7f53ec197630 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.424+0000 7f53e8ff9640 1 -- 172.21.15.17:0/2547711833 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4004740 con 0x7f53ec197630 2024-02-14T15:04:20.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.424+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec1a47f0 con 0x7f53ec197630 2024-02-14T15:04:20.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 Processor -- start 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53e8ff9640 1 -- 172.21.15.17:0/2547711833 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f53e400a040 con 0x7f53ec197630 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 -- start start 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcba60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbaec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcbaf210 0x55d5fcbb1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcb04c80 con 0x55d5fcba6690 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcb14320 con 0x55d5fcba5ce0 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.426+0000 7f53f4b81740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcb11850 con 0x55d5fcbaf210 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbaec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbaec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38276/0 (socket says 172.21.15.17:38276) 2024-02-14T15:04:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 -- 172.21.15.17:0/2043002008 learned_addr learned my addr 172.21.15.17:0/2043002008 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcbaf210 0x55d5fcbb1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcba60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 -- 172.21.15.17:0/2043002008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcbaf210 msgr2=0x55d5fcbb1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcbaf210 0x55d5fcbb1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 -- 172.21.15.17:0/2043002008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 msgr2=0x55d5fcba60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcba60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 -- 172.21.15.17:0/2043002008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5fcad3e30 con 0x55d5fcba6690 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcba60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53e8ff9640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53d40a2350 0x7f53d40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbaec40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00bd10 tx=0x7f53dc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdca96ea27cd449 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53d40a2350 0x7f53d40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53d1ffb640 1 -- 172.21.15.17:0/2043002008 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004030 con 0x55d5fcba6690 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53e8ff9640 1 -- 172.21.15.17:0/2547711833 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f53e40f0e40 con 0x7f53ec197630 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53d1ffb640 1 -- 172.21.15.17:0/2043002008 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc015070 con 0x55d5fcba6690 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.427+0000 7f53d1ffb640 1 -- 172.21.15.17:0/2043002008 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0116a0 con 0x55d5fcba6690 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 -- 172.21.15.17:0/2043002008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 msgr2=0x55d5fcbaec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbaec40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00bd10 tx=0x7f53dc0107a0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53d40a2350 0x7f53d40a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53ec198e80 tx=0x7f53e0005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 -- 172.21.15.17:0/2043002008 shutdown_connections 2024-02-14T15:04:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbaec40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcbaf210 0x55d5fcbb1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2043002008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcba60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 -- 172.21.15.17:0/2043002008 >> 172.21.15.17:0/2043002008 conn(0x55d5fcba03b0 msgr2=0x55d5fcba0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 -- 172.21.15.17:0/2043002008 shutdown_connections 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 -- 172.21.15.17:0/2043002008 wait complete. 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.428+0000 7f53f4b81740 1 -- start start 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 1 Processor -- start 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcbaf210 0x55d5fcbacc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcb0d030 con 0x55d5fcba6690 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fc7e0430 con 0x55d5fcbaf210 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fc79ce00 con 0x55d5fcba5ce0 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53f4b81740 10 client.0 _ll_register_callbacks cb 0x55d5fc81dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53eaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53eb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53eaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38280/0 (socket says 172.21.15.17:38280) 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53eaffd640 1 -- 172.21.15.17:0/2551814685 learned_addr learned my addr 172.21.15.17:0/2551814685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53ebfff640 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcbaf210 0x55d5fcbacc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.429+0000 7f53eb7fe640 1 -- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcbaf210 msgr2=0x55d5fcbacc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcbaf210 0x55d5fcbacc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53eb7fe640 1 -- 172.21.15.17:0/2551814685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 msgr2=0x55d5fcbac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53eb7fe640 1 -- 172.21.15.17:0/2551814685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc00b940 con 0x55d5fcba5ce0 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbac1d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55d5fcba1420 tx=0x7f53dc0058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7eb48b9703db226 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53caffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53caffd640 1 -- 172.21.15.17:0/2551814685 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc020070 con 0x55d5fcba5ce0 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5fc80c7e0 con 0x55d5fcba5ce0 2024-02-14T15:04:20.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53caffd640 1 -- 172.21.15.17:0/2551814685 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f53dc017070 con 0x55d5fcba5ce0 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53caffd640 1 -- 172.21.15.17:0/2551814685 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53dc003880 con 0x55d5fcba5ce0 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53f4b81740 10 client.25579 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d5fcb0b6b0 con 0x55d5fcba5ce0 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.430+0000 7f53caffd640 1 -- 172.21.15.17:0/2551814685 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d5fc80c7e0 con 0x55d5fcba5ce0 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53caffd640 1 -- 172.21.15.17:0/2551814685 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53dc0046c0 con 0x55d5fcba5ce0 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53f4b81740 10 client.25579 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53f4b81740 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53f4b81740 20 client.25579 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbbf2d0 0x55d5fcbc16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53f4b81740 4 client.25579 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5fc849720 con 0x55d5fcbbf2d0 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbbf2d0 0x55d5fcbc16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53caffd640 10 client.25579 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.431+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbbf2d0 0x55d5fcbc16e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d13993fcf42b8c server_cookie=fc64cbfe898a0818 in_seq=0 out_seq=0 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53caffd640 1 -- 172.21.15.17:0/2551814685 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5fc849720 con 0x55d5fcbbf2d0 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53caffd640 10 client.25579 handle_command_reply: tid=0 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 client.25579 shutdown 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba5ce0 msgr2=0x55d5fcbac1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbac1d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55d5fcba1420 tx=0x7f53dc0058c0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 shutdown_connections 2024-02-14T15:04:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbbf2d0 0x55d5fcbc16e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcbaf210 0x55d5fcbacc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53caffd640 0 client.25579 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2551814685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbac1d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 >> 172.21.15.17:0/2551814685 conn(0x55d5fcba03b0 msgr2=0x55d5fcba2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 shutdown_connections 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 1 -- 172.21.15.17:0/2551814685 wait complete. 2024-02-14T15:04:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.432+0000 7f53f4b81740 20 client.25579 trim_cache size 0 max 16384 2024-02-14T15:04:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.444+0000 7f53f4b81740 1 Processor -- start 2024-02-14T15:04:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.444+0000 7f53f4b81740 1 -- start start 2024-02-14T15:04:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcbae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbaca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcba9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53f4b81740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fca92130 con 0x55d5fcba6690 2024-02-14T15:04:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53f4b81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcb9bc70 con 0x55d5fcba5ce0 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53f4b81740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcbbf080 con 0x55d5fcba03b0 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcba9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcba9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38286/0 (socket says 172.21.15.17:38286) 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 -- 172.21.15.17:0/547211483 learned_addr learned my addr 172.21.15.17:0/547211483 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53eaffd640 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbaca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcbae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 -- 172.21.15.17:0/547211483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 msgr2=0x55d5fcbae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcbae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 -- 172.21.15.17:0/547211483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 msgr2=0x55d5fcbaca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbaca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 -- 172.21.15.17:0/547211483 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5fcaa7c90 con 0x55d5fcba6690 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcbae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53ebfff640 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcba9470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f53e40bd440 tx=0x7f53e40bcff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e4bf5b4c0c9317 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.445+0000 7f53d17fa640 1 -- 172.21.15.17:0/547211483 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40d5070 con 0x55d5fcba6690 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53d17fa640 1 -- 172.21.15.17:0/547211483 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53e40bd4b0 con 0x55d5fcba6690 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53d17fa640 1 -- 172.21.15.17:0/547211483 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40bd710 con 0x55d5fcba6690 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 -- 172.21.15.17:0/547211483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 msgr2=0x55d5fcba9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcba9470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f53e40bd440 tx=0x7f53e40bcff0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 -- 172.21.15.17:0/547211483 shutdown_connections 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcba9470 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcbaca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 --2- 172.21.15.17:0/547211483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcbae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 -- 172.21.15.17:0/547211483 >> 172.21.15.17:0/547211483 conn(0x55d5fcba6de0 msgr2=0x55d5fcba74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 -- 172.21.15.17:0/547211483 shutdown_connections 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.446+0000 7f53f4b81740 1 -- 172.21.15.17:0/547211483 wait complete. 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 -- start start 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 Processor -- start 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcc0fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcc100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbb5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcbb5a60 con 0x55d5fcba6690 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcbb5bd0 con 0x55d5fcba5ce0 2024-02-14T15:04:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fcbb5d90 con 0x55d5fcba03b0 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbb5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53f4b81740 10 client.0 _ll_register_callbacks cb 0x7f53dc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbb5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38298/0 (socket says 172.21.15.17:38298) 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 -- 172.21.15.17:0/2502745372 learned_addr learned my addr 172.21.15.17:0/2502745372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53eaffd640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcc100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcc0fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 -- 172.21.15.17:0/2502745372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 msgr2=0x55d5fcc0fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcc0fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 -- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 msgr2=0x55d5fcc100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcc100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53ebfff640 1 -- 172.21.15.17:0/2502745372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec1a47f0 con 0x55d5fcba6690 2024-02-14T15:04:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.447+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcc0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53ebfff640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbb5420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009ca0 tx=0x7f53e402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa63ac786215574e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53caffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53caffd640 1 -- 172.21.15.17:0/2502745372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40d5070 con 0x55d5fcba6690 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53caffd640 1 -- 172.21.15.17:0/2502745372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f53e40c1600 con 0x55d5fcba6690 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53dc015070 con 0x55d5fcba6690 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53caffd640 1 -- 172.21.15.17:0/2502745372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e40bd710 con 0x55d5fcba6690 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 10 client.16029 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d5fcacdaa0 con 0x55d5fcba6690 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53caffd640 1 -- 172.21.15.17:0/2502745372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc015070 con 0x55d5fcba6690 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53caffd640 1 -- 172.21.15.17:0/2502745372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53e4050020 con 0x55d5fcba6690 2024-02-14T15:04:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 10 client.16029 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 20 client.16029 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbaf210 0x55d5fcc19580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 4 client.16029 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.448+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5fc898890 con 0x55d5fcbaf210 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.449+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbaf210 0x55d5fcc19580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.449+0000 7f53eb7fe640 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbaf210 0x55d5fcc19580 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1740057a7c1200 server_cookie=80156553790b11ea in_seq=0 out_seq=0 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.449+0000 7f53caffd640 10 client.16029 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.449+0000 7f53caffd640 1 -- 172.21.15.17:0/2502745372 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5fc898890 con 0x55d5fcbaf210 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.449+0000 7f53caffd640 10 client.16029 handle_command_reply: tid=0 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.449+0000 7f53f4b81740 1 client.16029 shutdown 2024-02-14T15:04:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 msgr2=0x55d5fcbb5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbb5420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009ca0 tx=0x7f53e402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 shutdown_connections 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d5fcbaf210 0x55d5fcc19580 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d5fcba6690 0x55d5fcbb5420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d5fcba5ce0 0x55d5fcc100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53caffd640 0 client.16029 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 --2- 172.21.15.17:0/2502745372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d5fcba03b0 0x55d5fcc0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 >> 172.21.15.17:0/2502745372 conn(0x55d5fcba6de0 msgr2=0x55d5fcb9ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 shutdown_connections 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 1 -- 172.21.15.17:0/2502745372 wait complete. 2024-02-14T15:04:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f4b81740 20 client.16029 trim_cache size 0 max 16384 2024-02-14T15:04:20.463 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:04:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53d40a2350 msgr2=0x7f53d40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53d40a2350 0x7f53d40a4810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53ec198e80 tx=0x7f53e0005f90 comp rx=0 tx=0).stop 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec197630 msgr2=0x7f53ec197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec197630 0x7f53ec197ab0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f53e4004af0 tx=0x7f53e4032570 comp rx=0 tx=0).stop 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 shutdown_connections 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f53d40a2350 0x7f53d40a4810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53ec197630 0x7f53ec197ab0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ec105410 0x7f53ec19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 --2- 172.21.15.17:0/2547711833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53ec100340 0x7f53ec19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 >> 172.21.15.17:0/2547711833 conn(0x7f53ec0772e0 msgr2=0x7f53ec0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 shutdown_connections 2024-02-14T15:04:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:20.450+0000 7f53f1edd640 1 -- 172.21.15.17:0/2547711833 wait complete. 2024-02-14T15:04:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:20 smithi050 ceph-mon[36512]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-14T15:04:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:20 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3300862140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:04:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:20 smithi076 ceph-mon[37057]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-14T15:04:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:20 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3300862140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:04:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:20 smithi017 ceph-mon[32549]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-14T15:04:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:20 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3300862140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:04:21.466 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-14T15:04:21.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:04:21.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-14T15:04:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 -- 172.21.15.17:0/4127688127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 msgr2=0x7f64b010dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 --2- 172.21.15.17:0/4127688127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b010dc50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009ca0 tx=0x7f64a802c180 comp rx=0 tx=0).stop 2024-02-14T15:04:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 -- 172.21.15.17:0/4127688127 shutdown_connections 2024-02-14T15:04:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 --2- 172.21.15.17:0/4127688127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b010dc50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 --2- 172.21.15.17:0/4127688127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b0102d70 0x7f64b010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 --2- 172.21.15.17:0/4127688127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64b0102450 0x7f64b0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 -- 172.21.15.17:0/4127688127 >> 172.21.15.17:0/4127688127 conn(0x7f64b00772e0 msgr2=0x7f64b00776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 -- 172.21.15.17:0/4127688127 shutdown_connections 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 -- 172.21.15.17:0/4127688127 wait complete. 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 Processor -- start 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.572+0000 7f64b669a640 1 -- start start 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b669a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b0102450 0x7f64b01000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b669a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64b0102d70 0x7f64b01005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b669a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b0100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b669a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b01a5b80 con 0x7f64b010b820 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b669a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b01a5cf0 con 0x7f64b0102450 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b669a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b01a5eb0 con 0x7f64b0102d70 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b0100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b0100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47366/0 (socket says 172.21.15.17:47366) 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 -- 172.21.15.17:0/3888641246 learned_addr learned my addr 172.21.15.17:0/3888641246 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64affff640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b0102450 0x7f64b01000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64af7fe640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64b0102d70 0x7f64b01005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 -- 172.21.15.17:0/3888641246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64b0102d70 msgr2=0x7f64b01005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64b0102d70 0x7f64b01005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 -- 172.21.15.17:0/3888641246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b0102450 msgr2=0x7f64b01000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b0102450 0x7f64b01000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 -- 172.21.15.17:0/3888641246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009900 con 0x7f64b010b820 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.573+0000 7f64b4c37640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b0100b20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f64a802c790 tx=0x7f64a8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.574+0000 7f64ad7fa640 1 -- 172.21.15.17:0/3888641246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8003d80 con 0x7f64b010b820 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.574+0000 7f64ad7fa640 1 -- 172.21.15.17:0/3888641246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64a8032c70 con 0x7f64b010b820 2024-02-14T15:04:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.574+0000 7f64ad7fa640 1 -- 172.21.15.17:0/3888641246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80047f0 con 0x7f64b010b820 2024-02-14T15:04:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.574+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b01a6130 con 0x7f64b010b820 2024-02-14T15:04:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.574+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b01a6590 con 0x7f64b010b820 2024-02-14T15:04:21.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.576+0000 7f64b926b740 1 Processor -- start 2024-02-14T15:04:21.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.576+0000 7f64ad7fa640 1 -- 172.21.15.17:0/3888641246 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f64a800a040 con 0x7f64b010b820 2024-02-14T15:04:21.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64b926b740 1 -- start start 2024-02-14T15:04:21.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64b926b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64b926b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c30a10 0x55d940c390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64b926b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c395f0 0x55d940c3b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64b926b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940bebbd0 con 0x55d940c300f0 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64b926b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940b9e030 con 0x55d940c395f0 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64b926b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940b9b560 con 0x55d940c30a10 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47374/0 (socket says 172.21.15.17:47374) 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 -- 172.21.15.17:0/1823553355 learned_addr learned my addr 172.21.15.17:0/1823553355 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c395f0 0x55d940c3b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64affff640 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c30a10 0x55d940c390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 -- 172.21.15.17:0/1823553355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c30a10 msgr2=0x55d940c390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64ad7fa640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64980a2380 0x7f64980a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c30a10 0x55d940c390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 -- 172.21.15.17:0/1823553355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c395f0 msgr2=0x55d940c3b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c395f0 0x55d940c3b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.577+0000 7f64af7fe640 1 -- 172.21.15.17:0/1823553355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d940b59290 con 0x55d940c300f0 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.578+0000 7f64b4c37640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64980a2380 0x7f64980a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.578+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c304d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f64a4014880 tx=0x7f64a400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb06da7271fddd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.578+0000 7f64ad7fa640 1 -- 172.21.15.17:0/3888641246 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f64a80f0ea0 con 0x7f64b010b820 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.578+0000 7f64967fc640 1 -- 172.21.15.17:0/1823553355 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a40140d0 con 0x55d940c300f0 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.578+0000 7f64967fc640 1 -- 172.21.15.17:0/1823553355 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64a401a070 con 0x55d940c300f0 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.578+0000 7f64967fc640 1 -- 172.21.15.17:0/1823553355 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004e40 con 0x55d940c300f0 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b4c37640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64980a2380 0x7f64980a4840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64a80ee280 tx=0x7f64a80b9040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 -- 172.21.15.17:0/1823553355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 msgr2=0x55d940c304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c304d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f64a4014880 tx=0x7f64a400b400 comp rx=0 tx=0).stop 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 -- 172.21.15.17:0/1823553355 shutdown_connections 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c395f0 0x55d940c3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c30a10 0x55d940c390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 --2- 172.21.15.17:0/1823553355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c304d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 -- 172.21.15.17:0/1823553355 >> 172.21.15.17:0/1823553355 conn(0x55d940c2a820 msgr2=0x55d940c2ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 -- 172.21.15.17:0/1823553355 shutdown_connections 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 -- 172.21.15.17:0/1823553355 wait complete. 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 -- start start 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.579+0000 7f64b926b740 1 Processor -- start 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c36b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 0x55d940c370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940b96d40 con 0x55d940c300f0 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940b3be60 con 0x55d940c30a10 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94086a4b0 con 0x55d940c395f0 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 10 client.0 _ll_register_callbacks cb 0x55d940b528d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c36640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47376/0 (socket says 172.21.15.17:47376) 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 -- 172.21.15.17:0/2218243798 learned_addr learned my addr 172.21.15.17:0/2218243798 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64affff640 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c36b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 -- 172.21.15.17:0/2218243798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 msgr2=0x55d940c370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 0x55d940c370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 -- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 msgr2=0x55d940c36b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c36b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 -- 172.21.15.17:0/2218243798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a4014530 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64af7fe640 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c36640 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c010 tx=0x7f64a40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eef0e5e85768ee9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f648f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f648f7fe640 1 -- 172.21.15.17:0/2218243798 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004030 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f648f7fe640 1 -- 172.21.15.17:0/2218243798 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f64a4004190 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.580+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d940bbcdd0 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f648f7fe640 1 -- 172.21.15.17:0/2218243798 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64a4004350 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 10 client.16041 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d940b953c0 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f648f7fe640 1 -- 172.21.15.17:0/2218243798 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d940bbcdd0 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f648f7fe640 1 -- 172.21.15.17:0/2218243798 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64a4028030 con 0x55d940c300f0 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 10 client.16041 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:21.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 20 client.16041 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940c494d0 0x55d940c4b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 4 client.16041 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.581+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d940b69cb0 con 0x55d940c494d0 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.582+0000 7f64affff640 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940c494d0 0x55d940c4b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.582+0000 7f64affff640 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940c494d0 0x55d940c4b8f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733ce9b47b34aa20 server_cookie=b3d65bdbacfaa59b in_seq=0 out_seq=0 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.582+0000 7f648f7fe640 10 client.16041 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f648f7fe640 1 -- 172.21.15.17:0/2218243798 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d940b69cb0 con 0x55d940c494d0 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f648f7fe640 10 client.16041 handle_command_reply: tid=0 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 client.16041 shutdown 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 msgr2=0x55d940c36640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c36640 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c010 tx=0x7f64a40047b0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 shutdown_connections 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940c494d0 0x55d940c4b8f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 0x55d940c370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c36b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 --2- 172.21.15.17:0/2218243798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c36640 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f648f7fe640 0 client.16041 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 >> 172.21.15.17:0/2218243798 conn(0x55d940c2a820 msgr2=0x55d940c29a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 shutdown_connections 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 1 -- 172.21.15.17:0/2218243798 wait complete. 2024-02-14T15:04:21.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.583+0000 7f64b926b740 20 client.16041 trim_cache size 0 max 16384 2024-02-14T15:04:21.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.601+0000 7f64b926b740 1 Processor -- start 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.601+0000 7f64b926b740 1 -- start start 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64b926b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c38d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64b926b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c31970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64b926b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 0x55d940c31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64b926b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940b1c670 con 0x55d940c300f0 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64b926b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940c264d0 con 0x55d940c30a10 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64b926b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940c34bc0 con 0x55d940c395f0 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c38d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c38d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47384/0 (socket says 172.21.15.17:47384) 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 -- 172.21.15.17:0/3066839086 learned_addr learned my addr 172.21.15.17:0/3066839086 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 0x55d940c31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64affff640 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c31970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 -- 172.21.15.17:0/3066839086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 msgr2=0x55d940c31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 0x55d940c31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 -- 172.21.15.17:0/3066839086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 msgr2=0x55d940c31970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c31970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 -- 172.21.15.17:0/3066839086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d940b31c70 con 0x55d940c300f0 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64affff640 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c31970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f64af7fe640 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c38d70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f64a400b440 tx=0x7f64a4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19165a7ab0c373f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.602+0000 7f6495ffb640 1 -- 172.21.15.17:0/3066839086 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004820 con 0x55d940c300f0 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f6495ffb640 1 -- 172.21.15.17:0/3066839086 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64a40043b0 con 0x55d940c300f0 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f6495ffb640 1 -- 172.21.15.17:0/3066839086 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a400f930 con 0x55d940c300f0 2024-02-14T15:04:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 -- 172.21.15.17:0/3066839086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 msgr2=0x55d940c38d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c38d70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f64a400b440 tx=0x7f64a4004680 comp rx=0 tx=0).stop 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 -- 172.21.15.17:0/3066839086 shutdown_connections 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c395f0 0x55d940c31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c31970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 --2- 172.21.15.17:0/3066839086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c300f0 0x55d940c38d70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 -- 172.21.15.17:0/3066839086 >> 172.21.15.17:0/3066839086 conn(0x55d940c2a820 msgr2=0x55d940c50210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 -- 172.21.15.17:0/3066839086 shutdown_connections 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.603+0000 7f64b926b740 1 -- 172.21.15.17:0/3066839086 wait complete. 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 -- start start 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 Processor -- start 2024-02-14T15:04:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c300f0 0x55d940c3ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c3f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c395f0 0x55d940c3f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940c9f9a0 con 0x55d940c395f0 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940c9fae0 con 0x55d940c30a10 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940c9fca0 con 0x55d940c300f0 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64b926b740 10 client.0 _ll_register_callbacks cb 0x7f64a400be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c395f0 0x55d940c3f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c395f0 0x55d940c3f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47390/0 (socket says 172.21.15.17:47390) 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 -- 172.21.15.17:0/1798690097 learned_addr learned my addr 172.21.15.17:0/1798690097 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c300f0 0x55d940c3ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64affff640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c3f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 -- 172.21.15.17:0/1798690097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c300f0 msgr2=0x55d940c3ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c300f0 0x55d940c3ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 -- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 msgr2=0x55d940c3f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c3f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64af7fe640 1 -- 172.21.15.17:0/1798690097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d940b953c0 con 0x55d940c395f0 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.604+0000 7f64affff640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c3f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64af7fe640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c395f0 0x55d940c3f9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f64a4005c50 tx=0x7f64a400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7173053ac9bd8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f648f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f648f7fe640 1 -- 172.21.15.17:0/1798690097 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4005230 con 0x55d940c395f0 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f648f7fe640 1 -- 172.21.15.17:0/1798690097 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f64a4000e40 con 0x55d940c395f0 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64a401a070 con 0x55d940c395f0 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f648f7fe640 1 -- 172.21.15.17:0/1798690097 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64a4005390 con 0x55d940c395f0 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 10 client.16047 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:21.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d940b57af0 con 0x55d940c395f0 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f648f7fe640 1 -- 172.21.15.17:0/1798690097 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64a401a070 con 0x55d940c395f0 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f648f7fe640 1 -- 172.21.15.17:0/1798690097 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64a4028070 con 0x55d940c395f0 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 10 client.16047 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 20 client.16047 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940ca0f50 0x55d940ca3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 4 client.16047 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.605+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d940896860 con 0x55d940ca0f50 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.606+0000 7f64affff640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940ca0f50 0x55d940ca3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.606+0000 7f64affff640 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940ca0f50 0x55d940ca3340 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48d1c15462c99a4 server_cookie=34a6d8f2193f4542 in_seq=0 out_seq=0 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.606+0000 7f648f7fe640 10 client.16047 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f648f7fe640 1 -- 172.21.15.17:0/1798690097 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d940896860 con 0x55d940ca0f50 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f648f7fe640 10 client.16047 handle_command_reply: tid=0 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 client.16047 shutdown 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c395f0 msgr2=0x55d940c3f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c395f0 0x55d940c3f9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f64a4005c50 tx=0x7f64a400bd40 comp rx=0 tx=0).stop 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 shutdown_connections 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55d940ca0f50 0x55d940ca3340 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d940c395f0 0x55d940c3f9c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d940c30a10 0x55d940c3f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 --2- 172.21.15.17:0/1798690097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d940c300f0 0x55d940c3ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 >> 172.21.15.17:0/1798690097 conn(0x55d940c2a820 msgr2=0x55d940c31970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.607+0000 7f648f7fe640 0 client.16047 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 shutdown_connections 2024-02-14T15:04:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b926b740 1 -- 172.21.15.17:0/1798690097 wait complete. 2024-02-14T15:04:21.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b926b740 20 client.16047 trim_cache size 0 max 16384 2024-02-14T15:04:21.617 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:04:21.617 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2024-02-14T15:04:21.617 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_tag": "ca55f103-f801-48ca-88eb-b6c29bc571d2", 2024-02-14T15:04:21.617 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": "asynchronous" 2024-02-14T15:04:21.617 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:04:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64980a2380 msgr2=0x7f64980a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64980a2380 0x7f64980a4840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64a80ee280 tx=0x7f64a80b9040 comp rx=0 tx=0).stop 2024-02-14T15:04:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 msgr2=0x7f64b0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b0100b20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f64a802c790 tx=0x7f64a8034760 comp rx=0 tx=0).stop 2024-02-14T15:04:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 shutdown_connections 2024-02-14T15:04:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64980a2380 0x7f64980a4840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64b010b820 0x7f64b0100b20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64b0102d70 0x7f64b01005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 --2- 172.21.15.17:0/3888641246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64b0102450 0x7f64b01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 >> 172.21.15.17:0/3888641246 conn(0x7f64b00772e0 msgr2=0x7f64b00fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 shutdown_connections 2024-02-14T15:04:21.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.608+0000 7f64b669a640 1 -- 172.21.15.17:0/3888641246 wait complete. 2024-02-14T15:04:21.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:04:21.621 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:04:21.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 -- 172.21.15.17:0/1176491364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9360101420 msgr2=0x7f9360101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 --2- 172.21.15.17:0/1176491364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9360101420 0x7f9360101800 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9350009930 tx=0x7f935002bc30 comp rx=0 tx=0).stop 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 -- 172.21.15.17:0/1176491364 shutdown_connections 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 --2- 172.21.15.17:0/1176491364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93601064f0 0x7f936010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 --2- 172.21.15.17:0/1176491364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9360101dd0 0x7f9360105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 --2- 172.21.15.17:0/1176491364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9360101420 0x7f9360101800 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 -- 172.21.15.17:0/1176491364 >> 172.21.15.17:0/1176491364 conn(0x7f93600fcca0 msgr2=0x7f93600ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 -- 172.21.15.17:0/1176491364 shutdown_connections 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.724+0000 7f9364977640 1 -- 172.21.15.17:0/1176491364 wait complete. 2024-02-14T15:04:21.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 Processor -- start 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 -- start start 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9360101420 0x7f936019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93601064f0 0x7f936019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93601976f0 0x7f9360197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93601981c0 con 0x7f93601976f0 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9360198330 con 0x7f93601064f0 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f9364977640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93601984f0 con 0x7f9360101420 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93601976f0 0x7f9360197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93601976f0 0x7f9360197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47412/0 (socket says 172.21.15.17:47412) 2024-02-14T15:04:21.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 -- 172.21.15.17:0/3385335337 learned_addr learned my addr 172.21.15.17:0/3385335337 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935e59c640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9360101420 0x7f936019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935dd9b640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93601064f0 0x7f936019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 -- 172.21.15.17:0/3385335337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9360101420 msgr2=0x7f936019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9360101420 0x7f936019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 -- 172.21.15.17:0/3385335337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93601064f0 msgr2=0x7f936019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93601064f0 0x7f936019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.725+0000 7f935ed9d640 1 -- 172.21.15.17:0/3385335337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009560 con 0x7f93601976f0 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.726+0000 7f935ed9d640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93601976f0 0x7f9360197b70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f935800eaa0 tx=0x7f935800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.726+0000 7f934f7fe640 1 -- 172.21.15.17:0/3385335337 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935800cdb0 con 0x7f93601976f0 2024-02-14T15:04:21.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.726+0000 7f934f7fe640 1 -- 172.21.15.17:0/3385335337 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9358005020 con 0x7f93601976f0 2024-02-14T15:04:21.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.726+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93601a4490 con 0x7f93601976f0 2024-02-14T15:04:21.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.726+0000 7f934f7fe640 1 -- 172.21.15.17:0/3385335337 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580116b0 con 0x7f93601976f0 2024-02-14T15:04:21.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.726+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93601a4920 con 0x7f93601976f0 2024-02-14T15:04:21.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.728+0000 7f934f7fe640 1 -- 172.21.15.17:0/3385335337 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f9358005520 con 0x7f93601976f0 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.728+0000 7f936761d740 1 Processor -- start 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f936761d740 1 -- start start 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f934f7fe640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f93480a2350 0x7f93480a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f936761d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d39a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f936761d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f936761d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d44f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f936761d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133cf45b0 con 0x562133d42b30 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f936761d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133ca82b0 con 0x562133d39f50 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f936761d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133ca57e0 con 0x562133d39630 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d44f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935dd9b640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f93480a2350 0x7f93480a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d44f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47414/0 (socket says 172.21.15.17:47414) 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 -- 172.21.15.17:0/1305731287 learned_addr learned my addr 172.21.15.17:0/1305731287 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f934f7fe640 1 -- 172.21.15.17:0/3385335337 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f93580ce7f0 con 0x7f93601976f0 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935e59c640 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d39a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935dd9b640 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 -- 172.21.15.17:0/1305731287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 msgr2=0x562133d39a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d39a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 -- 172.21.15.17:0/1305731287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 msgr2=0x562133d425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935ed9d640 1 -- 172.21.15.17:0/1305731287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562133c63020 con 0x562133d42b30 2024-02-14T15:04:21.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.729+0000 7f935dd9b640 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f935ed9d640 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d44f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f93580cc1e0 tx=0x7f935801fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d08f6469a9efbf7 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f935dd9b640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f93480a2350 0x7f93480a4810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93540058a0 tx=0x7f935400b5b0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f934cff9640 1 -- 172.21.15.17:0/1305731287 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935809a640 con 0x562133d42b30 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f934cff9640 1 -- 172.21.15.17:0/1305731287 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f935809ace0 con 0x562133d42b30 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f934cff9640 1 -- 172.21.15.17:0/1305731287 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358005190 con 0x562133d42b30 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f936761d740 1 -- 172.21.15.17:0/1305731287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 msgr2=0x562133d44f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f936761d740 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d44f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f93580cc1e0 tx=0x7f935801fe50 comp rx=0 tx=0).stop 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f936761d740 1 -- 172.21.15.17:0/1305731287 shutdown_connections 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f936761d740 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d44f20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f936761d740 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f936761d740 1 --2- 172.21.15.17:0/1305731287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d39a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.730+0000 7f936761d740 1 -- 172.21.15.17:0/1305731287 >> 172.21.15.17:0/1305731287 conn(0x562133d33d60 msgr2=0x562133d34140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 -- 172.21.15.17:0/1305731287 shutdown_connections 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 -- 172.21.15.17:0/1305731287 wait complete. 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 -- start start 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 Processor -- start 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39630 0x562133d3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d40030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d42b30 0x562133d40570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133ca0fc0 con 0x562133d39f50 2024-02-14T15:04:21.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133974430 con 0x562133d39630 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133930e00 con 0x562133d42b30 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f936761d740 10 client.0 _ll_register_callbacks cb 0x562133c22870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d40030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d40030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47428/0 (socket says 172.21.15.17:47428) 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935dd9b640 1 -- 172.21.15.17:0/1030127657 learned_addr learned my addr 172.21.15.17:0/1030127657 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935ed9d640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d42b30 0x562133d40570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935dd9b640 1 -- 172.21.15.17:0/1030127657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d42b30 msgr2=0x562133d40570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935dd9b640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d42b30 0x562133d40570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935dd9b640 1 -- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39630 msgr2=0x562133d3faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.731+0000 7f935e59c640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39630 0x562133d3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f935dd9b640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39630 0x562133d3faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f935dd9b640 1 -- 172.21.15.17:0/1030127657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93601a4920 con 0x562133d39f50 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f935e59c640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39630 0x562133d3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f935dd9b640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d40030 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f93540107b0 tx=0x7f93540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cab66d17f2b594e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f9345ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f9345ffb640 1 -- 172.21.15.17:0/1030127657 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935401caa0 con 0x562133d39f50 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f9345ffb640 1 -- 172.21.15.17:0/1030127657 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f935401cc00 con 0x562133d39f50 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621339a07e0 con 0x562133d39f50 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f9345ffb640 1 -- 172.21.15.17:0/1030127657 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f935401cdc0 con 0x562133d39f50 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 10 client.16059 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562133c9f640 con 0x562133d39f50 2024-02-14T15:04:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f9345ffb640 1 -- 172.21.15.17:0/1030127657 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5621339a07e0 con 0x562133d39f50 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f9345ffb640 1 -- 172.21.15.17:0/1030127657 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9354028530 con 0x562133d39f50 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 10 client.16059 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 20 client.16059 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133d52e60 0x562133d552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 4 client.16059 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.732+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621339dd720 con 0x562133d52e60 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.733+0000 7f935e59c640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133d52e60 0x562133d552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.733+0000 7f935e59c640 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133d52e60 0x562133d552a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7f2645f957a189 server_cookie=d8aeda4e9c44b4f1 in_seq=0 out_seq=0 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.733+0000 7f9345ffb640 10 client.16059 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f9345ffb640 1 -- 172.21.15.17:0/1030127657 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5621339dd720 con 0x562133d52e60 2024-02-14T15:04:21.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f9345ffb640 10 client.16059 handle_command_reply: tid=0 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 client.16059 shutdown 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 msgr2=0x562133d40030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d40030 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f93540107b0 tx=0x7f93540046c0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 shutdown_connections 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133d52e60 0x562133d552a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d40030 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f9345ffb640 0 client.16059 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d42b30 0x562133d40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 --2- 172.21.15.17:0/1030127657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39630 0x562133d3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 >> 172.21.15.17:0/1030127657 conn(0x562133d33d60 msgr2=0x562133d35ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 shutdown_connections 2024-02-14T15:04:21.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 1 -- 172.21.15.17:0/1030127657 wait complete. 2024-02-14T15:04:21.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.734+0000 7f936761d740 20 client.16059 trim_cache size 0 max 16384 2024-02-14T15:04:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 Processor -- start 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 -- start start 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d42220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d3e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133c25fd0 con 0x562133d42b30 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621339dd720 con 0x562133d39f50 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f936761d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621339a07e0 con 0x562133d39630 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d3e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47440/0 (socket says 172.21.15.17:47440) 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 -- 172.21.15.17:0/4037832810 learned_addr learned my addr 172.21.15.17:0/4037832810 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935dd9b640 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d3e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935e59c640 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d42220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 -- 172.21.15.17:0/4037832810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 msgr2=0x562133d42220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d42220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 -- 172.21.15.17:0/4037832810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 msgr2=0x562133d3e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d3e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.749+0000 7f935ed9d640 1 -- 172.21.15.17:0/4037832810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562133c3bb30 con 0x562133d42b30 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f935ed9d640 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d3e900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f93580cbb70 tx=0x7f935809a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a505fafdc336794 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f9347fff640 1 -- 172.21.15.17:0/4037832810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935809a640 con 0x562133d42b30 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f9347fff640 1 -- 172.21.15.17:0/4037832810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93580a33d0 con 0x562133d42b30 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f9347fff640 1 -- 172.21.15.17:0/4037832810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358005390 con 0x562133d42b30 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 -- 172.21.15.17:0/4037832810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 msgr2=0x562133d3e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d3e900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f93580cbb70 tx=0x7f935809a940 comp rx=0 tx=0).stop 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 -- 172.21.15.17:0/4037832810 shutdown_connections 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d42b30 0x562133d3e900 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d39f50 0x562133d3e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 --2- 172.21.15.17:0/4037832810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d42220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 -- 172.21.15.17:0/4037832810 >> 172.21.15.17:0/4037832810 conn(0x562133d33d60 msgr2=0x562133d3add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 -- 172.21.15.17:0/4037832810 shutdown_connections 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.750+0000 7f936761d740 1 -- 172.21.15.17:0/4037832810 wait complete. 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 -- start start 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 Processor -- start 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d49650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d49b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d42b30 0x562133d47b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133d481f0 con 0x562133d39f50 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133d48360 con 0x562133d42b30 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133d48520 con 0x562133d39630 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f936761d740 10 client.0 _ll_register_callbacks cb 0x7f9358005820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d49b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d49b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47452/0 (socket says 172.21.15.17:47452) 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 -- 172.21.15.17:0/124699738 learned_addr learned my addr 172.21.15.17:0/124699738 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935e59c640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d49650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935ed9d640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d42b30 0x562133d47b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 -- 172.21.15.17:0/124699738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 msgr2=0x562133d49650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d49650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 -- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d42b30 msgr2=0x562133d47b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d42b30 0x562133d47b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 -- 172.21.15.17:0/124699738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562133c63020 con 0x562133d39f50 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935ed9d640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d42b30 0x562133d47b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f935dd9b640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d49b90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9354028980 tx=0x7f9354028740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c59ab0cb4b1553 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.751+0000 7f9345ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f9345ffb640 1 -- 172.21.15.17:0/124699738 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935400b390 con 0x562133d39f50 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f9345ffb640 1 -- 172.21.15.17:0/124699738 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f9354005040 con 0x562133d39f50 2024-02-14T15:04:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f935809ace0 con 0x562133d39f50 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f9345ffb640 1 -- 172.21.15.17:0/124699738 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9354005200 con 0x562133d39f50 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 10 client.16065 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562133c61880 con 0x562133d39f50 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f9345ffb640 1 -- 172.21.15.17:0/124699738 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f935809ace0 con 0x562133d39f50 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f9345ffb640 1 -- 172.21.15.17:0/124699738 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f935403c020 con 0x562133d39f50 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 10 client.16065 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 20 client.16065 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133daea10 0x562133db0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 4 client.16065 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:04:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562133a2c890 con 0x562133daea10 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.752+0000 7f935e59c640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133daea10 0x562133db0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f9345ffb640 10 client.16065 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f935e59c640 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133daea10 0x562133db0e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0674fc1dbb11ccf server_cookie=ec51bd16a366f08b in_seq=0 out_seq=0 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f9345ffb640 1 -- 172.21.15.17:0/124699738 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562133a2c890 con 0x562133daea10 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f9345ffb640 10 client.16065 handle_command_reply: tid=0 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 client.16065 shutdown 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 msgr2=0x562133d49b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d49b90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9354028980 tx=0x7f9354028740 comp rx=0 tx=0).stop 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 shutdown_connections 2024-02-14T15:04:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562133daea10 0x562133db0e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562133d39f50 0x562133d49b90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f9345ffb640 0 client.16065 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562133d42b30 0x562133d47b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 --2- 172.21.15.17:0/124699738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562133d39630 0x562133d49650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 >> 172.21.15.17:0/124699738 conn(0x562133d33d60 msgr2=0x562133d325d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 shutdown_connections 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 1 -- 172.21.15.17:0/124699738 wait complete. 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.753+0000 7f936761d740 20 client.16065 trim_cache size 0 max 16384 2024-02-14T15:04:21.765 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": { 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "ca55f103-f801-48ca-88eb-b6c29bc571d2": { 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/", 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "tag": "ca55f103-f801-48ca-88eb-b6c29bc571d2", 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "options": "recursive,force" 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:04:21.766 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f93480a2350 msgr2=0x7f93480a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f93480a2350 0x7f93480a4810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93540058a0 tx=0x7f935400b5b0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93601976f0 msgr2=0x7f9360197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93601976f0 0x7f9360197b70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f935800eaa0 tx=0x7f935800ef60 comp rx=0 tx=0).stop 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 shutdown_connections 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f93480a2350 0x7f93480a4810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93601976f0 0x7f9360197b70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93601064f0 0x7f936019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 --2- 172.21.15.17:0/3385335337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9360101420 0x7f936019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 >> 172.21.15.17:0/3385335337 conn(0x7f93600fcca0 msgr2=0x7f93600fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:21.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 shutdown_connections 2024-02-14T15:04:21.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:21.754+0000 7f9364977640 1 -- 172.21.15.17:0/3385335337 wait complete. 2024-02-14T15:04:21.769 INFO:tasks.cephfs.filesystem:scrub status for tag:ca55f103-f801-48ca-88eb-b6c29bc571d2 - {'path': '/', 'tag': 'ca55f103-f801-48ca-88eb-b6c29bc571d2', 'options': 'recursive,force'} 2024-02-14T15:04:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:22 smithi050 ceph-mon[36512]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-02-14T15:04:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:22 smithi050 ceph-mon[36512]: scrub queued for path: / 2024-02-14T15:04:23.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:22 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:23.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:22 smithi050 ceph-mon[36512]: scrub summary: active paths [/] 2024-02-14T15:04:23.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:22 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:22 smithi076 ceph-mon[37057]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-02-14T15:04:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:22 smithi076 ceph-mon[37057]: scrub queued for path: / 2024-02-14T15:04:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:22 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:22 smithi076 ceph-mon[37057]: scrub summary: active paths [/] 2024-02-14T15:04:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:22 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:22 smithi017 ceph-mon[32549]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-02-14T15:04:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:22 smithi017 ceph-mon[32549]: scrub queued for path: / 2024-02-14T15:04:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:22 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:22 smithi017 ceph-mon[32549]: scrub summary: active paths [/] 2024-02-14T15:04:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:22 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:24.064 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:23 smithi050 ceph-mon[36512]: scrub summary: idle 2024-02-14T15:04:24.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:23 smithi076 ceph-mon[37057]: scrub summary: idle 2024-02-14T15:04:24.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:23 smithi017 ceph-mon[32549]: scrub summary: idle 2024-02-14T15:04:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:24 smithi050 ceph-mon[36512]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-14T15:04:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:24 smithi076 ceph-mon[37057]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-14T15:04:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:24 smithi017 ceph-mon[32549]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-14T15:04:27.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:26 smithi050 ceph-mon[36512]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:27.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:26 smithi076 ceph-mon[37057]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:26 smithi017 ceph-mon[32549]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:29.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:28 smithi050 ceph-mon[36512]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:28 smithi076 ceph-mon[37057]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:28 smithi017 ceph-mon[32549]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:30.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:29 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:04:30.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:29 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:04:30.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:29 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:04:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:30 smithi050 ceph-mon[36512]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-14T15:04:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:30 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:30 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:30 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:30 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:30 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:30 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:30 smithi076 ceph-mon[37057]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-14T15:04:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:30 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:30 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:30 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:30 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:30 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:30 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:30 smithi017 ceph-mon[32549]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-14T15:04:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:30 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:30 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:30 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:30 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:30 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:31.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:30 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:32.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:31 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:04:32.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:31 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:04:32.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:31 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:31 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:04:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:31 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:04:32.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:31 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:32.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:31 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:04:32.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:31 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:04:32.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:31 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:04:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.920+0000 7fd3e7484640 1 -- 172.21.15.17:0/3122856792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100cf0 msgr2=0x7fd3e0104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.920+0000 7fd3e7484640 1 --2- 172.21.15.17:0/3122856792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100cf0 0x7fd3e0104d30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009930 tx=0x7fd3d002bc30 comp rx=0 tx=0).stop 2024-02-14T15:04:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 -- 172.21.15.17:0/3122856792 shutdown_connections 2024-02-14T15:04:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 --2- 172.21.15.17:0/3122856792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100cf0 0x7fd3e0104d30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 --2- 172.21.15.17:0/3122856792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e0105410 0x7fd3e010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 --2- 172.21.15.17:0/3122856792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e0100340 0x7fd3e0100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 -- 172.21.15.17:0/3122856792 >> 172.21.15.17:0/3122856792 conn(0x7fd3e0077220 msgr2=0x7fd3e0077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 -- 172.21.15.17:0/3122856792 shutdown_connections 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 -- 172.21.15.17:0/3122856792 wait complete. 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 Processor -- start 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.921+0000 7fd3e7484640 1 -- start start 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e7484640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100340 0x7fd3e019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e7484640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e0105410 0x7fd3e019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e7484640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e0199870 0x7fd3e0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e7484640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e019a320 con 0x7fd3e0100340 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e7484640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e019a490 con 0x7fd3e0199870 2024-02-14T15:04:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e7484640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e019a600 con 0x7fd3e0105410 2024-02-14T15:04:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100340 0x7fd3e019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100340 0x7fd3e019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57904/0 (socket says 172.21.15.17:57904) 2024-02-14T15:04:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 -- 172.21.15.17:0/2739743129 learned_addr learned my addr 172.21.15.17:0/2739743129 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e0105410 0x7fd3e019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e0199870 0x7fd3e0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 -- 172.21.15.17:0/2739743129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e0105410 msgr2=0x7fd3e019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e0105410 0x7fd3e019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 -- 172.21.15.17:0/2739743129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e0199870 msgr2=0x7fd3e0199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e0199870 0x7fd3e0199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 -- 172.21.15.17:0/2739743129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d0009560 con 0x7fd3e0100340 2024-02-14T15:04:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e0199870 0x7fd3e0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3e5220640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100340 0x7fd3e019f6b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c920 tx=0x7fd3dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.922+0000 7fd3d67fc640 1 -- 172.21.15.17:0/2739743129 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc007c70 con 0x7fd3e0100340 2024-02-14T15:04:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.923+0000 7fd3d67fc640 1 -- 172.21.15.17:0/2739743129 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3dc007dd0 con 0x7fd3e0100340 2024-02-14T15:04:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.923+0000 7fd3d67fc640 1 -- 172.21.15.17:0/2739743129 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc018860 con 0x7fd3e0100340 2024-02-14T15:04:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.923+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e01a6560 con 0x7fd3e0100340 2024-02-14T15:04:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.923+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e01a69f0 con 0x7fd3e0100340 2024-02-14T15:04:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3d67fc640 1 -- 172.21.15.17:0/2739743129 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fd3dc00fac0 con 0x7fd3e0100340 2024-02-14T15:04:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 Processor -- start 2024-02-14T15:04:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 -- start start 2024-02-14T15:04:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde821c720 0x55fde821cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821d040 0x55fde82256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde8228010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde818a9e0 con 0x55fde8225c20 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde8187f10 con 0x55fde821d040 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3ea053740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde81836f0 con 0x55fde821c720 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde8228010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde8228010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57916/0 (socket says 172.21.15.17:57916) 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/3060913684 learned_addr learned my addr 172.21.15.17:0/3060913684 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.926+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde821c720 0x55fde821cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3d67fc640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd3cc0a23c0 0x7fd3cc0a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e5220640 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821d040 0x55fde82256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/3060913684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde821c720 msgr2=0x55fde821cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde821c720 0x55fde821cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/3060913684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821d040 msgr2=0x55fde82256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821d040 0x55fde82256e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/3060913684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde7e94920 con 0x55fde8225c20 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e5220640 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821d040 0x55fde82256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3d67fc640 1 -- 172.21.15.17:0/2739743129 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd3dc0ce7a0 con 0x7fd3e0100340 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd3cc0a23c0 0x7fd3cc0a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde8228010 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005ab0 tx=0x7fd3d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b3ce799ea28491 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3cb7fe640 1 -- 172.21.15.17:0/3060913684 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00049a0 con 0x55fde8225c20 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.927+0000 7fd3cb7fe640 1 -- 172.21.15.17:0/3060913684 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0004b00 con 0x55fde8225c20 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3cb7fe640 1 -- 172.21.15.17:0/3060913684 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0005210 con 0x55fde8225c20 2024-02-14T15:04:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 -- 172.21.15.17:0/3060913684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 msgr2=0x55fde8228010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde8228010 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0005ab0 tx=0x7fd3d0004770 comp rx=0 tx=0).stop 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd3cc0a23c0 0x7fd3cc0a4880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005fd0 tx=0x7fd3d8004ab0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 -- 172.21.15.17:0/3060913684 shutdown_connections 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde8228010 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821d040 0x55fde82256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 --2- 172.21.15.17:0/3060913684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde821c720 0x55fde821cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 -- 172.21.15.17:0/3060913684 >> 172.21.15.17:0/3060913684 conn(0x55fde8216e50 msgr2=0x55fde8217230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 -- 172.21.15.17:0/3060913684 shutdown_connections 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.928+0000 7fd3ea053740 1 -- 172.21.15.17:0/3060913684 wait complete. 2024-02-14T15:04:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 -- start start 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 Processor -- start 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8225610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde821d040 0x55fde82229a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde8222fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde811ffa0 con 0x55fde821d040 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde7e57250 con 0x55fde821c720 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde8109200 con 0x55fde8225c20 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3ea053740 10 client.0 _ll_register_callbacks cb 0x55fde8178b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde821d040 0x55fde82229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde821d040 0x55fde82229a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57922/0 (socket says 172.21.15.17:57922) 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.929+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2310534125 learned_addr learned my addr 172.21.15.17:0/2310534125 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e5220640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8225610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde8222fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2310534125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 msgr2=0x55fde8222fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde8222fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 msgr2=0x55fde8225610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8225610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2310534125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e0068a20 con 0x55fde821d040 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde8222fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde821d040 0x55fde82229a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3d002c170 tx=0x7fd3d00051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f35ed057de1abf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/2310534125 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00049a0 con 0x55fde821d040 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/2310534125 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd3d004d070 con 0x55fde821d040 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.930+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fde7ed41b0 con 0x55fde821d040 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/2310534125 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3d003f440 con 0x55fde821d040 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 10 client.16077 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fde8181d70 con 0x55fde821d040 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/2310534125 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fde7ed41b0 con 0x55fde821d040 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/2310534125 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd3d003c050 con 0x55fde821d040 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 10 client.16077 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 20 client.16077 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde8235540 0x55fde8237960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 4 client.16077 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.931+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fde7ec0550 con 0x55fde8235540 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.932+0000 7fd3e5220640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde8235540 0x55fde8237960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.932+0000 7fd3c8ff9640 10 client.16077 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.932+0000 7fd3e5220640 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde8235540 0x55fde8237960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ec67ed4f87ff13 server_cookie=ddec86862f392a82 in_seq=0 out_seq=0 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/2310534125 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fde7ec0550 con 0x55fde8235540 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3c8ff9640 10 client.16077 handle_command_reply: tid=0 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 client.16077 shutdown 2024-02-14T15:04:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde821d040 msgr2=0x55fde82229a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde821d040 0x55fde82229a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd3d002c170 tx=0x7fd3d00051c0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 shutdown_connections 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde8235540 0x55fde8237960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde821d040 0x55fde82229a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde8222fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2310534125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8225610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3c8ff9640 0 client.16077 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 >> 172.21.15.17:0/2310534125 conn(0x55fde8216e50 msgr2=0x55fde8218ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 shutdown_connections 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 1 -- 172.21.15.17:0/2310534125 wait complete. 2024-02-14T15:04:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.933+0000 7fd3ea053740 20 client.16077 trim_cache size 0 max 16384 2024-02-14T15:04:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 Processor -- start 2024-02-14T15:04:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 -- start start 2024-02-14T15:04:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8222940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde82250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225610 0x55fde821fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde82125a0 con 0x55fde8225c20 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde821c160 con 0x55fde821c720 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3ea053740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde821c380 con 0x55fde8225610 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde82250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde82250d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57924/0 (socket says 172.21.15.17:57924) 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e5220640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8222940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2973770223 learned_addr learned my addr 172.21.15.17:0/2973770223 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225610 0x55fde821fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2973770223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225610 msgr2=0x55fde821fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225610 0x55fde821fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2973770223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 msgr2=0x55fde8222940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8222940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/2973770223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde811e820 con 0x55fde8225c20 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.947+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde82250d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0034660 tx=0x7fd3d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7e9a9ea3b8d364 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3caffd640 1 -- 172.21.15.17:0/2973770223 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00049a0 con 0x55fde8225c20 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3caffd640 1 -- 172.21.15.17:0/2973770223 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0034440 con 0x55fde8225c20 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3caffd640 1 -- 172.21.15.17:0/2973770223 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d003fb80 con 0x55fde8225c20 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3ea053740 1 -- 172.21.15.17:0/2973770223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 msgr2=0x55fde82250d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde82250d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0034660 tx=0x7fd3d0005fd0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3ea053740 1 -- 172.21.15.17:0/2973770223 shutdown_connections 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225c20 0x55fde82250d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225610 0x55fde821fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3ea053740 1 --2- 172.21.15.17:0/2973770223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde8222940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.948+0000 7fd3ea053740 1 -- 172.21.15.17:0/2973770223 >> 172.21.15.17:0/2973770223 conn(0x55fde8216e50 msgr2=0x55fde8222230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 -- 172.21.15.17:0/2973770223 shutdown_connections 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 -- 172.21.15.17:0/2973770223 wait complete. 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 -- start start 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 Processor -- start 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde822cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225610 0x55fde822b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde822ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225610 0x55fde822b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde822c170 con 0x55fde8225610 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde822c2e0 con 0x55fde821c720 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3ea053740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde828ba80 con 0x55fde8225c20 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3e4a1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225610 0x55fde822b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57934/0 (socket says 172.21.15.17:57934) 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.949+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/1992559006 learned_addr learned my addr 172.21.15.17:0/1992559006 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3ea053740 10 client.0 _ll_register_callbacks cb 0x7fd3d002c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e5220640 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde822cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e5a21640 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde822ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/1992559006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 msgr2=0x55fde822ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde822ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 msgr2=0x55fde822cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde822cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e4a1f640 1 -- 172.21.15.17:0/1992559006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fde8181d70 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3e4a1f640 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225610 0x55fde822b460 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0057000 tx=0x7fd3d00347d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f47b815ddf032c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/1992559006 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00051c0 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/1992559006 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd3d004d040 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fde7e83600 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/1992559006 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3d0005540 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3ea053740 10 client.16083 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.950+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fde8144720 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/1992559006 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fde7e83600 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/1992559006 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd3d003c070 con 0x55fde8225610 2024-02-14T15:04:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3ea053740 10 client.16083 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3ea053740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3ea053740 20 client.16083 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3ea053740 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde828ce70 0x55fde828f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3ea053740 4 client.16083 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fde7e2f520 con 0x55fde828ce70 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.951+0000 7fd3e5220640 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde828ce70 0x55fde828f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.952+0000 7fd3e5220640 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde828ce70 0x55fde828f260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57757d96e73bf2f5 server_cookie=6d4c1efc8bdde2d8 in_seq=0 out_seq=0 2024-02-14T15:04:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.952+0000 7fd3c8ff9640 10 client.16083 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.955+0000 7fd3c8ff9640 1 -- 172.21.15.17:0/1992559006 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fde7e2f520 con 0x55fde828ce70 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.955+0000 7fd3c8ff9640 10 client.16083 handle_command_reply: tid=0 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.955+0000 7fd3ea053740 1 client.16083 shutdown 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225610 msgr2=0x55fde822b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225610 0x55fde822b460 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0057000 tx=0x7fd3d00347d0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 shutdown_connections 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fde828ce70 0x55fde828f260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fde8225610 0x55fde822b460 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fde8225c20 0x55fde822ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 --2- 172.21.15.17:0/1992559006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fde821c720 0x55fde822cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 >> 172.21.15.17:0/1992559006 conn(0x55fde8216e50 msgr2=0x55fde8215630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3c8ff9640 0 client.16083 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 shutdown_connections 2024-02-14T15:04:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 1 -- 172.21.15.17:0/1992559006 wait complete. 2024-02-14T15:04:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.956+0000 7fd3ea053740 20 client.16083 trim_cache size 0 max 16384 2024-02-14T15:04:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:04:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:04:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:04:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:04:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd3cc0a23c0 msgr2=0x7fd3cc0a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd3cc0a23c0 0x7fd3cc0a4880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8005fd0 tx=0x7fd3d8004ab0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100340 msgr2=0x7fd3e019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100340 0x7fd3e019f6b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c920 tx=0x7fd3dc00cde0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 shutdown_connections 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd3cc0a23c0 0x7fd3cc0a4880 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e0199870 0x7fd3e0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e0105410 0x7fd3e019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 --2- 172.21.15.17:0/2739743129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3e0100340 0x7fd3e019f6b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 >> 172.21.15.17:0/2739743129 conn(0x7fd3e0077220 msgr2=0x7fd3e010b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 shutdown_connections 2024-02-14T15:04:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:32.957+0000 7fd3e7484640 1 -- 172.21.15.17:0/2739743129 wait complete. 2024-02-14T15:04:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:32 smithi050 ceph-mon[36512]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:33.166 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:32 smithi017 ceph-mon[32549]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:32 smithi076 ceph-mon[37057]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:34 smithi050 ceph-mon[36512]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:04:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:34 smithi076 ceph-mon[37057]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:04:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:34 smithi017 ceph-mon[32549]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:04:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:36 smithi050 ceph-mon[36512]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:04:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:36 smithi076 ceph-mon[37057]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:04:37.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:36 smithi017 ceph-mon[32549]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-02-14T15:04:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:38 smithi050 ceph-mon[36512]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:39.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:38 smithi076 ceph-mon[37057]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:38 smithi017 ceph-mon[32549]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-14T15:04:41.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:40 smithi050 ceph-mon[36512]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:40 smithi076 ceph-mon[37057]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:40 smithi017 ceph-mon[32549]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:42 smithi050 ceph-mon[36512]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:42 smithi076 ceph-mon[37057]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:42 smithi017 ceph-mon[32549]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:44 smithi050 ceph-mon[36512]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:44 smithi076 ceph-mon[37057]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:44 smithi017 ceph-mon[32549]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.091+0000 7f8aa0702640 1 -- 172.21.15.17:0/194475108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100cf0 msgr2=0x7f8a98104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.091+0000 7f8aa0702640 1 --2- 172.21.15.17:0/194475108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100cf0 0x7f8a98104d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009860 tx=0x7f8a8802bb70 comp rx=0 tx=0).stop 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 -- 172.21.15.17:0/194475108 shutdown_connections 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 --2- 172.21.15.17:0/194475108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a98105410 0x7f8a9810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 --2- 172.21.15.17:0/194475108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100cf0 0x7f8a98104d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 --2- 172.21.15.17:0/194475108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98100340 0x7f8a98100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 -- 172.21.15.17:0/194475108 >> 172.21.15.17:0/194475108 conn(0x7f8a98077220 msgr2=0x7f8a98077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 -- 172.21.15.17:0/194475108 shutdown_connections 2024-02-14T15:04:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 -- 172.21.15.17:0/194475108 wait complete. 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 Processor -- start 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.092+0000 7f8aa0702640 1 -- start start 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8aa0702640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100340 0x7f8a9819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8aa0702640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98105410 0x7f8a9819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8aa0702640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a981965b0 0x7f8a98196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8aa0702640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98196fa0 con 0x7f8a98105410 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8aa0702640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98197110 con 0x7f8a981965b0 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8aa0702640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a981972d0 con 0x7f8a98100340 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9dc9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98105410 0x7f8a9819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100340 0x7f8a9819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9dc9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98105410 0x7f8a9819c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33432/0 (socket says 172.21.15.17:33432) 2024-02-14T15:04:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9dc9d640 1 -- 172.21.15.17:0/2374305073 learned_addr learned my addr 172.21.15.17:0/2374305073 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a981965b0 0x7f8a98196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2374305073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a981965b0 msgr2=0x7f8a98196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a981965b0 0x7f8a98196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98105410 msgr2=0x7f8a9819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98105410 0x7f8a9819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2374305073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a94009560 con 0x7f8a98100340 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.093+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98105410 0x7f8a9819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.094+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100340 0x7f8a9819c400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a94005fd0 tx=0x7f8a9400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.094+0000 7f8a8f7fe640 1 -- 172.21.15.17:0/2374305073 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400cd00 con 0x7f8a98100340 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.094+0000 7f8a8f7fe640 1 -- 172.21.15.17:0/2374305073 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a9400ce60 con 0x7f8a98100340 2024-02-14T15:04:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.094+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a88009490 con 0x7f8a98100340 2024-02-14T15:04:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.095+0000 7f8a8f7fe640 1 -- 172.21.15.17:0/2374305073 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a940115d0 con 0x7f8a98100340 2024-02-14T15:04:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.095+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a98068d00 con 0x7f8a98100340 2024-02-14T15:04:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.097+0000 7f8a8f7fe640 1 -- 172.21.15.17:0/2374305073 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f8a940035f0 con 0x7f8a98100340 2024-02-14T15:04:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.098+0000 7f8a8f7fe640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8a840a22a0 0x7f8a840a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.098+0000 7f8aa33ae740 1 Processor -- start 2024-02-14T15:04:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.098+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8a840a22a0 0x7f8a840a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.098+0000 7f8a8f7fe640 1 -- 172.21.15.17:0/2374305073 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8a940d0470 con 0x7f8a98100340 2024-02-14T15:04:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.098+0000 7f8aa33ae740 1 -- start start 2024-02-14T15:04:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5000af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e5001030 0x55b0e5003420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4f619e0 con 0x55b0e4ff3530 2024-02-14T15:04:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4f5ef10 con 0x55b0e5001030 2024-02-14T15:04:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4f5a6f0 con 0x55b0e4ff3ee0 2024-02-14T15:04:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8a9ec9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e5001030 0x55b0e5003420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8a9ec9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e5001030 0x55b0e5003420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57362/0 (socket says 172.21.15.17:57362) 2024-02-14T15:04:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8a9ec9f640 1 -- 172.21.15.17:0/514798690 learned_addr learned my addr 172.21.15.17:0/514798690 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8a840a22a0 0x7f8a840a4760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a981976f0 tx=0x7f8a8802e040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.099+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5000af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a9ec9f640 1 -- 172.21.15.17:0/514798690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 msgr2=0x55b0e5000af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5000af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a9ec9f640 1 -- 172.21.15.17:0/514798690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 msgr2=0x55b0e4ff3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a9ec9f640 1 -- 172.21.15.17:0/514798690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0e4c6b920 con 0x55b0e5001030 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e5001030 0x55b0e5003420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009f80 tx=0x7f8a900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9f1005b530836e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a8cff9640 1 -- 172.21.15.17:0/514798690 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90004d20 con 0x55b0e5001030 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a8cff9640 1 -- 172.21.15.17:0/514798690 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a90004e80 con 0x55b0e5001030 2024-02-14T15:04:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.100+0000 7f8a8cff9640 1 -- 172.21.15.17:0/514798690 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90005280 con 0x55b0e5001030 2024-02-14T15:04:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 -- 172.21.15.17:0/514798690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e5001030 msgr2=0x55b0e5003420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e5001030 0x55b0e5003420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009f80 tx=0x7f8a900118d0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 -- 172.21.15.17:0/514798690 shutdown_connections 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e5001030 0x55b0e5003420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5000af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/514798690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 -- 172.21.15.17:0/514798690 >> 172.21.15.17:0/514798690 conn(0x55b0e4fede50 msgr2=0x55b0e4fee230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 -- 172.21.15.17:0/514798690 shutdown_connections 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.101+0000 7f8aa33ae740 1 -- 172.21.15.17:0/514798690 wait complete. 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.102+0000 7f8aa33ae740 1 -- start start 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.102+0000 7f8aa33ae740 1 Processor -- start 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.102+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e5000980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.102+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4ffdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.102+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e4ffe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4ef6fa0 con 0x55b0e4ff3530 2024-02-14T15:04:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4c2e250 con 0x55b0e4ff3ee0 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4ee0200 con 0x55b0e5001030 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e5000980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e5000980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33450/0 (socket says 172.21.15.17:33450) 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2721775983 learned_addr learned my addr 172.21.15.17:0/2721775983 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8aa33ae740 10 client.0 _ll_register_callbacks cb 0x55b0e4f4fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4ffdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e4ffe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2721775983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 msgr2=0x55b0e4ffe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e4ffe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 msgr2=0x55b0e4ffdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4ffdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2721775983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a90009c00 con 0x55b0e4ff3530 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e5000980 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8a9409c760 tx=0x7f8a94003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c3cb25e38ff6b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a81ffb640 1 -- 172.21.15.17:0/2721775983 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a940a0bb0 con 0x55b0e4ff3530 2024-02-14T15:04:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0e4cab1b0 con 0x55b0e4ff3530 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.103+0000 7f8a81ffb640 1 -- 172.21.15.17:0/2721775983 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8a940a5200 con 0x55b0e4ff3530 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8a81ffb640 1 -- 172.21.15.17:0/2721775983 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a9409ccc0 con 0x55b0e4ff3530 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 10 client.16095 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b0e4f58d70 con 0x55b0e4ff3530 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8a81ffb640 1 -- 172.21.15.17:0/2721775983 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a940b9080 con 0x55b0e4ff3530 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8a81ffb640 1 -- 172.21.15.17:0/2721775983 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a940cda60 con 0x55b0e4ff3530 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 10 client.16095 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 20 client.16095 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e4ff76c0 0x55b0e4ff7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 4 client.16095 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.104+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0e4c97550 con 0x55b0e4ff76c0 2024-02-14T15:04:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.105+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e4ff76c0 0x55b0e4ff7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.105+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e4ff76c0 0x55b0e4ff7ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf804affa179d1b0 server_cookie=253887d1afd82089 in_seq=0 out_seq=0 2024-02-14T15:04:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.105+0000 7f8a81ffb640 10 client.16095 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8a81ffb640 1 -- 172.21.15.17:0/2721775983 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b0e4c97550 con 0x55b0e4ff76c0 2024-02-14T15:04:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8a81ffb640 10 client.16095 handle_command_reply: tid=0 2024-02-14T15:04:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 client.16095 shutdown 2024-02-14T15:04:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 msgr2=0x55b0e5000980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e5000980 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8a9409c760 tx=0x7f8a94003960 comp rx=0 tx=0).stop 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 shutdown_connections 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e4ff76c0 0x55b0e4ff7ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e4ffe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4ffdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2721775983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e5000980 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 >> 172.21.15.17:0/2721775983 conn(0x55b0e4fede50 msgr2=0x55b0e4fefba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8a81ffb640 0 client.16095 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 shutdown_connections 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2721775983 wait complete. 2024-02-14T15:04:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.106+0000 7f8aa33ae740 20 client.16095 trim_cache size 0 max 16384 2024-02-14T15:04:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 Processor -- start 2024-02-14T15:04:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 -- start start 2024-02-14T15:04:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4fef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e5000440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4ffce80 con 0x55b0e4ff3530 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4c97550 con 0x55b0e4ff3ee0 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e4fe95a0 con 0x55b0e5001030 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33462/0 (socket says 172.21.15.17:33462) 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2689007691 learned_addr learned my addr 172.21.15.17:0/2689007691 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e5000440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.133+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4fef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2689007691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 msgr2=0x55b0e5000440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e5000440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2689007691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 msgr2=0x55b0e4fef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4fef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9e49e640 1 -- 172.21.15.17:0/2689007691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0e4ef5820 con 0x55b0e4ff3530 2024-02-14T15:04:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e5000440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4fef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff9450 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009230 tx=0x7f8a9409cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491600f462dab5be server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a83fff640 1 -- 172.21.15.17:0/2689007691 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011440 con 0x55b0e4ff3530 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a83fff640 1 -- 172.21.15.17:0/2689007691 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b0e4cab1b0 con 0x55b0e4ff3530 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.134+0000 7f8a83fff640 1 -- 172.21.15.17:0/2689007691 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94004000 con 0x55b0e4ff3530 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.135+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2689007691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 msgr2=0x55b0e4ff9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.135+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff9450 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009230 tx=0x7f8a9409cbe0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.135+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2689007691 shutdown_connections 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.135+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e5001030 0x55b0e5000440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.135+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3ee0 0x55b0e4fef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.135+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/2689007691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e4ff3530 0x55b0e4ff9450 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.135+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2689007691 >> 172.21.15.17:0/2689007691 conn(0x55b0e4fede50 msgr2=0x55b0e4fef2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.136+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2689007691 shutdown_connections 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.136+0000 7f8aa33ae740 1 -- 172.21.15.17:0/2689007691 wait complete. 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.136+0000 7f8aa33ae740 1 -- start start 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.136+0000 7f8aa33ae740 1 Processor -- start 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3530 0x55b0e501cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5017f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8aa33ae740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e5001030 0x55b0e5018660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e5018d30 con 0x55b0e5001030 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8aa33ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e506ae80 con 0x55b0e4ff3530 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3530 0x55b0e501cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8a9ec9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e5001030 0x55b0e5018660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8a9e49e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3530 0x55b0e501cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57402/0 (socket says 172.21.15.17:57402) 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8a9e49e640 1 -- 172.21.15.17:0/3694866012 learned_addr learned my addr 172.21.15.17:0/3694866012 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5017f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.137+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e506afc0 con 0x55b0e4ff3ee0 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8aa33ae740 10 client.0 _ll_register_callbacks cb 0x7f8a90009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9e49e640 1 -- 172.21.15.17:0/3694866012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 msgr2=0x55b0e5017f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5017f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9e49e640 1 -- 172.21.15.17:0/3694866012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e5001030 msgr2=0x55b0e5018660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e5001030 0x55b0e5018660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9e49e640 1 -- 172.21.15.17:0/3694866012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9810d7f0 con 0x55b0e4ff3530 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5017f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9ec9f640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e5001030 0x55b0e5018660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a9e49e640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3530 0x55b0e501cbb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a9409c9f0 tx=0x7f8a9409cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ca63b6d58aeffe server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:04:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a81ffb640 1 -- 172.21.15.17:0/3694866012 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011440 con 0x55b0e4ff3530 2024-02-14T15:04:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0e4c5a600 con 0x55b0e4ff3530 2024-02-14T15:04:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.138+0000 7f8a81ffb640 1 -- 172.21.15.17:0/3694866012 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8a9409beb0 con 0x55b0e4ff3530 2024-02-14T15:04:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.139+0000 7f8a81ffb640 1 -- 172.21.15.17:0/3694866012 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a940cd7d0 con 0x55b0e4ff3530 2024-02-14T15:04:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.139+0000 7f8aa33ae740 10 client.25508 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.139+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b0e4c6b920 con 0x55b0e4ff3530 2024-02-14T15:04:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.139+0000 7f8a81ffb640 1 -- 172.21.15.17:0/3694866012 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b0e4c5a600 con 0x55b0e4ff3530 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.139+0000 7f8a81ffb640 1 -- 172.21.15.17:0/3694866012 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a940b0ab0 con 0x55b0e4ff3530 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8aa33ae740 10 client.25508 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8aa33ae740 10 client.25508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8aa33ae740 20 client.25508 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e506c1f0 0x55b0e506e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8aa33ae740 4 client.25508 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b0e4c06520 con 0x55b0e506c1f0 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e506c1f0 0x55b0e506e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8a9dc9d640 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e506c1f0 0x55b0e506e5e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6193623e9fe20de6 server_cookie=6bc5de5e67fe599c in_seq=0 out_seq=0 2024-02-14T15:04:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.140+0000 7f8a81ffb640 10 client.25508 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8a81ffb640 1 -- 172.21.15.17:0/3694866012 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b0e4c06520 con 0x55b0e506c1f0 2024-02-14T15:04:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8a81ffb640 10 client.25508 handle_command_reply: tid=0 2024-02-14T15:04:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 client.25508 shutdown 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3530 msgr2=0x55b0e501cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3530 0x55b0e501cbb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a9409c9f0 tx=0x7f8a9409cf70 comp rx=0 tx=0).stop 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 shutdown_connections 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55b0e506c1f0 0x55b0e506e5e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0e5001030 0x55b0e5018660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0e4ff3ee0 0x55b0e5017f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 --2- 172.21.15.17:0/3694866012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e4ff3530 0x55b0e501cbb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 >> 172.21.15.17:0/3694866012 conn(0x55b0e4fede50 msgr2=0x55b0e4fecec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.144+0000 7f8a81ffb640 0 client.25508 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 shutdown_connections 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa33ae740 1 -- 172.21.15.17:0/3694866012 wait complete. 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa33ae740 20 client.25508 trim_cache size 0 max 16384 2024-02-14T15:04:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:04:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:04:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:04:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:04:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8a840a22a0 msgr2=0x7f8a840a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8a840a22a0 0x7f8a840a4760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a981976f0 tx=0x7f8a8802e040 comp rx=0 tx=0).stop 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100340 msgr2=0x7f8a9819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100340 0x7f8a9819c400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a94005fd0 tx=0x7f8a9400ee50 comp rx=0 tx=0).stop 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 shutdown_connections 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8a840a22a0 0x7f8a840a4760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a98105410 0x7f8a9819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a981965b0 0x7f8a98196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.145+0000 7f8aa0702640 1 --2- 172.21.15.17:0/2374305073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a98100340 0x7f8a9819c400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.146+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 >> 172.21.15.17:0/2374305073 conn(0x7f8a98077220 msgr2=0x7f8a980fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.146+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 shutdown_connections 2024-02-14T15:04:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:46.146+0000 7f8aa0702640 1 -- 172.21.15.17:0/2374305073 wait complete. 2024-02-14T15:04:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:46 smithi050 ceph-mon[36512]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:46 smithi076 ceph-mon[37057]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:46 smithi017 ceph-mon[32549]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:48 smithi050 ceph-mon[36512]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:48 smithi076 ceph-mon[37057]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:48 smithi017 ceph-mon[32549]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:50 smithi050 ceph-mon[36512]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:04:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:50 smithi076 ceph-mon[37057]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:04:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:50 smithi017 ceph-mon[32549]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-14T15:04:51.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:04:51.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:04:51.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 -- 172.21.15.17:0/3911735349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920105410 msgr2=0x7fb92010cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 --2- 172.21.15.17:0/3911735349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920105410 0x7fb92010cfe0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb918009cd0 tx=0x7fb91802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 -- 172.21.15.17:0/3911735349 shutdown_connections 2024-02-14T15:04:51.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 --2- 172.21.15.17:0/3911735349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920105410 0x7fb92010cfe0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 --2- 172.21.15.17:0/3911735349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb920100cf0 0x7fb920104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 --2- 172.21.15.17:0/3911735349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb920100340 0x7fb920100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 -- 172.21.15.17:0/3911735349 >> 172.21.15.17:0/3911735349 conn(0x7fb9200772e0 msgr2=0x7fb9200776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 -- 172.21.15.17:0/3911735349 shutdown_connections 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.879+0000 7fb924b02640 1 -- 172.21.15.17:0/3911735349 wait complete. 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 Processor -- start 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 -- start start 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb920100340 0x7fb92019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb920105410 0x7fb92019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920197690 0x7fb920197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb920198160 con 0x7fb920197690 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9201982d0 con 0x7fb920100340 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb924b02640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb920198490 con 0x7fb920105410 2024-02-14T15:04:51.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920197690 0x7fb920197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920197690 0x7fb920197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55450/0 (socket says 172.21.15.17:55450) 2024-02-14T15:04:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb91ed9d640 1 -- 172.21.15.17:0/1407851369 learned_addr learned my addr 172.21.15.17:0/1407851369 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.880+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb920105410 0x7fb92019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb91e59c640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb920100340 0x7fb92019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb91dd9b640 1 -- 172.21.15.17:0/1407851369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb920100340 msgr2=0x7fb92019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb920100340 0x7fb92019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb91dd9b640 1 -- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920197690 msgr2=0x7fb920197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920197690 0x7fb920197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb91dd9b640 1 -- 172.21.15.17:0/1407851369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918009900 con 0x7fb920105410 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb920105410 0x7fb92019db10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb914005fd0 tx=0x7fb91400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb90f7fe640 1 -- 172.21.15.17:0/1407851369 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914009c00 con 0x7fb920105410 2024-02-14T15:04:51.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.881+0000 7fb90f7fe640 1 -- 172.21.15.17:0/1407851369 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb914004510 con 0x7fb920105410 2024-02-14T15:04:51.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.882+0000 7fb90f7fe640 1 -- 172.21.15.17:0/1407851369 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914003e40 con 0x7fb920105410 2024-02-14T15:04:51.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.882+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9201a4430 con 0x7fb920105410 2024-02-14T15:04:51.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.882+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9201a48c0 con 0x7fb920105410 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.885+0000 7fb90f7fe640 1 -- 172.21.15.17:0/1407851369 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fb914003fa0 con 0x7fb920105410 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.885+0000 7fb9277af740 1 Processor -- start 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.885+0000 7fb9277af740 1 -- start start 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb9277af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a62863e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb9277af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a62864360 0x561a6286ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb9277af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286cf40 0x561a6286f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb9277af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6281a5b0 con 0x561a62864360 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb9277af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a627ce2b0 con 0x561a6286cf40 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb9277af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a627cb7e0 con 0x561a62863a40 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a62864360 0x561a6286ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a62864360 0x561a6286ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55466/0 (socket says 172.21.15.17:55466) 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 -- 172.21.15.17:0/3087769460 learned_addr learned my addr 172.21.15.17:0/3087769460 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286cf40 0x561a6286f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a62863e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 -- 172.21.15.17:0/3087769460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 msgr2=0x561a62863e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a62863e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 -- 172.21.15.17:0/3087769460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286cf40 msgr2=0x561a6286f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286cf40 0x561a6286f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91e59c640 1 -- 172.21.15.17:0/3087769460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a62789020 con 0x561a62864360 2024-02-14T15:04:51.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a62863e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.886+0000 7fb90f7fe640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb9080a2230 0x7fb9080a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.887+0000 7fb91e59c640 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a62864360 0x561a6286ca00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb910009d20 tx=0x7fb910011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61b0c51db230239 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.887+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb9080a2230 0x7fb9080a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.887+0000 7fb90f7fe640 1 -- 172.21.15.17:0/1407851369 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb9140ce060 con 0x7fb920105410 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.887+0000 7fb90cff9640 1 -- 172.21.15.17:0/3087769460 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910019630 con 0x561a62864360 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.887+0000 7fb90cff9640 1 -- 172.21.15.17:0/3087769460 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb910019c70 con 0x561a62864360 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.887+0000 7fb90cff9640 1 -- 172.21.15.17:0/3087769460 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91000ec70 con 0x561a62864360 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 -- 172.21.15.17:0/3087769460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a62864360 msgr2=0x561a6286ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a62864360 0x561a6286ca00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb910009d20 tx=0x7fb910011810 comp rx=0 tx=0).stop 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb9080a2230 0x7fb9080a46f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb918006010 tx=0x7fb918032d70 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 -- 172.21.15.17:0/3087769460 shutdown_connections 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a62864360 0x561a6286ca00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286cf40 0x561a6286f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 --2- 172.21.15.17:0/3087769460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a62863e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 -- 172.21.15.17:0/3087769460 >> 172.21.15.17:0/3087769460 conn(0x561a6285e170 msgr2=0x561a6285e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 -- 172.21.15.17:0/3087769460 shutdown_connections 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 -- 172.21.15.17:0/3087769460 wait complete. 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.888+0000 7fb9277af740 1 -- start start 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 1 Processor -- start 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a6286a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a6286a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286aae0 0x561a62878ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a627c6fc0 con 0x561a6286cf40 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6249a430 con 0x561a62863a40 2024-02-14T15:04:51.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a62456e00 con 0x561a6286aae0 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb9277af740 10 client.0 _ll_register_callbacks cb 0x561a62748870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a6286a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a6286a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a6286a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57322/0 (socket says 172.21.15.17:57322) 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a6286a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55476/0 (socket says 172.21.15.17:55476) 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 -- 172.21.15.17:0/3478966251 learned_addr learned my addr 172.21.15.17:0/3478966251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286aae0 0x561a62878ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 -- 172.21.15.17:0/3478966251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286aae0 msgr2=0x561a62878ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286aae0 0x561a62878ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 -- 172.21.15.17:0/3478966251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 msgr2=0x561a6286a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a6286a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91e59c640 1 -- 172.21.15.17:0/3478966251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb910009950 con 0x561a62863a40 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.889+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a6286a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb91e59c640 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a6286a060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x561a62863510 tx=0x7fb91000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0ff7393c21febd server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:51.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb905ffb640 1 -- 172.21.15.17:0/3478966251 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910005230 con 0x561a62863a40 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a624c67e0 con 0x561a62863a40 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb905ffb640 1 -- 172.21.15.17:0/3478966251 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb910005390 con 0x561a62863a40 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb905ffb640 1 -- 172.21.15.17:0/3478966251 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb910009ed0 con 0x561a62863a40 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb9277af740 10 client.25520 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561a627c5640 con 0x561a62863a40 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.890+0000 7fb905ffb640 1 -- 172.21.15.17:0/3478966251 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561a624c67e0 con 0x561a62863a40 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb905ffb640 1 -- 172.21.15.17:0/3478966251 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb910004850 con 0x561a62863a40 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb9277af740 10 client.25520 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb9277af740 10 client.25520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb9277af740 20 client.25520 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb9277af740 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a6287d7f0 0x561a6287fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb9277af740 4 client.25520 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a62503720 con 0x561a6287d7f0 2024-02-14T15:04:51.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a6287d7f0 0x561a6287fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb905ffb640 10 client.25520 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:51.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.891+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a6287d7f0 0x561a6287fc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12f36e160bf0b37 server_cookie=74e8b96b62295bbd in_seq=0 out_seq=0 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb905ffb640 1 -- 172.21.15.17:0/3478966251 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a62503720 con 0x561a6287d7f0 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb905ffb640 10 client.25520 handle_command_reply: tid=0 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 client.25520 shutdown 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 msgr2=0x561a6286a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a6286a060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x561a62863510 tx=0x7fb91000eeb0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 shutdown_connections 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a6287d7f0 0x561a6287fc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb905ffb640 0 client.25520 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a6286a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286aae0 0x561a62878ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 --2- 172.21.15.17:0/3478966251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a6286a060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 >> 172.21.15.17:0/3478966251 conn(0x561a6285e170 msgr2=0x561a6285fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 shutdown_connections 2024-02-14T15:04:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 1 -- 172.21.15.17:0/3478966251 wait complete. 2024-02-14T15:04:51.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.892+0000 7fb9277af740 20 client.25520 trim_cache size 0 max 16384 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.907+0000 7fb9277af740 1 Processor -- start 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.907+0000 7fb9277af740 1 -- start start 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb9277af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a628788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb9277af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286aae0 0x561a62878e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb9277af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a628672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb9277af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6274bfd0 con 0x561a6286cf40 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb9277af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a628556a0 con 0x561a6286aae0 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb9277af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a62864060 con 0x561a62863a40 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a628672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a628672e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55490/0 (socket says 172.21.15.17:55490) 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 -- 172.21.15.17:0/1938784072 learned_addr learned my addr 172.21.15.17:0/1938784072 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 -- 172.21.15.17:0/1938784072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 msgr2=0x561a628788f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91e59c640 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a628788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a628788f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 -- 172.21.15.17:0/1938784072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286aae0 msgr2=0x561a62878e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286aae0 0x561a62878e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286aae0 0x561a62878e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.908+0000 7fb91ed9d640 1 -- 172.21.15.17:0/1938784072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a62761b30 con 0x561a6286cf40 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.909+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a628672e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb918034440 tx=0x7fb918041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8eb6d066cdaa240 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.909+0000 7fb907fff640 1 -- 172.21.15.17:0/1938784072 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180326b0 con 0x561a6286cf40 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.909+0000 7fb907fff640 1 -- 172.21.15.17:0/1938784072 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb918050070 con 0x561a6286cf40 2024-02-14T15:04:51.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.909+0000 7fb907fff640 1 -- 172.21.15.17:0/1938784072 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918004590 con 0x561a6286cf40 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 -- 172.21.15.17:0/1938784072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 msgr2=0x561a628672e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a628672e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb918034440 tx=0x7fb918041000 comp rx=0 tx=0).stop 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 -- 172.21.15.17:0/1938784072 shutdown_connections 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286cf40 0x561a628672e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a6286aae0 0x561a62878e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 --2- 172.21.15.17:0/1938784072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a62863a40 0x561a628788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 -- 172.21.15.17:0/1938784072 >> 172.21.15.17:0/1938784072 conn(0x561a6285e170 msgr2=0x561a62868930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 -- 172.21.15.17:0/1938784072 shutdown_connections 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 -- 172.21.15.17:0/1938784072 wait complete. 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 -- start start 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 Processor -- start 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.910+0000 7fb9277af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a62874360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb9277af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286aae0 0x561a62872700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb9277af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286cf40 0x561a62872df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb9277af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a628734f0 con 0x561a6286aae0 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb9277af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a628d3d30 con 0x561a62863a40 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb9277af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a628d3e70 con 0x561a6286cf40 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb9277af740 10 client.0 _ll_register_callbacks cb 0x7fb910009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286aae0 0x561a62872700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286cf40 0x561a62872df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286cf40 0x561a62872df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:45770/0 (socket says 172.21.15.17:45770) 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286aae0 0x561a62872700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55500/0 (socket says 172.21.15.17:55500) 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 -- 172.21.15.17:0/168577370 learned_addr learned my addr 172.21.15.17:0/168577370 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91e59c640 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a62874360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 -- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 msgr2=0x561a62874360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a62874360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 -- 172.21.15.17:0/168577370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286aae0 msgr2=0x561a62872700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:51.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286aae0 0x561a62872700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 -- 172.21.15.17:0/168577370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92010d7f0 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91dd9b640 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286aae0 0x561a62872700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb91ed9d640 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286cf40 0x561a62872df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb918004af0 tx=0x7fb918032980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f591fcf0b0a703b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.911+0000 7fb905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb905ffb640 1 -- 172.21.15.17:0/168577370 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918032b70 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb910019c70 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb905ffb640 1 -- 172.21.15.17:0/168577370 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb918004490 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb905ffb640 1 -- 172.21.15.17:0/168577370 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb918004650 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 10 client.25654 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561a62787880 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb905ffb640 1 -- 172.21.15.17:0/168577370 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb910019c70 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb905ffb640 1 -- 172.21.15.17:0/168577370 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb918047040 con 0x561a6286cf40 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 10 client.25654 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 20 client.25654 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:51.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a628d53b0 0x561a628d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 4 client.25654 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.912+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a62552890 con 0x561a628d53b0 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.913+0000 7fb91e59c640 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a628d53b0 0x561a628d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.913+0000 7fb905ffb640 10 client.25654 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.913+0000 7fb91e59c640 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a628d53b0 0x561a628d77a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3c8c5aaf55f29b server_cookie=6f12d70e3f800e39 in_seq=0 out_seq=0 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.913+0000 7fb905ffb640 1 -- 172.21.15.17:0/168577370 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561a62552890 con 0x561a628d53b0 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.913+0000 7fb905ffb640 10 client.25654 handle_command_reply: tid=0 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.913+0000 7fb9277af740 1 client.25654 shutdown 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286cf40 msgr2=0x561a62872df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286cf40 0x561a62872df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb918004af0 tx=0x7fb918032980 comp rx=0 tx=0).stop 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 shutdown_connections 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561a628d53b0 0x561a628d77a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561a6286aae0 0x561a62872700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a6286cf40 0x561a62872df0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb905ffb640 0 client.25654 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:51.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 --2- 172.21.15.17:0/168577370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a62863a40 0x561a62874360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 >> 172.21.15.17:0/168577370 conn(0x561a6285e170 msgr2=0x561a628688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 shutdown_connections 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 1 -- 172.21.15.17:0/168577370 wait complete. 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb9277af740 20 client.25654 trim_cache size 0 max 16384 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "no active scrubs running", 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": {} 2024-02-14T15:04:51.920 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb9080a2230 msgr2=0x7fb9080a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb9080a2230 0x7fb9080a46f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb918006010 tx=0x7fb918032d70 comp rx=0 tx=0).stop 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb920105410 msgr2=0x7fb92019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb920105410 0x7fb92019db10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb914005fd0 tx=0x7fb91400dde0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 shutdown_connections 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb9080a2230 0x7fb9080a46f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb920197690 0x7fb920197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb920105410 0x7fb92019db10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 --2- 172.21.15.17:0/1407851369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb920100340 0x7fb92019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:51.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.914+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 >> 172.21.15.17:0/1407851369 conn(0x7fb9200772e0 msgr2=0x7fb9200687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:51.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.915+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 shutdown_connections 2024-02-14T15:04:51.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:51.915+0000 7fb924b02640 1 -- 172.21.15.17:0/1407851369 wait complete. 2024-02-14T15:04:51.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-14T15:04:51.926 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:04:52.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 -- 172.21.15.17:0/1021063446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 msgr2=0x7f4f54102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/1021063446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f54102830 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009900 tx=0x7f4f4402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 -- 172.21.15.17:0/1021063446 shutdown_connections 2024-02-14T15:04:52.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/1021063446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f5410b820 0x7f4f5410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/1021063446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f54102d70 0x7f4f5410b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/1021063446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f54102830 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 -- 172.21.15.17:0/1021063446 >> 172.21.15.17:0/1021063446 conn(0x7f4f540772b0 msgr2=0x7f4f540776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 -- 172.21.15.17:0/1021063446 shutdown_connections 2024-02-14T15:04:52.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.029+0000 7f4f5a06f640 1 -- 172.21.15.17:0/1021063446 wait complete. 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.030+0000 7f4f5a06f640 1 Processor -- start 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.030+0000 7f4f5a06f640 1 -- start start 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.030+0000 7f4f5a06f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f541000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.030+0000 7f4f5a06f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f54102d70 0x7f4f541005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f5a06f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f5410b820 0x7f4f54100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f541000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f5a06f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f541a5be0 con 0x7f4f54102450 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f541000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55530/0 (socket says 172.21.15.17:55530) 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f5a06f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f541a5d50 con 0x7f4f5410b820 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f537fe640 1 -- 172.21.15.17:0/3356020835 learned_addr learned my addr 172.21.15.17:0/3356020835 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f541a5f10 con 0x7f4f54102d70 2024-02-14T15:04:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f52ffd640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f54102d70 0x7f4f541005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f53fff640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f5410b820 0x7f4f54100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f52ffd640 1 -- 172.21.15.17:0/3356020835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f5410b820 msgr2=0x7f4f54100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f52ffd640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f5410b820 0x7f4f54100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f52ffd640 1 -- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 msgr2=0x7f4f541000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f52ffd640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f541000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f52ffd640 1 -- 172.21.15.17:0/3356020835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44009560 con 0x7f4f54102d70 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f537fe640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f541000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.031+0000 7f4f52ffd640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f54102d70 0x7f4f541005e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e9f0 tx=0x7f4f4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.032+0000 7f4f50ff9640 1 -- 172.21.15.17:0/3356020835 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800ccc0 con 0x7f4f54102d70 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.032+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f541a61f0 con 0x7f4f54102d70 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.032+0000 7f4f50ff9640 1 -- 172.21.15.17:0/3356020835 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f4800ce20 con 0x7f4f54102d70 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.032+0000 7f4f50ff9640 1 -- 172.21.15.17:0/3356020835 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48018920 con 0x7f4f54102d70 2024-02-14T15:04:52.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.032+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f541a6750 con 0x7f4f54102d70 2024-02-14T15:04:52.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.033+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f540680b0 con 0x7f4f54102d70 2024-02-14T15:04:52.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.035+0000 7f4f50ff9640 1 -- 172.21.15.17:0/3356020835 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f4f48018a80 con 0x7f4f54102d70 2024-02-14T15:04:52.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.035+0000 7f4f50ff9640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4f3c0a2350 0x7f4f3c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.035+0000 7f4f50ff9640 1 -- 172.21.15.17:0/3356020835 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f4f48014070 con 0x7f4f54102d70 2024-02-14T15:04:52.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.035+0000 7f4f537fe640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4f3c0a2350 0x7f4f3c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.036+0000 7f4f537fe640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4f3c0a2350 0x7f4f3c0a4810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f44000c00 tx=0x7f4f440058d0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.041+0000 7f4f50ff9640 1 -- 172.21.15.17:0/3356020835 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f48052d20 con 0x7f4f54102d70 2024-02-14T15:04:52.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.482+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f541015d0 con 0x7f4f54102d70 2024-02-14T15:04:52.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.483+0000 7f4f50ff9640 1 -- 172.21.15.17:0/3356020835 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f4f480970e0 con 0x7f4f54102d70 2024-02-14T15:04:52.485 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:04:52.485 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:03:09.607099+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:04:52.485 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4f3c0a2350 msgr2=0x7f4f3c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4f3c0a2350 0x7f4f3c0a4810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f44000c00 tx=0x7f4f440058d0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f54102d70 msgr2=0x7f4f541005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f54102d70 0x7f4f541005e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e9f0 tx=0x7f4f4800eeb0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 shutdown_connections 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4f3c0a2350 0x7f4f3c0a4810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f5410b820 0x7f4f54100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f54102d70 0x7f4f541005e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 --2- 172.21.15.17:0/3356020835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54102450 0x7f4f541000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.486+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 >> 172.21.15.17:0/3356020835 conn(0x7f4f540772b0 msgr2=0x7f4f540fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.487+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 shutdown_connections 2024-02-14T15:04:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.487+0000 7f4f5a06f640 1 -- 172.21.15.17:0/3356020835 wait complete. 2024-02-14T15:04:52.503 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:04:52.503 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.614+0000 7fe96f29d640 1 -- 172.21.15.17:0/1943138826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102450 msgr2=0x7fe968102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.614+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1943138826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102450 0x7fe968102830 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe964009900 tx=0x7fe96402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.614+0000 7fe96f29d640 1 -- 172.21.15.17:0/1943138826 shutdown_connections 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.614+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1943138826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe96810b820 0x7fe96810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.614+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1943138826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe968102d70 0x7fe96810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.614+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1943138826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102450 0x7fe968102830 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.614+0000 7fe96f29d640 1 -- 172.21.15.17:0/1943138826 >> 172.21.15.17:0/1943138826 conn(0x7fe9680772e0 msgr2=0x7fe9680776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.615+0000 7fe96f29d640 1 -- 172.21.15.17:0/1943138826 shutdown_connections 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.615+0000 7fe96f29d640 1 -- 172.21.15.17:0/1943138826 wait complete. 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.615+0000 7fe96f29d640 1 Processor -- start 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.615+0000 7fe96f29d640 1 -- start start 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96f29d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe968102450 0x7fe96819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96f29d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102d70 0x7fe96819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96f29d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe96810b820 0x7fe9681976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96f29d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe968197e30 con 0x7fe968102d70 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96f29d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe968197fa0 con 0x7fe968102450 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96f29d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe968198160 con 0x7fe96810b820 2024-02-14T15:04:52.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102d70 0x7fe96819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102d70 0x7fe96819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55544/0 (socket says 172.21.15.17:55544) 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 -- 172.21.15.17:0/1909028598 learned_addr learned my addr 172.21.15.17:0/1909028598 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96d039640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe968102450 0x7fe96819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96d83a640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe96810b820 0x7fe9681976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 -- 172.21.15.17:0/1909028598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe96810b820 msgr2=0x7fe9681976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe96810b820 0x7fe9681976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 -- 172.21.15.17:0/1909028598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe968102450 msgr2=0x7fe96819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe968102450 0x7fe96819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 -- 172.21.15.17:0/1909028598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964009560 con 0x7fe968102d70 2024-02-14T15:04:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96d83a640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe96810b820 0x7fe9681976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:52.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96d039640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe968102450 0x7fe96819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.616+0000 7fe96c838640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102d70 0x7fe96819dad0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe958009b60 tx=0x7fe958007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.617+0000 7fe95e7fc640 1 -- 172.21.15.17:0/1909028598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958014070 con 0x7fe968102d70 2024-02-14T15:04:52.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.617+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9681983e0 con 0x7fe968102d70 2024-02-14T15:04:52.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.617+0000 7fe95e7fc640 1 -- 172.21.15.17:0/1909028598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe958009d70 con 0x7fe968102d70 2024-02-14T15:04:52.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.617+0000 7fe95e7fc640 1 -- 172.21.15.17:0/1909028598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958005280 con 0x7fe968102d70 2024-02-14T15:04:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.617+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681001b0 con 0x7fe968102d70 2024-02-14T15:04:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.619+0000 7fe95e7fc640 1 -- 172.21.15.17:0/1909028598 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fe958018030 con 0x7fe968102d70 2024-02-14T15:04:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.620+0000 7fe95e7fc640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe9540a2350 0x7fe9540a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.620+0000 7fe95e7fc640 1 -- 172.21.15.17:0/1909028598 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe9580d1810 con 0x7fe968102d70 2024-02-14T15:04:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.620+0000 7fe96d039640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe9540a2350 0x7fe9540a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.620+0000 7fe971e73740 1 Processor -- start 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.620+0000 7fe971e73740 1 -- start start 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe971e73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30644c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe9540a2350 0x7fe9540a4810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe964006010 tx=0x7fe964005e40 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe971e73740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644c690 0x55d306454c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe971e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306457660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe971e73740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3063aac80 con 0x55d30644c690 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe971e73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3063ba320 con 0x55d30644bce0 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe971e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3063b7850 con 0x55d306455210 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96c838640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644c690 0x55d306454c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30644c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96c838640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644c690 0x55d306454c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55550/0 (socket says 172.21.15.17:55550) 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30644c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57386/0 (socket says 172.21.15.17:57386) 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96c838640 1 -- 172.21.15.17:0/2095184266 learned_addr learned my addr 172.21.15.17:0/2095184266 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d83a640 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306457660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 -- 172.21.15.17:0/2095184266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 msgr2=0x55d306457660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306457660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 -- 172.21.15.17:0/2095184266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644c690 msgr2=0x55d306454c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644c690 0x55d306454c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96d039640 1 -- 172.21.15.17:0/2095184266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d306379e30 con 0x55d30644bce0 2024-02-14T15:04:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.621+0000 7fe96c838640 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644c690 0x55d306454c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.622+0000 7fe96d039640 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30644c0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe964000c00 tx=0x7fe96403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a079e2a2c253d34 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.622+0000 7fe9537fe640 1 -- 172.21.15.17:0/2095184266 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964004040 con 0x55d30644bce0 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.622+0000 7fe9537fe640 1 -- 172.21.15.17:0/2095184266 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe964004df0 con 0x55d30644bce0 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.622+0000 7fe9537fe640 1 -- 172.21.15.17:0/2095184266 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964003810 con 0x55d30644bce0 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 -- 172.21.15.17:0/2095184266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 msgr2=0x55d30644c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30644c0c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe964000c00 tx=0x7fe96403f000 comp rx=0 tx=0).stop 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 -- 172.21.15.17:0/2095184266 shutdown_connections 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644c690 0x55d306454c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306457660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 --2- 172.21.15.17:0/2095184266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30644c0c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 -- 172.21.15.17:0/2095184266 >> 172.21.15.17:0/2095184266 conn(0x55d3064463b0 msgr2=0x55d306446790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 -- 172.21.15.17:0/2095184266 shutdown_connections 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 -- 172.21.15.17:0/2095184266 wait complete. 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 -- start start 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.623+0000 7fe971e73740 1 Processor -- start 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe971e73740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe971e73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d306452600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe971e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306452b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe971e73740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3063b3030 con 0x55d30644bce0 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe971e73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d306086430 con 0x55d30644c690 2024-02-14T15:04:52.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe971e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d306042e00 con 0x55d306455210 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe971e73740 10 client.0 _ll_register_callbacks cb 0x55d3060c3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96d039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96d039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55562/0 (socket says 172.21.15.17:55562) 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96c838640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d306452600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96c838640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d306452600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57394/0 (socket says 172.21.15.17:57394) 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96d039640 1 -- 172.21.15.17:0/2222246940 learned_addr learned my addr 172.21.15.17:0/2222246940 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96d83a640 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306452b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96c838640 1 -- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 msgr2=0x55d306452b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96c838640 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306452b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96c838640 1 -- 172.21.15.17:0/2222246940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 msgr2=0x55d3064520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96c838640 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.624+0000 7fe96c838640 1 -- 172.21.15.17:0/2222246940 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96810e8d0 con 0x55d30644c690 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe96c838640 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d306452600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe95809d950 tx=0x7fe95800f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb8c2cf2d1c55e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe950ff9640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe950ff9640 1 -- 172.21.15.17:0/2222246940 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9580d15d0 con 0x55d30644c690 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3060b27e0 con 0x55d30644c690 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe950ff9640 1 -- 172.21.15.17:0/2222246940 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe958088040 con 0x55d30644c690 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe950ff9640 1 -- 172.21.15.17:0/2222246940 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9580a1da0 con 0x55d30644c690 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe971e73740 10 client.25541 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d3063b16b0 con 0x55d30644c690 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.625+0000 7fe950ff9640 1 -- 172.21.15.17:0/2222246940 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe958082080 con 0x55d30644c690 2024-02-14T15:04:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.626+0000 7fe950ff9640 1 -- 172.21.15.17:0/2222246940 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9580cedf0 con 0x55d30644c690 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.626+0000 7fe971e73740 10 client.25541 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.626+0000 7fe971e73740 10 client.25541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.626+0000 7fe971e73740 20 client.25541 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.626+0000 7fe971e73740 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.626+0000 7fe971e73740 4 client.25541 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.626+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3060ef720 con 0x55d3064650e0 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.627+0000 7fe96d039640 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.627+0000 7fe96d039640 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064674f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8736ff103a6d40dd server_cookie=ee668b3362e2dad in_seq=0 out_seq=0 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.627+0000 7fe950ff9640 10 client.25541 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe950ff9640 1 -- 172.21.15.17:0/2222246940 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3060ef720 con 0x55d3064650e0 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe950ff9640 10 client.25541 handle_command_reply: tid=0 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 client.25541 shutdown 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 msgr2=0x55d306452600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d306452600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe95809d950 tx=0x7fe95800f960 comp rx=0 tx=0).stop 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 shutdown_connections 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064674f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe950ff9640 0 client.25541 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d306452b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d306452600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 --2- 172.21.15.17:0/2222246940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 >> 172.21.15.17:0/2222246940 conn(0x55d3064463b0 msgr2=0x55d306448100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 shutdown_connections 2024-02-14T15:04:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 1 -- 172.21.15.17:0/2222246940 wait complete. 2024-02-14T15:04:52.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.628+0000 7fe971e73740 20 client.25541 trim_cache size 0 max 16384 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.651+0000 7fe971e73740 1 Processor -- start 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe971e73740 1 -- start start 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe971e73740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe971e73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d30644f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe971e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d30644f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe971e73740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d306338130 con 0x55d30644bce0 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe971e73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3060ef720 con 0x55d30644c690 2024-02-14T15:04:52.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe971e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30644b040 con 0x55d306455210 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55576/0 (socket says 172.21.15.17:55576) 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 -- 172.21.15.17:0/3879191749 learned_addr learned my addr 172.21.15.17:0/3879191749 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d83a640 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d30644f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96c838640 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d30644f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 -- 172.21.15.17:0/3879191749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 msgr2=0x55d30644f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d30644f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 -- 172.21.15.17:0/3879191749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 msgr2=0x55d30644f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d30644f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.652+0000 7fe96d039640 1 -- 172.21.15.17:0/3879191749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d30634dc90 con 0x55d30644bce0 2024-02-14T15:04:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.653+0000 7fe96d039640 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064547f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe964009230 tx=0x7fe96402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323e27b7d44b5a06 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.653+0000 7fe952ffd640 1 -- 172.21.15.17:0/3879191749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964003810 con 0x55d30644bce0 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.653+0000 7fe952ffd640 1 -- 172.21.15.17:0/3879191749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe96405b070 con 0x55d30644bce0 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.653+0000 7fe952ffd640 1 -- 172.21.15.17:0/3879191749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964005490 con 0x55d30644bce0 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 -- 172.21.15.17:0/3879191749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 msgr2=0x55d3064547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064547f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe964009230 tx=0x7fe96402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 -- 172.21.15.17:0/3879191749 shutdown_connections 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d306455210 0x55d30644f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644c690 0x55d30644f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 --2- 172.21.15.17:0/3879191749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d30644bce0 0x55d3064547f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 -- 172.21.15.17:0/3879191749 >> 172.21.15.17:0/3879191749 conn(0x55d3064463b0 msgr2=0x55d30644d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 -- 172.21.15.17:0/3879191749 shutdown_connections 2024-02-14T15:04:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 -- 172.21.15.17:0/3879191749 wait complete. 2024-02-14T15:04:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 -- start start 2024-02-14T15:04:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.654+0000 7fe971e73740 1 Processor -- start 2024-02-14T15:04:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe971e73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30645c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe971e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d30644c690 0x55d30645c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe971e73740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d306455210 0x55d30645a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe971e73740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30645adf0 con 0x55d306455210 2024-02-14T15:04:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe971e73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30645af60 con 0x55d30644bce0 2024-02-14T15:04:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe971e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30645b120 con 0x55d30644c690 2024-02-14T15:04:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe971e73740 10 client.0 _ll_register_callbacks cb 0x7fe964009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d83a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d306455210 0x55d30645a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96c838640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d30644c690 0x55d30645c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d039640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30645c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96c838640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d30644c690 0x55d30645c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:45826/0 (socket says 172.21.15.17:45826) 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96c838640 1 -- 172.21.15.17:0/2726819569 learned_addr learned my addr 172.21.15.17:0/2726819569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d83a640 1 -- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d30644c690 msgr2=0x55d30645c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d83a640 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d30644c690 0x55d30645c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d83a640 1 -- 172.21.15.17:0/2726819569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 msgr2=0x55d30645c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d83a640 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30645c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d83a640 1 -- 172.21.15.17:0/2726819569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d306379e30 con 0x55d306455210 2024-02-14T15:04:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96c838640 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d30644c690 0x55d30645c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe96d83a640 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d306455210 0x55d30645a720 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe960012cf0 tx=0x7fe960011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7808907315be41 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.655+0000 7fe950ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe950ff9640 1 -- 172.21.15.17:0/2726819569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960009e20 con 0x55d306455210 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe950ff9640 1 -- 172.21.15.17:0/2726819569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe960004b20 con 0x55d306455210 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d306451ef0 con 0x55d306455210 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe950ff9640 1 -- 172.21.15.17:0/2726819569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe960004ce0 con 0x55d306455210 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 10 client.16143 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d306373aa0 con 0x55d306455210 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe950ff9640 1 -- 172.21.15.17:0/2726819569 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d306451ef0 con 0x55d306455210 2024-02-14T15:04:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe950ff9640 1 -- 172.21.15.17:0/2726819569 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe96001d790 con 0x55d306455210 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 10 client.16143 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 10 client.16143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 20 client.16143 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 4 client.16143 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.656+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe964004df0 con 0x55d3064650e0 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe96d039640 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe950ff9640 10 client.16143 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe96d039640 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064beb60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23b8dc01ec0504df server_cookie=e1b0daf63d2ee6bf in_seq=0 out_seq=0 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe950ff9640 1 -- 172.21.15.17:0/2726819569 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe964004df0 con 0x55d3064650e0 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe950ff9640 10 client.16143 handle_command_reply: tid=0 2024-02-14T15:04:52.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 client.16143 shutdown 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d306455210 msgr2=0x55d30645a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d306455210 0x55d30645a720 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe960012cf0 tx=0x7fe960011810 comp rx=0 tx=0).stop 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 shutdown_connections 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55d3064650e0 0x55d3064beb60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe950ff9640 0 client.16143 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d306455210 0x55d30645a720 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d30644c690 0x55d30645c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 --2- 172.21.15.17:0/2726819569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d30644bce0 0x55d30645c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.657+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 >> 172.21.15.17:0/2726819569 conn(0x55d3064463b0 msgr2=0x55d306450990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 shutdown_connections 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe971e73740 1 -- 172.21.15.17:0/2726819569 wait complete. 2024-02-14T15:04:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe971e73740 20 client.16143 trim_cache size 0 max 16384 2024-02-14T15:04:52.667 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe9540a2350 msgr2=0x7fe9540a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe9540a2350 0x7fe9540a4810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe964006010 tx=0x7fe964005e40 comp rx=0 tx=0).stop 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102d70 msgr2=0x7fe96819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102d70 0x7fe96819dad0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe958009b60 tx=0x7fe958007590 comp rx=0 tx=0).stop 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 shutdown_connections 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe9540a2350 0x7fe9540a4810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe968102d70 0x7fe96819dad0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe96810b820 0x7fe9681976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 --2- 172.21.15.17:0/1909028598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe968102450 0x7fe96819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.658+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 >> 172.21.15.17:0/1909028598 conn(0x7fe9680772e0 msgr2=0x7fe9680fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.659+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 shutdown_connections 2024-02-14T15:04:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.659+0000 7fe96f29d640 1 -- 172.21.15.17:0/1909028598 wait complete. 2024-02-14T15:04:52.670 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:04:52.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-14T15:04:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 -- 172.21.15.17:0/667229147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c081008f0 msgr2=0x7f7c0810d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 --2- 172.21.15.17:0/667229147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c081008f0 0x7f7c0810d640 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009900 tx=0x7f7bfc02bcd0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 -- 172.21.15.17:0/667229147 shutdown_connections 2024-02-14T15:04:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 --2- 172.21.15.17:0/667229147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c081008f0 0x7f7c0810d640 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 --2- 172.21.15.17:0/667229147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c0810db80 0x7f7c0810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 --2- 172.21.15.17:0/667229147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c080fffd0 0x7f7c081003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 -- 172.21.15.17:0/667229147 >> 172.21.15.17:0/667229147 conn(0x7f7c080772e0 msgr2=0x7f7c080776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 -- 172.21.15.17:0/667229147 shutdown_connections 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.779+0000 7f7c0e977640 1 -- 172.21.15.17:0/667229147 wait complete. 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 Processor -- start 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 -- start start 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c080fffd0 0x7f7c08199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c0810db80 0x7f7c0819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0819e680 0x7f7c0819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0819f070 con 0x7f7c080fffd0 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0819f1e0 con 0x7f7c0810db80 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0e977640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0819f350 con 0x7f7c0819e680 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c080fffd0 0x7f7c08199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c080fffd0 0x7f7c08199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55594/0 (socket says 172.21.15.17:55594) 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c07fff640 1 -- 172.21.15.17:0/3809599485 learned_addr learned my addr 172.21.15.17:0/3809599485 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c077fe640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c0810db80 0x7f7c0819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.780+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0819e680 0x7f7c0819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c07fff640 1 -- 172.21.15.17:0/3809599485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0819e680 msgr2=0x7f7c0819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c07fff640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0819e680 0x7f7c0819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c07fff640 1 -- 172.21.15.17:0/3809599485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c0810db80 msgr2=0x7f7c0819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c07fff640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c0810db80 0x7f7c0819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c07fff640 1 -- 172.21.15.17:0/3809599485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c00009840 con 0x7f7c080fffd0 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0819e680 0x7f7c0819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c07fff640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c080fffd0 0x7f7c08199dc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800e9f0 tx=0x7f7bf800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c057fa640 1 -- 172.21.15.17:0/3809599485 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8009720 con 0x7f7c080fffd0 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc009560 con 0x7f7c080fffd0 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c057fa640 1 -- 172.21.15.17:0/3809599485 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bf800cd80 con 0x7f7c080fffd0 2024-02-14T15:04:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c057fa640 1 -- 172.21.15.17:0/3809599485 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8010690 con 0x7f7c080fffd0 2024-02-14T15:04:52.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.781+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c081a6c20 con 0x7f7c080fffd0 2024-02-14T15:04:52.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c057fa640 1 -- 172.21.15.17:0/3809599485 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f7bf80046c0 con 0x7f7c080fffd0 2024-02-14T15:04:52.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 Processor -- start 2024-02-14T15:04:52.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 -- start start 2024-02-14T15:04:52.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c057fa640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7bf00a2350 0x7f7bf00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fddb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fd2ec80 con 0x55ff6fdd0690 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fd3e320 con 0x55ff6fdd9210 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c1161e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fd3b850 con 0x55ff6fdcfce0 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c057fa640 1 -- 172.21.15.17:0/3809599485 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f7bf8014070 con 0x7f7c080fffd0 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7bf00a2350 0x7f7bf00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.784+0000 7f7c077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55596/0 (socket says 172.21.15.17:55596) 2024-02-14T15:04:52.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 -- 172.21.15.17:0/1396926130 learned_addr learned my addr 172.21.15.17:0/1396926130 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c07fff640 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fddb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 -- 172.21.15.17:0/1396926130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 msgr2=0x55ff6fdd00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 -- 172.21.15.17:0/1396926130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 msgr2=0x55ff6fddb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fddb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 -- 172.21.15.17:0/1396926130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6fcfde30 con 0x55ff6fdd0690 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7bf00a2350 0x7f7bf00a4810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c0806a920 tx=0x7f7c00013d10 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7c077fe640 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd8c40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009900 tx=0x7f7bfc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b761e4f59a7898c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.785+0000 7f7bee7fc640 1 -- 172.21.15.17:0/1396926130 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004740 con 0x55ff6fdd0690 2024-02-14T15:04:52.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7bee7fc640 1 -- 172.21.15.17:0/1396926130 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bfc0048a0 con 0x55ff6fdd0690 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7bee7fc640 1 -- 172.21.15.17:0/1396926130 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc005050 con 0x55ff6fdd0690 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 -- 172.21.15.17:0/1396926130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 msgr2=0x55ff6fdd8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd8c40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009900 tx=0x7f7bfc005fd0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 -- 172.21.15.17:0/1396926130 shutdown_connections 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd8c40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fddb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1396926130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 -- 172.21.15.17:0/1396926130 >> 172.21.15.17:0/1396926130 conn(0x55ff6fdca3b0 msgr2=0x55ff6fdca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 -- 172.21.15.17:0/1396926130 shutdown_connections 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.786+0000 7f7c1161e740 1 -- 172.21.15.17:0/1396926130 wait complete. 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 -- start start 2024-02-14T15:04:52.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 Processor -- start 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdd9210 0x55ff6fdd6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fd37030 con 0x55ff6fdd0690 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fa0a430 con 0x55ff6fdcfce0 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6f9c6e00 con 0x55ff6fdd9210 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c1161e740 10 client.0 _ll_register_callbacks cb 0x55ff6fa47be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55598/0 (socket says 172.21.15.17:55598) 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c077fe640 1 -- 172.21.15.17:0/4263308349 learned_addr learned my addr 172.21.15.17:0/4263308349 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c07fff640 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.787+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdd9210 0x55ff6fdd6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c077fe640 1 -- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdd9210 msgr2=0x55ff6fdd6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c077fe640 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdd9210 0x55ff6fdd6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c077fe640 1 -- 172.21.15.17:0/4263308349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 msgr2=0x55ff6fdd6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c077fe640 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c077fe640 1 -- 172.21.15.17:0/4263308349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c08110b00 con 0x55ff6fdd0690 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c077fe640 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd6690 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc047000 tx=0x7f7bfc02c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717796077e58f14d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7be77fe640 1 -- 172.21.15.17:0/4263308349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004740 con 0x55ff6fdd0690 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff6fa367e0 con 0x55ff6fdd0690 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7be77fe640 1 -- 172.21.15.17:0/4263308349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f7bfc004ff0 con 0x55ff6fdd0690 2024-02-14T15:04:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7be77fe640 1 -- 172.21.15.17:0/4263308349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bfc0041f0 con 0x55ff6fdd0690 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c1161e740 10 client.16155 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.788+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff6fd356b0 con 0x55ff6fdd0690 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7be77fe640 1 -- 172.21.15.17:0/4263308349 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ff6fa367e0 con 0x55ff6fdd0690 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7be77fe640 1 -- 172.21.15.17:0/4263308349 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7bfc03c050 con 0x55ff6fdd0690 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7c1161e740 10 client.16155 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7c1161e740 10 client.16155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7c1161e740 20 client.16155 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7c1161e740 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fdf1630 0x55ff6fdf3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7c1161e740 4 client.16155 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff6fa73720 con 0x55ff6fdf1630 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.789+0000 7f7c07fff640 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fdf1630 0x55ff6fdf3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.790+0000 7f7be77fe640 10 client.16155 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.790+0000 7f7c07fff640 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fdf1630 0x55ff6fdf3a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5aae2eaed83e6f33 server_cookie=10daf13368b76654 in_seq=0 out_seq=0 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.790+0000 7f7be77fe640 1 -- 172.21.15.17:0/4263308349 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff6fa73720 con 0x55ff6fdf1630 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.790+0000 7f7be77fe640 10 client.16155 handle_command_reply: tid=0 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.790+0000 7f7c1161e740 1 client.16155 shutdown 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 msgr2=0x55ff6fdd6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd6690 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc047000 tx=0x7f7bfc02c3f0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 shutdown_connections 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fdf1630 0x55ff6fdf3a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd0690 0x55ff6fdd6690 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdd9210 0x55ff6fdd6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 --2- 172.21.15.17:0/4263308349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 >> 172.21.15.17:0/4263308349 conn(0x55ff6fdca3b0 msgr2=0x55ff6fdcc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7be77fe640 0 client.16155 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 shutdown_connections 2024-02-14T15:04:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 1 -- 172.21.15.17:0/4263308349 wait complete. 2024-02-14T15:04:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.791+0000 7f7c1161e740 20 client.16155 trim_cache size 0 max 16384 2024-02-14T15:04:52.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 Processor -- start 2024-02-14T15:04:52.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 -- start start 2024-02-14T15:04:52.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd9210 0x55ff6fdf83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdf8930 0x55ff6fdd1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fcbc130 con 0x55ff6fdd9210 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fdced70 con 0x55ff6fdcfce0 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c1161e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fdee540 con 0x55ff6fdf8930 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd9210 0x55ff6fdf83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd9210 0x55ff6fdf83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55608/0 (socket says 172.21.15.17:55608) 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 -- 172.21.15.17:0/3426645820 learned_addr learned my addr 172.21.15.17:0/3426645820 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c07fff640 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdf8930 0x55ff6fdd1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 -- 172.21.15.17:0/3426645820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdf8930 msgr2=0x55ff6fdd1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdf8930 0x55ff6fdd1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 -- 172.21.15.17:0/3426645820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 msgr2=0x55ff6fdd8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.803+0000 7f7c077fe640 1 -- 172.21.15.17:0/3426645820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6fcd1c90 con 0x55ff6fdd9210 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.804+0000 7f7c077fe640 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd9210 0x55ff6fdf83f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009900 tx=0x7f7bfc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e699885339a690 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.804+0000 7f7bedffb640 1 -- 172.21.15.17:0/3426645820 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc005850 con 0x55ff6fdd9210 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.804+0000 7f7bedffb640 1 -- 172.21.15.17:0/3426645820 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bfc04e070 con 0x55ff6fdd9210 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.804+0000 7f7bedffb640 1 -- 172.21.15.17:0/3426645820 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc005050 con 0x55ff6fdd9210 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- 172.21.15.17:0/3426645820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd9210 msgr2=0x55ff6fdf83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd9210 0x55ff6fdf83f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009900 tx=0x7f7bfc005fd0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- 172.21.15.17:0/3426645820 shutdown_connections 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdd9210 0x55ff6fdf83f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdf8930 0x55ff6fdd1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 --2- 172.21.15.17:0/3426645820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdcfce0 0x55ff6fdd8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- 172.21.15.17:0/3426645820 >> 172.21.15.17:0/3426645820 conn(0x55ff6fdca3b0 msgr2=0x55ff6fdca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- 172.21.15.17:0/3426645820 shutdown_connections 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- 172.21.15.17:0/3426645820 wait complete. 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- start start 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 Processor -- start 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fde7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fde78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdf8930 0x55ff6fde7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fe47b80 con 0x55ff6fdf8930 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fe47cc0 con 0x55ff6fdd9210 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.805+0000 7f7c1161e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6fe47e00 con 0x55ff6fdcfce0 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c1161e740 10 client.0 _ll_register_callbacks cb 0x7f7bf8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:52.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fde7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fde7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:45894/0 (socket says 172.21.15.17:45894) 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 -- 172.21.15.17:0/1149513647 learned_addr learned my addr 172.21.15.17:0/1149513647 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdf8930 0x55ff6fde7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c077fe640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fde78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 -- 172.21.15.17:0/1149513647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 msgr2=0x55ff6fde78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fde78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 -- 172.21.15.17:0/1149513647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdf8930 msgr2=0x55ff6fde7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdf8930 0x55ff6fde7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 -- 172.21.15.17:0/1149513647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff6fd356b0 con 0x55ff6fdcfce0 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c077fe640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fde78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:52.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c0cf14640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdf8930 0x55ff6fde7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7c07fff640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fde7380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8097340 tx=0x7f7bf8004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f54b34d263e47d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7be77fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.806+0000 7f7be77fe640 1 -- 172.21.15.17:0/1149513647 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80a36d0 con 0x55ff6fdcfce0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7be77fe640 1 -- 172.21.15.17:0/1149513647 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f7bf80975a0 con 0x55ff6fdcfce0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bfc0048a0 con 0x55ff6fdcfce0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7be77fe640 1 -- 172.21.15.17:0/1149513647 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bf8097760 con 0x55ff6fdcfce0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 10 client.25696 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff6fcf7aa0 con 0x55ff6fdcfce0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7be77fe640 1 -- 172.21.15.17:0/1149513647 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bf80a3240 con 0x55ff6fdcfce0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7be77fe640 1 -- 172.21.15.17:0/1149513647 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7bf801f060 con 0x55ff6fdcfce0 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 10 client.25696 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:52.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 10 client.25696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 20 client.25696 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fe49670 0x55ff6fe4ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 4 client.25696 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f7bfc0048a0 con 0x55ff6fe49670 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.807+0000 7f7c077fe640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fe49670 0x55ff6fe4ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7be77fe640 10 client.25696 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c077fe640 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fe49670 0x55ff6fe4ba60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da76d52bf572934c server_cookie=9c8dc9bdcf3af70d in_seq=0 out_seq=0 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7be77fe640 1 -- 172.21.15.17:0/1149513647 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f7bfc0048a0 con 0x55ff6fe49670 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7be77fe640 10 client.25696 handle_command_reply: tid=0 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 client.25696 shutdown 2024-02-14T15:04:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 msgr2=0x55ff6fde7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fde7380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8097340 tx=0x7f7bf8004a70 comp rx=0 tx=0).stop 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 shutdown_connections 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55ff6fe49670 0x55ff6fe4ba60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff6fdf8930 0x55ff6fde7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff6fdd9210 0x55ff6fde78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 --2- 172.21.15.17:0/1149513647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff6fdcfce0 0x55ff6fde7380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 >> 172.21.15.17:0/1149513647 conn(0x55ff6fdca3b0 msgr2=0x55ff6fdd1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7be77fe640 0 client.25696 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 shutdown_connections 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.808+0000 7f7c1161e740 1 -- 172.21.15.17:0/1149513647 wait complete. 2024-02-14T15:04:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c1161e740 20 client.25696 trim_cache size 0 max 16384 2024-02-14T15:04:52.818 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7bf00a2350 msgr2=0x7f7bf00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7bf00a2350 0x7f7bf00a4810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c0806a920 tx=0x7f7c00013d10 comp rx=0 tx=0).stop 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c080fffd0 msgr2=0x7f7c08199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c080fffd0 0x7f7c08199dc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800e9f0 tx=0x7f7bf800edf0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 shutdown_connections 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7bf00a2350 0x7f7bf00a4810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0819e680 0x7f7c0819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c0810db80 0x7f7c0819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 --2- 172.21.15.17:0/3809599485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c080fffd0 0x7f7c08199dc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 >> 172.21.15.17:0/3809599485 conn(0x7f7c080772e0 msgr2=0x7f7c0810f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.809+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 shutdown_connections 2024-02-14T15:04:52.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.810+0000 7f7c0e977640 1 -- 172.21.15.17:0/3809599485 wait complete. 2024-02-14T15:04:52.821 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:04:52.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-14T15:04:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 -- 172.21.15.17:0/2681489234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 msgr2=0x7f6eb0100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/2681489234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb0100720 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009930 tx=0x7f6eac02bc30 comp rx=0 tx=0).stop 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 -- 172.21.15.17:0/2681489234 shutdown_connections 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/2681489234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6eb0105410 0x7f6eb010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/2681489234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6eb0100cf0 0x7f6eb0104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/2681489234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb0100720 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 -- 172.21.15.17:0/2681489234 >> 172.21.15.17:0/2681489234 conn(0x7f6eb00772e0 msgr2=0x7f6eb00776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 -- 172.21.15.17:0/2681489234 shutdown_connections 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.946+0000 7f6eb79e2640 1 -- 172.21.15.17:0/2681489234 wait complete. 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 Processor -- start 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 -- start start 2024-02-14T15:04:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6eb0105410 0x7f6eb019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6eb01976d0 0x7f6eb0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb01981a0 con 0x7f6eb0100340 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0198310 con 0x7f6eb0105410 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.947+0000 7f6eb79e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb01984d0 con 0x7f6eb01976d0 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55624/0 (socket says 172.21.15.17:55624) 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 -- 172.21.15.17:0/3380655250 learned_addr learned my addr 172.21.15.17:0/3380655250 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb5f7f640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6eb01976d0 0x7f6eb0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6eb0105410 0x7f6eb019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 -- 172.21.15.17:0/3380655250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6eb01976d0 msgr2=0x7f6eb0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6eb01976d0 0x7f6eb0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 -- 172.21.15.17:0/3380655250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6eb0105410 msgr2=0x7f6eb019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6eb0105410 0x7f6eb019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 -- 172.21.15.17:0/3380655250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea0009490 con 0x7f6eb0100340 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb5f7f640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6eb01976d0 0x7f6eb0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb577e640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb019d610 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009230 tx=0x7f6eac02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6ea67fc640 1 -- 172.21.15.17:0/3380655250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004350 con 0x7f6eb0100340 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6ea67fc640 1 -- 172.21.15.17:0/3380655250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eac0044b0 con 0x7f6eb0100340 2024-02-14T15:04:52.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac009560 con 0x7f6eb0100340 2024-02-14T15:04:52.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6ea67fc640 1 -- 172.21.15.17:0/3380655250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac003d70 con 0x7f6eb0100340 2024-02-14T15:04:52.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.948+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb01a47b0 con 0x7f6eb0100340 2024-02-14T15:04:52.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.951+0000 7f6ea67fc640 1 -- 172.21.15.17:0/3380655250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f6eac004df0 con 0x7f6eb0100340 2024-02-14T15:04:52.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.951+0000 7f6ea67fc640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6e9c0a23d0 0x7f6e9c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6ea67fc640 1 -- 172.21.15.17:0/3380655250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6eac0f0480 con 0x7f6eb0100340 2024-02-14T15:04:52.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6e9c0a23d0 0x7f6e9c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 Processor -- start 2024-02-14T15:04:52.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 -- start start 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecb7c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb84c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb87660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecadac80 con 0x5631ecb7c690 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecaea320 con 0x5631ecb7bce0 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.952+0000 7f6eba5ac740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecae7850 con 0x5631ecb85210 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6e9c0a23d0 0x7f6e9c0a4890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0198f20 tx=0x7f6ea0005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecb7c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb4f7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb84c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecb7c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57500/0 (socket says 172.21.15.17:57500) 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 -- 172.21.15.17:0/1524019690 learned_addr learned my addr 172.21.15.17:0/1524019690 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb4f7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb84c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55636/0 (socket says 172.21.15.17:55636) 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb5f7f640 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb87660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 -- 172.21.15.17:0/1524019690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 msgr2=0x5631ecb87660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb87660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 -- 172.21.15.17:0/1524019690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 msgr2=0x5631ecb84c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb84c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 -- 172.21.15.17:0/1524019690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631ecaa9e30 con 0x5631ecb7bce0 2024-02-14T15:04:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.953+0000 7f6eb577e640 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecb7c0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009900 tx=0x7f6eac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3158425c6db7077 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.954+0000 7f6e9b7fe640 1 -- 172.21.15.17:0/1524019690 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0bc9d0 con 0x5631ecb7bce0 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.954+0000 7f6e9b7fe640 1 -- 172.21.15.17:0/1524019690 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eac0dc040 con 0x5631ecb7bce0 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1524019690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 msgr2=0x5631ecb7c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecb7c0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009900 tx=0x7f6eac005fd0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1524019690 shutdown_connections 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb84c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb87660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1524019690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecb7c0c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1524019690 >> 172.21.15.17:0/1524019690 conn(0x5631ecb763b0 msgr2=0x5631ecb76790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1524019690 shutdown_connections 2024-02-14T15:04:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1524019690 wait complete. 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 -- start start 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.955+0000 7f6eba5ac740 1 Processor -- start 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.956+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7bce0 0x5631ecb821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.956+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7c690 0x5631ecb826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.956+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb85210 0x5631ecb82c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.956+0000 7f6eba5ac740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecae3030 con 0x5631ecb7bce0 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.956+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7bce0 0x5631ecb821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.956+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7bce0 0x5631ecb821b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55650/0 (socket says 172.21.15.17:55650) 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.956+0000 7f6eb577e640 1 -- 172.21.15.17:0/822086849 learned_addr learned my addr 172.21.15.17:0/822086849 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ec7b6430 con 0x5631ecb85210 2024-02-14T15:04:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7c690 0x5631ecb826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb5f7f640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb85210 0x5631ecb82c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ec772e00 con 0x5631ecb7c690 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eba5ac740 10 client.0 _ll_register_callbacks cb 0x5631ec7f3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb4f7d640 1 -- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb85210 msgr2=0x5631ecb82c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb85210 0x5631ecb82c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb4f7d640 1 -- 172.21.15.17:0/822086849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7bce0 msgr2=0x5631ecb821b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7bce0 0x5631ecb821b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb4f7d640 1 -- 172.21.15.17:0/822086849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb01a47b0 con 0x5631ecb7c690 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb577e640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7bce0 0x5631ecb821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7c690 0x5631ecb826f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5631ecb7ca90 tx=0x7f6ea0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51501c75ffb9f61d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.957+0000 7f6e98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6e98ff9640 1 -- 172.21.15.17:0/822086849 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0020e50 con 0x5631ecb7c690 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eac0bade0 con 0x5631ecb7c690 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6e98ff9640 1 -- 172.21.15.17:0/822086849 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6ea0018070 con 0x5631ecb7c690 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6e98ff9640 1 -- 172.21.15.17:0/822086849 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea0029440 con 0x5631ecb7c690 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6eba5ac740 10 client.25708 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5631ecae16b0 con 0x5631ecb7c690 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6e98ff9640 1 -- 172.21.15.17:0/822086849 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eac0bade0 con 0x5631ecb7c690 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6e98ff9640 1 -- 172.21.15.17:0/822086849 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ea00287f0 con 0x5631ecb7c690 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6eba5ac740 10 client.25708 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6eba5ac740 10 client.25708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6eba5ac740 20 client.25708 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.958+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecb99690 0x5631ecb9bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.959+0000 7f6eba5ac740 4 client.25708 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.959+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631ec7e27e0 con 0x5631ecb99690 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.959+0000 7f6eb577e640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecb99690 0x5631ecb9bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.959+0000 7f6e98ff9640 10 client.25708 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.959+0000 7f6eb577e640 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecb99690 0x5631ecb9bab0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9b065e2f155282 server_cookie=a6a3a890ea93a509 in_seq=0 out_seq=0 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6e98ff9640 1 -- 172.21.15.17:0/822086849 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631ec7e27e0 con 0x5631ecb99690 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6e98ff9640 10 client.25708 handle_command_reply: tid=0 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 client.25708 shutdown 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7c690 msgr2=0x5631ecb826f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7c690 0x5631ecb826f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5631ecb7ca90 tx=0x7f6ea0004730 comp rx=0 tx=0).stop 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 shutdown_connections 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecb99690 0x5631ecb9bab0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb85210 0x5631ecb82c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7c690 0x5631ecb826f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/822086849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7bce0 0x5631ecb821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 >> 172.21.15.17:0/822086849 conn(0x5631ecb763b0 msgr2=0x5631ecb78100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6e98ff9640 0 client.25708 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 shutdown_connections 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.960+0000 7f6eba5ac740 1 -- 172.21.15.17:0/822086849 wait complete. 2024-02-14T15:04:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.961+0000 7f6eba5ac740 20 client.25708 trim_cache size 0 max 16384 2024-02-14T15:04:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.973+0000 7f6eba5ac740 1 Processor -- start 2024-02-14T15:04:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.973+0000 7f6eba5ac740 1 -- start start 2024-02-14T15:04:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.973+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7bce0 0x5631ecb84890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.973+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7c690 0x5631ecb94be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.973+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb85210 0x5631ecb7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eba5ac740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631eca68130 con 0x5631ecb85210 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eba5ac740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecb786f0 con 0x5631ecb7c690 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eba5ac740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecb87860 con 0x5631ecb7bce0 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb5f7f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb85210 0x5631ecb7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb5f7f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb85210 0x5631ecb7f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55664/0 (socket says 172.21.15.17:55664) 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb5f7f640 1 -- 172.21.15.17:0/1787715967 learned_addr learned my addr 172.21.15.17:0/1787715967 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb577e640 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7bce0 0x5631ecb84890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7c690 0x5631ecb94be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb577e640 1 -- 172.21.15.17:0/1787715967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7c690 msgr2=0x5631ecb94be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb577e640 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7c690 0x5631ecb94be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb577e640 1 -- 172.21.15.17:0/1787715967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb85210 msgr2=0x5631ecb7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb577e640 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb85210 0x5631ecb7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb577e640 1 -- 172.21.15.17:0/1787715967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631eca7dc90 con 0x5631ecb7bce0 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb5f7f640 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb85210 0x5631ecb7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6eb577e640 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7bce0 0x5631ecb84890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0ed910 tx=0x7f6eac0bd210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb741bfc99e66233 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.974+0000 7f6e9affd640 1 -- 172.21.15.17:0/1787715967 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0bc9d0 con 0x5631ecb7bce0 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.975+0000 7f6e9affd640 1 -- 172.21.15.17:0/1787715967 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eac0baa70 con 0x5631ecb7bce0 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.975+0000 7f6e9affd640 1 -- 172.21.15.17:0/1787715967 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0c4f40 con 0x5631ecb7bce0 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.975+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1787715967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7bce0 msgr2=0x5631ecb84890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.975+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7bce0 0x5631ecb84890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0ed910 tx=0x7f6eac0bd210 comp rx=0 tx=0).stop 2024-02-14T15:04:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1787715967 shutdown_connections 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb85210 0x5631ecb7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7c690 0x5631ecb94be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/1787715967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb7bce0 0x5631ecb84890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1787715967 >> 172.21.15.17:0/1787715967 conn(0x5631ecb763b0 msgr2=0x5631ecb944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1787715967 shutdown_connections 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 -- 172.21.15.17:0/1787715967 wait complete. 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 -- start start 2024-02-14T15:04:52.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.976+0000 7f6eba5ac740 1 Processor -- start 2024-02-14T15:04:52.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecba07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eba5ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eb4f7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecba07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eb577e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecba07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57516/0 (socket says 172.21.15.17:57516) 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eb4f7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb9eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55680/0 (socket says 172.21.15.17:55680) 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eb577e640 1 -- 172.21.15.17:0/3183934561 learned_addr learned my addr 172.21.15.17:0/3183934561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb9f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecb9f9a0 con 0x5631ecb7c690 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecbefaa0 con 0x5631ecb7bce0 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.977+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631ecbefbe0 con 0x5631ecb85210 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb5f7f640 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb9f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb577e640 1 -- 172.21.15.17:0/3183934561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 msgr2=0x5631ecb9f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb577e640 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb9f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb577e640 1 -- 172.21.15.17:0/3183934561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 msgr2=0x5631ecb9eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb577e640 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb9eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb577e640 1 -- 172.21.15.17:0/3183934561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea8009cc0 con 0x5631ecb7bce0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eba5ac740 10 client.0 _ll_register_callbacks cb 0x7f6eac0b8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eb577e640 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecba07e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005ec0 tx=0x7f6eac0c0ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cefd04634e6b7a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6e98ff9640 10 client.25583 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eac0dc040 con 0x5631ecb7bce0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6e98ff9640 1 -- 172.21.15.17:0/3183934561 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0c51f0 con 0x5631ecb7bce0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6e98ff9640 1 -- 172.21.15.17:0/3183934561 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6eac0bc9d0 con 0x5631ecb7bce0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6e98ff9640 1 -- 172.21.15.17:0/3183934561 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac0c4ce0 con 0x5631ecb7bce0 2024-02-14T15:04:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.978+0000 7f6eba5ac740 10 client.25583 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x5631ecaa9e30 con 0x5631ecb7bce0 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6e98ff9640 1 -- 172.21.15.17:0/3183934561 <== mon.2 v2:172.21.15.50:3300/0 4 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6eac03c070 con 0x5631ecb7bce0 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6eba5ac740 10 client.25583 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6eba5ac740 10 client.25583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6eba5ac740 20 client.25583 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecbf0af0 0x5631ecbf2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6eba5ac740 4 client.25583 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.979+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5631ec81f720 con 0x5631ecbf0af0 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecbf0af0 0x5631ecbf2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6e98ff9640 10 client.25583 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eb4f7d640 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecbf0af0 0x5631ecbf2f90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f72cd9cd66e982 server_cookie=166adfbe2b638137 in_seq=0 out_seq=0 2024-02-14T15:04:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6e98ff9640 1 -- 172.21.15.17:0/3183934561 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5631ec81f720 con 0x5631ecbf0af0 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6e98ff9640 10 client.25583 handle_command_reply: tid=0 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 client.25583 shutdown 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 msgr2=0x5631ecba07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecba07e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005ec0 tx=0x7f6eac0c0ca0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 shutdown_connections 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631ecbf0af0 0x5631ecbf2f90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6e98ff9640 0 client.25583 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631ecb7c690 0x5631ecb9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631ecb85210 0x5631ecb9f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 --2- 172.21.15.17:0/3183934561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631ecb7bce0 0x5631ecba07e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 >> 172.21.15.17:0/3183934561 conn(0x5631ecb763b0 msgr2=0x5631ecb94d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.980+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 shutdown_connections 2024-02-14T15:04:52.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eba5ac740 1 -- 172.21.15.17:0/3183934561 wait complete. 2024-02-14T15:04:52.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eba5ac740 20 client.25583 trim_cache size 0 max 16384 2024-02-14T15:04:52.990 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:04:52.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6e9c0a23d0 msgr2=0x7f6e9c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6e9c0a23d0 0x7f6e9c0a4890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0198f20 tx=0x7f6ea0005f90 comp rx=0 tx=0).stop 2024-02-14T15:04:52.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 msgr2=0x7f6eb019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:52.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb019d610 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009230 tx=0x7f6eac02c6c0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 shutdown_connections 2024-02-14T15:04:52.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6e9c0a23d0 0x7f6e9c0a4890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6eb01976d0 0x7f6eb0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6eb0105410 0x7f6eb019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 --2- 172.21.15.17:0/3380655250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6eb0100340 0x7f6eb019d610 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 >> 172.21.15.17:0/3380655250 conn(0x7f6eb00772e0 msgr2=0x7f6eb0068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 shutdown_connections 2024-02-14T15:04:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:52.981+0000 7f6eb79e2640 1 -- 172.21.15.17:0/3380655250 wait complete. 2024-02-14T15:04:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:52 smithi050 ceph-mon[36512]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:52 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3356020835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:04:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:52 smithi076 ceph-mon[37057]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:52 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3356020835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:04:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:52 smithi017 ceph-mon[32549]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-14T15:04:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:52 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3356020835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:04:53.994 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-14T15:04:53.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:04:53.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 -- 172.21.15.17:0/2098886832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28100cf0 msgr2=0x7fda28104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 --2- 172.21.15.17:0/2098886832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28100cf0 0x7fda28104d30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda14009890 tx=0x7fda1402bb70 comp rx=0 tx=0).stop 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 -- 172.21.15.17:0/2098886832 shutdown_connections 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 --2- 172.21.15.17:0/2098886832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda28105410 0x7fda2810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 --2- 172.21.15.17:0/2098886832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28100cf0 0x7fda28104d30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 --2- 172.21.15.17:0/2098886832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28100340 0x7fda28100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 -- 172.21.15.17:0/2098886832 >> 172.21.15.17:0/2098886832 conn(0x7fda280772e0 msgr2=0x7fda280776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 -- 172.21.15.17:0/2098886832 shutdown_connections 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.107+0000 7fda2ec25640 1 -- 172.21.15.17:0/2098886832 wait complete. 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 Processor -- start 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 -- start start 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda28100340 0x7fda2819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28105410 0x7fda2819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28197700 0x7fda28197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda281981d0 con 0x7fda28197700 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda28198340 con 0x7fda28105410 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2ec25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda28198500 con 0x7fda28100340 2024-02-14T15:04:54.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2d1c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28197700 0x7fda28197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2d1c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28197700 0x7fda28197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55696/0 (socket says 172.21.15.17:55696) 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2d1c2640 1 -- 172.21.15.17:0/1310378943 learned_addr learned my addr 172.21.15.17:0/1310378943 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda27fff640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28105410 0x7fda2819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.108+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda28100340 0x7fda2819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2d1c2640 1 -- 172.21.15.17:0/1310378943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda28100340 msgr2=0x7fda2819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2d1c2640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda28100340 0x7fda2819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2d1c2640 1 -- 172.21.15.17:0/1310378943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28105410 msgr2=0x7fda2819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2d1c2640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28105410 0x7fda2819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2d1c2640 1 -- 172.21.15.17:0/1310378943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c009560 con 0x7fda28197700 2024-02-14T15:04:54.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2d1c2640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28197700 0x7fda28197b80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fda2000ec70 tx=0x7fda2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda25ffb640 1 -- 172.21.15.17:0/1310378943 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20003b70 con 0x7fda28197700 2024-02-14T15:04:54.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda20009840 con 0x7fda28197700 2024-02-14T15:04:54.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda25ffb640 1 -- 172.21.15.17:0/1310378943 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda2000ee80 con 0x7fda28197700 2024-02-14T15:04:54.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda25ffb640 1 -- 172.21.15.17:0/1310378943 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20010810 con 0x7fda28197700 2024-02-14T15:04:54.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.109+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda14009490 con 0x7fda28197700 2024-02-14T15:04:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.112+0000 7fda317f0740 1 Processor -- start 2024-02-14T15:04:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.112+0000 7fda25ffb640 1 -- 172.21.15.17:0/1310378943 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fda20004020 con 0x7fda28197700 2024-02-14T15:04:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.112+0000 7fda317f0740 1 -- start start 2024-02-14T15:04:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda317f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda317f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda317f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda317f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d2878bd0 con 0x5601d28c65f0 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda317f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d282b030 con 0x5601d28bd0f0 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda317f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d2828560 con 0x5601d28bda10 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda25ffb640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fda180a2350 0x7fda180a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55704/0 (socket says 172.21.15.17:55704) 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda27fff640 1 -- 172.21.15.17:0/25100517 learned_addr learned my addr 172.21.15.17:0/25100517 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda27fff640 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda25ffb640 1 -- 172.21.15.17:0/1310378943 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fda20014070 con 0x7fda28197700 2024-02-14T15:04:54.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.113+0000 7fda2d1c2640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fda180a2350 0x7fda180a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda27fff640 1 -- 172.21.15.17:0/25100517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 msgr2=0x5601d28c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda2d1c2640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fda180a2350 0x7fda180a4810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fda200cbe90 tx=0x7fda20009b50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda27fff640 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda27fff640 1 -- 172.21.15.17:0/25100517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 msgr2=0x5601d28c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda27fff640 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda27fff640 1 -- 172.21.15.17:0/25100517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601d27e6290 con 0x5601d28bd0f0 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda27fff640 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28bd4d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda140097b0 tx=0x7fda140058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=304960830199a3b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.114+0000 7fda12ffd640 1 -- 172.21.15.17:0/25100517 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14004330 con 0x5601d28bd0f0 2024-02-14T15:04:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.115+0000 7fda12ffd640 1 -- 172.21.15.17:0/25100517 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda14047070 con 0x5601d28bd0f0 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.115+0000 7fda12ffd640 1 -- 172.21.15.17:0/25100517 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1402c9e0 con 0x5601d28bd0f0 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- 172.21.15.17:0/25100517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 msgr2=0x5601d28bd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28bd4d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda140097b0 tx=0x7fda140058d0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- 172.21.15.17:0/25100517 shutdown_connections 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 --2- 172.21.15.17:0/25100517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28bd4d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- 172.21.15.17:0/25100517 >> 172.21.15.17:0/25100517 conn(0x5601d28b7820 msgr2=0x5601d28b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- 172.21.15.17:0/25100517 shutdown_connections 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- 172.21.15.17:0/25100517 wait complete. 2024-02-14T15:04:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- start start 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 Processor -- start 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d2823d40 con 0x5601d28c65f0 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d27c8e60 con 0x5601d28bd0f0 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.116+0000 7fda317f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d24f74b0 con 0x5601d28bda10 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda317f0740 10 client.0 _ll_register_callbacks cb 0x5601d27df8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55712/0 (socket says 172.21.15.17:55712) 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 -- 172.21.15.17:0/2520572651 learned_addr learned my addr 172.21.15.17:0/2520572651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 -- 172.21.15.17:0/2520572651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 msgr2=0x5601d28c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 -- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 msgr2=0x5601d28c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 -- 172.21.15.17:0/2520572651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2810d7f0 con 0x5601d28c65f0 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda27fff640 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c3b50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fda1402c0b0 tx=0x7fda14005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32058884ca763c0e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.117+0000 7fda07fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:54.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda07fff640 1 -- 172.21.15.17:0/2520572651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1402c8c0 con 0x5601d28c65f0 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda07fff640 1 -- 172.21.15.17:0/2520572651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fda1402ca20 con 0x5601d28c65f0 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601d2849dd0 con 0x5601d28c65f0 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda07fff640 1 -- 172.21.15.17:0/2520572651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda1402cbe0 con 0x5601d28c65f0 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 10 client.16191 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5601d28223c0 con 0x5601d28c65f0 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda07fff640 1 -- 172.21.15.17:0/2520572651 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda1405f020 con 0x5601d28c65f0 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda07fff640 1 -- 172.21.15.17:0/2520572651 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda14032a10 con 0x5601d28c65f0 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 10 client.16191 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 20 client.16191 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d28dea30 0x5601d28e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 4 client.16191 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.118+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601d27f6cb0 con 0x5601d28dea30 2024-02-14T15:04:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.119+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d28dea30 0x5601d28e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.119+0000 7fda07fff640 10 client.16191 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.119+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d28dea30 0x5601d28e0e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24016dd5a39ea872 server_cookie=71654fc49f1c58dc in_seq=0 out_seq=0 2024-02-14T15:04:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda07fff640 1 -- 172.21.15.17:0/2520572651 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601d27f6cb0 con 0x5601d28dea30 2024-02-14T15:04:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda07fff640 10 client.16191 handle_command_reply: tid=0 2024-02-14T15:04:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 client.16191 shutdown 2024-02-14T15:04:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 msgr2=0x5601d28c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c3b50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fda1402c0b0 tx=0x7fda14005d20 comp rx=0 tx=0).stop 2024-02-14T15:04:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 shutdown_connections 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d28dea30 0x5601d28e0e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28c65f0 0x5601d28c3b50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28bda10 0x5601d28c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda07fff640 0 client.16191 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 --2- 172.21.15.17:0/2520572651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bd0f0 0x5601d28c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 >> 172.21.15.17:0/2520572651 conn(0x5601d28b7820 msgr2=0x5601d28b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 shutdown_connections 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 1 -- 172.21.15.17:0/2520572651 wait complete. 2024-02-14T15:04:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.120+0000 7fda317f0740 20 client.16191 trim_cache size 0 max 16384 2024-02-14T15:04:54.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 Processor -- start 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 -- start start 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 0x5601d28c2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d27a9670 con 0x5601d28bd0f0 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d28c3400 con 0x5601d28bda10 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda317f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d28c2960 con 0x5601d28c65f0 2024-02-14T15:04:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55718/0 (socket says 172.21.15.17:55718) 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda27fff640 1 -- 172.21.15.17:0/1162244153 learned_addr learned my addr 172.21.15.17:0/1162244153 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.136+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda27fff640 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 0x5601d28c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda27fff640 1 -- 172.21.15.17:0/1162244153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 msgr2=0x5601d28c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda27fff640 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 0x5601d28c2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda27fff640 1 -- 172.21.15.17:0/1162244153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 msgr2=0x5601d28c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda27fff640 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda27fff640 1 -- 172.21.15.17:0/1162244153 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601d27bec70 con 0x5601d28bd0f0 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:54.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda27fff640 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28c5d40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fda140044e0 tx=0x7fda1402bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b015c2be0a74f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda127fc640 1 -- 172.21.15.17:0/1162244153 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140058b0 con 0x5601d28bd0f0 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda127fc640 1 -- 172.21.15.17:0/1162244153 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda1405f040 con 0x5601d28bd0f0 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.137+0000 7fda127fc640 1 -- 172.21.15.17:0/1162244153 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1402c8a0 con 0x5601d28bd0f0 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 -- 172.21.15.17:0/1162244153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 msgr2=0x5601d28c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28c5d40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fda140044e0 tx=0x7fda1402bb70 comp rx=0 tx=0).stop 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 -- 172.21.15.17:0/1162244153 shutdown_connections 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 0x5601d28c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 --2- 172.21.15.17:0/1162244153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28c5d40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 -- 172.21.15.17:0/1162244153 >> 172.21.15.17:0/1162244153 conn(0x5601d28b7820 msgr2=0x5601d28be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 -- 172.21.15.17:0/1162244153 shutdown_connections 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 -- 172.21.15.17:0/1162244153 wait complete. 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.138+0000 7fda317f0740 1 -- start start 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 1 Processor -- start 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28db8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 0x5601d28d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d28d86f0 con 0x5601d28bd0f0 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d28d8860 con 0x5601d28bda10 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d28d8a20 con 0x5601d28c65f0 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda317f0740 10 client.0 _ll_register_callbacks cb 0x7fda1c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:54.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28db8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28db8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55726/0 (socket says 172.21.15.17:55726) 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda27fff640 1 -- 172.21.15.17:0/923509296 learned_addr learned my addr 172.21.15.17:0/923509296 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.139+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda27fff640 1 -- 172.21.15.17:0/923509296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 msgr2=0x5601d28d8180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda27fff640 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 0x5601d28d8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda27fff640 1 -- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 msgr2=0x5601d28d7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda27fff640 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28d7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda27fff640 1 -- 172.21.15.17:0/923509296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c0125d0 con 0x5601d28bd0f0 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda27fff640 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28db8c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fda14005fd0 tx=0x7fda140057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e24087fa6c55c2a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda07fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda07fff640 1 -- 172.21.15.17:0/923509296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1402c270 con 0x5601d28bd0f0 2024-02-14T15:04:54.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda07fff640 1 -- 172.21.15.17:0/923509296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fda14000e40 con 0x5601d28bd0f0 2024-02-14T15:04:54.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda14047070 con 0x5601d28bd0f0 2024-02-14T15:04:54.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda07fff640 1 -- 172.21.15.17:0/923509296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda14005870 con 0x5601d28bd0f0 2024-02-14T15:04:54.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda317f0740 10 client.16197 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:54.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.140+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5601d28223c0 con 0x5601d28bd0f0 2024-02-14T15:04:54.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.141+0000 7fda07fff640 1 -- 172.21.15.17:0/923509296 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda14047070 con 0x5601d28bd0f0 2024-02-14T15:04:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda07fff640 1 -- 172.21.15.17:0/923509296 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda1402c820 con 0x5601d28bd0f0 2024-02-14T15:04:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda317f0740 10 client.16197 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda317f0740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda317f0740 20 client.16197 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda317f0740 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d2935fc0 0x5601d29383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda317f0740 4 client.16197 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-14T15:04:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5601d2523860 con 0x5601d2935fc0 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.142+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d2935fc0 0x5601d29383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.143+0000 7fda07fff640 10 client.16197 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.143+0000 7fda2c9c1640 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d2935fc0 0x5601d29383b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae996877f8d05af9 server_cookie=ad6a0777e27b4fc2 in_seq=0 out_seq=0 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda07fff640 1 -- 172.21.15.17:0/923509296 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5601d2523860 con 0x5601d2935fc0 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda07fff640 10 client.16197 handle_command_reply: tid=0 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 client.16197 shutdown 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 msgr2=0x5601d28db8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28db8c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fda14005fd0 tx=0x7fda140057e0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 shutdown_connections 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5601d2935fc0 0x5601d29383b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5601d28c65f0 0x5601d28d8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601d28bda10 0x5601d28d7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda07fff640 0 client.16197 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 --2- 172.21.15.17:0/923509296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5601d28bd0f0 0x5601d28db8c0 secure :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fda14005fd0 tx=0x7fda140057e0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.144+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 >> 172.21.15.17:0/923509296 conn(0x5601d28b7820 msgr2=0x5601d28c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 shutdown_connections 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda317f0740 1 -- 172.21.15.17:0/923509296 wait complete. 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda317f0740 20 client.16197 trim_cache size 0 max 16384 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:04:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2024-02-14T15:04:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_tag": "f59a0b82-2b8a-4106-b486-d6268cdf0921", 2024-02-14T15:04:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": "asynchronous" 2024-02-14T15:04:54.152 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:04:54.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fda180a2350 msgr2=0x7fda180a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fda180a2350 0x7fda180a4810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fda200cbe90 tx=0x7fda20009b50 comp rx=0 tx=0).stop 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28197700 msgr2=0x7fda28197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28197700 0x7fda28197b80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fda2000ec70 tx=0x7fda2000c6a0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 shutdown_connections 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fda180a2350 0x7fda180a4810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda28197700 0x7fda28197b80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda28105410 0x7fda2819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 --2- 172.21.15.17:0/1310378943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda28100340 0x7fda2819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 >> 172.21.15.17:0/1310378943 conn(0x7fda280772e0 msgr2=0x7fda280688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.145+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 shutdown_connections 2024-02-14T15:04:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.146+0000 7fda2ec25640 1 -- 172.21.15.17:0/1310378943 wait complete. 2024-02-14T15:04:54.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:04:54.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:04:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 -- 172.21.15.17:0/3677587303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e8410b820 msgr2=0x7f8e8410dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/3677587303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e8410b820 0x7f8e8410dc50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009ca0 tx=0x7f8e7c02c180 comp rx=0 tx=0).stop 2024-02-14T15:04:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 -- 172.21.15.17:0/3677587303 shutdown_connections 2024-02-14T15:04:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/3677587303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e8410b820 0x7f8e8410dc50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/3677587303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e84102d70 0x7f8e8410b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/3677587303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e84102450 0x7f8e84102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 -- 172.21.15.17:0/3677587303 >> 172.21.15.17:0/3677587303 conn(0x7f8e840772e0 msgr2=0x7f8e840776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 -- 172.21.15.17:0/3677587303 shutdown_connections 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.263+0000 7f8e8b55c640 1 -- 172.21.15.17:0/3677587303 wait complete. 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 Processor -- start 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 -- start start 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e84102450 0x7f8e8419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e84102d70 0x7f8e8419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8410b820 0x7f8e84197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84197e00 con 0x7f8e84102d70 2024-02-14T15:04:54.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84197f70 con 0x7f8e84102450 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.264+0000 7f8e8b55c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84198130 con 0x7f8e8410b820 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e84102d70 0x7f8e8419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e84102d70 0x7f8e8419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55734/0 (socket says 172.21.15.17:55734) 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 -- 172.21.15.17:0/1967984991 learned_addr learned my addr 172.21.15.17:0/1967984991 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e89af9640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8410b820 0x7f8e84197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e892f8640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e84102450 0x7f8e8419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 -- 172.21.15.17:0/1967984991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8410b820 msgr2=0x7f8e84197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8410b820 0x7f8e84197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 -- 172.21.15.17:0/1967984991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e84102450 msgr2=0x7f8e8419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e84102450 0x7f8e8419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 -- 172.21.15.17:0/1967984991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c009900 con 0x7f8e84102d70 2024-02-14T15:04:54.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e84102d70 0x7f8e8419daf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400d950 tx=0x7f8e7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e7a7fc640 1 -- 172.21.15.17:0/1967984991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74009bb0 con 0x7f8e84102d70 2024-02-14T15:04:54.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e84198410 con 0x7f8e84102d70 2024-02-14T15:04:54.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e7a7fc640 1 -- 172.21.15.17:0/1967984991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e74009d10 con 0x7f8e84102d70 2024-02-14T15:04:54.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e7a7fc640 1 -- 172.21.15.17:0/1967984991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400f720 con 0x7f8e84102d70 2024-02-14T15:04:54.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.265+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e841a4530 con 0x7f8e84102d70 2024-02-14T15:04:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.268+0000 7f8e7a7fc640 1 -- 172.21.15.17:0/1967984991 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f8e74010a60 con 0x7f8e84102d70 2024-02-14T15:04:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.268+0000 7f8e8e203740 1 Processor -- start 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.268+0000 7f8e8e203740 1 -- start start 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc579ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579cf50 0x55edc57a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc57a5b30 0x55edc57a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e8e203740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc57575b0 con 0x55edc579c630 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e8e203740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc570b2b0 con 0x55edc579cf50 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e8e203740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc57087e0 con 0x55edc57a5b30 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e7a7fc640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e700a23a0 0x7f8e700a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc579ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc579ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55736/0 (socket says 172.21.15.17:55736) 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 -- 172.21.15.17:0/2199682536 learned_addr learned my addr 172.21.15.17:0/2199682536 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e892f8640 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579cf50 0x55edc57a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e88af7640 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc57a5b30 0x55edc57a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e700a23a0 0x7f8e700a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e7a7fc640 1 -- 172.21.15.17:0/1967984991 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8e740ce1b0 con 0x7f8e84102d70 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 -- 172.21.15.17:0/2199682536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc57a5b30 msgr2=0x55edc57a7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc57a5b30 0x55edc57a7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 -- 172.21.15.17:0/2199682536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579cf50 msgr2=0x55edc57a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579cf50 0x55edc57a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e89af9640 1 -- 172.21.15.17:0/2199682536 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edc56c6020 con 0x55edc579c630 2024-02-14T15:04:54.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.269+0000 7f8e88af7640 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc57a5b30 0x55edc57a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.270+0000 7f8e89af9640 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc579ca10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009ca0 tx=0x7f8e7c0349a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8bdf53e9b9eb8f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.270+0000 7f8e89af9640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e700a23a0 0x7f8e700a4860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c036000 tx=0x7f8e7c03c000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.270+0000 7f8e6f7fe640 1 -- 172.21.15.17:0/2199682536 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c048070 con 0x55edc579c630 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.270+0000 7f8e6f7fe640 1 -- 172.21.15.17:0/2199682536 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e7c032470 con 0x55edc579c630 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.270+0000 7f8e6f7fe640 1 -- 172.21.15.17:0/2199682536 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004780 con 0x55edc579c630 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 -- 172.21.15.17:0/2199682536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 msgr2=0x55edc579ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc579ca10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009ca0 tx=0x7f8e7c0349a0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 -- 172.21.15.17:0/2199682536 shutdown_connections 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc57a5b30 0x55edc57a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579cf50 0x55edc57a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 --2- 172.21.15.17:0/2199682536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc579ca10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 -- 172.21.15.17:0/2199682536 >> 172.21.15.17:0/2199682536 conn(0x55edc5796d60 msgr2=0x55edc5797140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 -- 172.21.15.17:0/2199682536 shutdown_connections 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 -- 172.21.15.17:0/2199682536 wait complete. 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 -- start start 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.271+0000 7f8e8e203740 1 Processor -- start 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e8e203740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5703fc0 con 0x55edc57a5b30 2024-02-14T15:04:54.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e8e203740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc53d7430 con 0x55edc579c630 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e8e203740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5393e00 con 0x55edc579cf50 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e8e203740 10 client.0 _ll_register_callbacks cb 0x55edc5685870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e89af9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e88af7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e892f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e892f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57600/0 (socket says 172.21.15.17:57600) 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e88af7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:46036/0 (socket says 172.21.15.17:46036) 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e892f8640 1 -- 172.21.15.17:0/3929588269 learned_addr learned my addr 172.21.15.17:0/3929588269 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e88af7640 1 -- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 msgr2=0x55edc57a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e88af7640 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e88af7640 1 -- 172.21.15.17:0/3929588269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 msgr2=0x55edc57a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e88af7640 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e88af7640 1 -- 172.21.15.17:0/3929588269 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8410e8d0 con 0x55edc579cf50 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e89af9640 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.272+0000 7f8e892f8640 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e88af7640 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a3140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e740cb420 tx=0x7f8e7409a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1cc129752ae3b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e6cff9640 1 -- 172.21.15.17:0/3929588269 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7409a6a0 con 0x55edc579cf50 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e6cff9640 1 -- 172.21.15.17:0/3929588269 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8e7409e740 con 0x55edc579cf50 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edc5440720 con 0x55edc579cf50 2024-02-14T15:04:54.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e6cff9640 1 -- 172.21.15.17:0/3929588269 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e740a2d90 con 0x55edc579cf50 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e8e203740 10 client.25741 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55edc5702640 con 0x55edc579cf50 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.273+0000 7f8e6cff9640 1 -- 172.21.15.17:0/3929588269 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e740a2900 con 0x55edc579cf50 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e6cff9640 1 -- 172.21.15.17:0/3929588269 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e740104d0 con 0x55edc579cf50 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e8e203740 10 client.25741 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e8e203740 10 client.25741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e8e203740 20 client.25741 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e8e203740 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc57ba200 0x55edc57bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e8e203740 4 client.25741 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edc5440720 con 0x55edc57ba200 2024-02-14T15:04:54.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.274+0000 7f8e892f8640 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc57ba200 0x55edc57bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e892f8640 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc57ba200 0x55edc57bc5f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a941d3139cabf9 server_cookie=989405785998a662 in_seq=0 out_seq=0 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e6cff9640 10 client.25741 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e6cff9640 1 -- 172.21.15.17:0/3929588269 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55edc5440720 con 0x55edc57ba200 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e6cff9640 10 client.25741 handle_command_reply: tid=0 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 client.25741 shutdown 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 msgr2=0x55edc57a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a3140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e740cb420 tx=0x7f8e7409a470 comp rx=0 tx=0).stop 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 shutdown_connections 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc57ba200 0x55edc57bc5f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a3140 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 --2- 172.21.15.17:0/3929588269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 >> 172.21.15.17:0/3929588269 conn(0x55edc5796d60 msgr2=0x55edc5798ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.275+0000 7f8e6cff9640 0 client.25741 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.276+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 shutdown_connections 2024-02-14T15:04:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.276+0000 7f8e8e203740 1 -- 172.21.15.17:0/3929588269 wait complete. 2024-02-14T15:04:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.276+0000 7f8e8e203740 20 client.25741 trim_cache size 0 max 16384 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 Processor -- start 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 -- start start 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc57a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc57a5b30 0x55edc57a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5688fd0 con 0x55edc579c630 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc579e840 con 0x55edc57a5b30 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e8e203740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc579c070 con 0x55edc579cf50 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e892f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc57a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e892f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc57a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55758/0 (socket says 172.21.15.17:55758) 2024-02-14T15:04:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e892f8640 1 -- 172.21.15.17:0/1771141300 learned_addr learned my addr 172.21.15.17:0/1771141300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e892f8640 1 -- 172.21.15.17:0/1771141300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 msgr2=0x55edc57a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e89af9640 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc57a5b30 0x55edc57a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.291+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e892f8640 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e892f8640 1 -- 172.21.15.17:0/1771141300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc57a5b30 msgr2=0x55edc57a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e892f8640 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc57a5b30 0x55edc57a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e892f8640 1 -- 172.21.15.17:0/1771141300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edc569eb30 con 0x55edc579c630 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e89af9640 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc57a5b30 0x55edc57a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e892f8640 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc57a5330 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004200 tx=0x7f8e8000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d074767bdc9f7b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e6effd640 1 -- 172.21.15.17:0/1771141300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80016070 con 0x55edc579c630 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e6effd640 1 -- 172.21.15.17:0/1771141300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e80017040 con 0x55edc579c630 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.292+0000 7f8e6effd640 1 -- 172.21.15.17:0/1771141300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80004e40 con 0x55edc579c630 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 -- 172.21.15.17:0/1771141300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 msgr2=0x55edc57a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc57a5330 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004200 tx=0x7f8e8000b390 comp rx=0 tx=0).stop 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 -- 172.21.15.17:0/1771141300 shutdown_connections 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc57a5b30 0x55edc57a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1771141300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc579c630 0x55edc57a5330 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 -- 172.21.15.17:0/1771141300 >> 172.21.15.17:0/1771141300 conn(0x55edc5796d60 msgr2=0x55edc57c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 -- 172.21.15.17:0/1771141300 shutdown_connections 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 -- 172.21.15.17:0/1771141300 wait complete. 2024-02-14T15:04:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 -- start start 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.293+0000 7f8e8e203740 1 Processor -- start 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e8e203740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e8e203740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc57b1660 con 0x55edc57a5b30 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e8e203740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5810720 con 0x55edc579c630 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e8e203740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5810860 con 0x55edc579cf50 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e8e203740 10 client.0 _ll_register_callbacks cb 0x7f8e7c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e89af9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57614/0 (socket says 172.21.15.17:57614) 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 -- 172.21.15.17:0/1303862782 learned_addr learned my addr 172.21.15.17:0/1303862782 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e89af9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55766/0 (socket says 172.21.15.17:55766) 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 -- 172.21.15.17:0/1303862782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 msgr2=0x55edc57afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 -- 172.21.15.17:0/1303862782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 msgr2=0x55edc57b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 -- 172.21.15.17:0/1303862782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e800044f0 con 0x55edc579c630 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e89af9640 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e892f8640 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57af800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e80022040 tx=0x7f8e80004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58b7e20bbe0d47c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.294+0000 7f8e6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e6cff9640 1 -- 172.21.15.17:0/1303862782 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80016040 con 0x55edc579c630 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edc54037e0 con 0x55edc579c630 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e6cff9640 1 -- 172.21.15.17:0/1303862782 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8e80005020 con 0x55edc579c630 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e6cff9640 1 -- 172.21.15.17:0/1303862782 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e800051e0 con 0x55edc579c630 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 10 client.25616 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55edc56c4880 con 0x55edc579c630 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e6cff9640 1 -- 172.21.15.17:0/1303862782 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55edc54037e0 con 0x55edc579c630 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e6cff9640 1 -- 172.21.15.17:0/1303862782 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e80011440 con 0x55edc579c630 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 10 client.25616 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 20 client.25616 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc5811f30 0x55edc5814350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 4 client.25616 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.295+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55edc548f890 con 0x55edc5811f30 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc5811f30 0x55edc5814350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e6cff9640 10 client.25616 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e88af7640 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc5811f30 0x55edc5814350 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e130cb4f11fe7c server_cookie=6815ebe04e0b81c2 in_seq=0 out_seq=0 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e6cff9640 1 -- 172.21.15.17:0/1303862782 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55edc548f890 con 0x55edc5811f30 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e6cff9640 10 client.25616 handle_command_reply: tid=0 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 client.25616 shutdown 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 msgr2=0x55edc57af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57af800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e80022040 tx=0x7f8e80004d20 comp rx=0 tx=0).stop 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 shutdown_connections 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55edc5811f30 0x55edc5814350 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55edc57a5b30 0x55edc57b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e6cff9640 0 client.25616 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edc579cf50 0x55edc57afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 --2- 172.21.15.17:0/1303862782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55edc579c630 0x55edc57af800 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.296+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 >> 172.21.15.17:0/1303862782 conn(0x55edc5796d60 msgr2=0x55edc57a1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 shutdown_connections 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8e203740 1 -- 172.21.15.17:0/1303862782 wait complete. 2024-02-14T15:04:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8e203740 20 client.25616 trim_cache size 0 max 16384 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": { 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: "f59a0b82-2b8a-4106-b486-d6268cdf0921": { 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/", 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: "tag": "f59a0b82-2b8a-4106-b486-d6268cdf0921", 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: "options": "recursive,force" 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:04:54.307 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:04:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e700a23a0 msgr2=0x7f8e700a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e700a23a0 0x7f8e700a4860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c036000 tx=0x7f8e7c03c000 comp rx=0 tx=0).stop 2024-02-14T15:04:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e84102d70 msgr2=0x7f8e8419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e84102d70 0x7f8e8419daf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400d950 tx=0x7f8e7400de10 comp rx=0 tx=0).stop 2024-02-14T15:04:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 shutdown_connections 2024-02-14T15:04:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e700a23a0 0x7f8e700a4860 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e84102d70 0x7f8e8419daf0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8410b820 0x7f8e84197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 --2- 172.21.15.17:0/1967984991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e84102450 0x7f8e8419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 >> 172.21.15.17:0/1967984991 conn(0x7f8e840772e0 msgr2=0x7f8e840fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 shutdown_connections 2024-02-14T15:04:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:04:54.297+0000 7f8e8b55c640 1 -- 172.21.15.17:0/1967984991 wait complete. 2024-02-14T15:04:54.310 INFO:tasks.cephfs.filesystem:scrub status for tag:f59a0b82-2b8a-4106-b486-d6268cdf0921 - {'path': '/', 'tag': 'f59a0b82-2b8a-4106-b486-d6268cdf0921', 'options': 'recursive,force'} 2024-02-14T15:04:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:54 smithi050 ceph-mon[36512]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:55.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:54 smithi050 ceph-mon[36512]: scrub queued for path: / 2024-02-14T15:04:55.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:54 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:55.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:54 smithi050 ceph-mon[36512]: scrub summary: active paths [/] 2024-02-14T15:04:55.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:54 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:54 smithi076 ceph-mon[37057]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:54 smithi076 ceph-mon[37057]: scrub queued for path: / 2024-02-14T15:04:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:54 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:54 smithi076 ceph-mon[37057]: scrub summary: active paths [/] 2024-02-14T15:04:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:54 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:54 smithi017 ceph-mon[32549]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:54 smithi017 ceph-mon[32549]: scrub queued for path: / 2024-02-14T15:04:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:54 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:54 smithi017 ceph-mon[32549]: scrub summary: active paths [/] 2024-02-14T15:04:55.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:54 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:04:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:56 smithi050 ceph-mon[36512]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:57.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:56 smithi050 ceph-mon[36512]: mgrmap e27: x(active, since 90s), standbys: y, z 2024-02-14T15:04:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:56 smithi076 ceph-mon[37057]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:56 smithi076 ceph-mon[37057]: mgrmap e27: x(active, since 90s), standbys: y, z 2024-02-14T15:04:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:56 smithi017 ceph-mon[32549]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-14T15:04:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:56 smithi017 ceph-mon[32549]: mgrmap e27: x(active, since 90s), standbys: y, z 2024-02-14T15:04:57.319 INFO:tasks.workunit.client.0.smithi017.stderr:Updating files: 96% (11853/12222) Updating files: 97% (11856/12222) Updating files: 98% (11978/12222) Updating files: 99% (12100/12222) Updating files: 100% (12222/12222) Updating files: 100% (12222/12222), done. 2024-02-14T15:04:57.664 INFO:tasks.workunit.client.0.smithi017.stderr:Note: switching to 'a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0'. 2024-02-14T15:04:57.664 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-02-14T15:04:57.664 INFO:tasks.workunit.client.0.smithi017.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr:state without impacting any branches by switching back to a branch. 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr: git switch -c 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr:Or undo this operation with: 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-02-14T15:04:57.665 INFO:tasks.workunit.client.0.smithi017.stderr: git switch - 2024-02-14T15:04:57.666 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-02-14T15:04:57.666 INFO:tasks.workunit.client.0.smithi017.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-14T15:04:57.666 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-02-14T15:04:57.666 INFO:tasks.workunit.client.0.smithi017.stderr:HEAD is now at a6fe3fa3adb Merge PR #52258 into wip-vshankar-testing-20240206.051942 2024-02-14T15:04:57.667 DEBUG:teuthology.orchestra.run.smithi017:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-14T15:04:57.698 INFO:tasks.workunit.client.0.smithi017.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-14T15:04:57.700 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-14T15:04:57.700 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-14T15:04:58.466 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-14T15:04:58.538 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-14T15:04:58.579 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-14T15:04:58.581 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-14T15:04:58.581 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-14T15:04:58.620 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-14T15:04:58.624 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:04:58.624 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-14T15:04:58.679 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-02-14T15:04:58.680 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-02-14T15:04:58.681 DEBUG:teuthology.orchestra.run.smithi017:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-02-14T15:04:58.740 INFO:tasks.workunit.client.0.smithi017.stderr:+ dbench 1 2024-02-14T15:04:58.748 INFO:tasks.workunit.client.0.smithi017.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-14T15:04:58.748 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-02-14T15:04:58.748 INFO:tasks.workunit.client.0.smithi017.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-14T15:04:58.758 INFO:tasks.workunit.client.0.smithi017.stdout:0 of 1 processes prepared for launch 0 sec 2024-02-14T15:04:58.858 INFO:tasks.workunit.client.0.smithi017.stdout:1 of 1 processes prepared for launch 0 sec 2024-02-14T15:04:58.858 INFO:tasks.workunit.client.0.smithi017.stdout:releasing clients 2024-02-14T15:04:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:58 smithi050 ceph-mon[36512]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-02-14T15:04:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:04:58 smithi050 ceph-mon[36512]: scrub summary: idle 2024-02-14T15:04:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:58 smithi076 ceph-mon[37057]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-02-14T15:04:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:04:58 smithi076 ceph-mon[37057]: scrub summary: idle 2024-02-14T15:04:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 -- 172.21.15.17:0/3748683827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30100cf0 msgr2=0x7f5f30104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 --2- 172.21.15.17:0/3748683827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30100cf0 0x7f5f30104d30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009930 tx=0x7f5f2402bc30 comp rx=0 tx=0).stop 2024-02-14T15:04:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 -- 172.21.15.17:0/3748683827 shutdown_connections 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 --2- 172.21.15.17:0/3748683827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30100cf0 0x7f5f30104d30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 --2- 172.21.15.17:0/3748683827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f30105410 0x7f5f3010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 --2- 172.21.15.17:0/3748683827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f30100340 0x7f5f30100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 -- 172.21.15.17:0/3748683827 >> 172.21.15.17:0/3748683827 conn(0x7f5f30077220 msgr2=0x7f5f30077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 -- 172.21.15.17:0/3748683827 shutdown_connections 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.244+0000 7f5f36119640 1 -- 172.21.15.17:0/3748683827 wait complete. 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 Processor -- start 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 -- start start 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f30100340 0x7f5f3019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30105410 0x7f5f3019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f301965b0 0x7f5f30196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30196fa0 con 0x7f5f30105410 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30197110 con 0x7f5f301965b0 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f36119640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f301972d0 con 0x7f5f30100340 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.245+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30105410 0x7f5f3019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30105410 0x7f5f3019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55784/0 (socket says 172.21.15.17:55784) 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 -- 172.21.15.17:0/706888671 learned_addr learned my addr 172.21.15.17:0/706888671 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f30100340 0x7f5f3019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f301965b0 0x7f5f30196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 -- 172.21.15.17:0/706888671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f30100340 msgr2=0x7f5f3019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f30100340 0x7f5f3019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 -- 172.21.15.17:0/706888671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f301965b0 msgr2=0x7f5f30196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f301965b0 0x7f5f30196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 -- 172.21.15.17:0/706888671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24009560 con 0x7f5f30105410 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2effd640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30105410 0x7f5f3019c940 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009230 tx=0x7f5f24004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2cff9640 1 -- 172.21.15.17:0/706888671 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240043d0 con 0x7f5f30105410 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f301a32c0 con 0x7f5f30105410 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2cff9640 1 -- 172.21.15.17:0/706888671 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f240052d0 con 0x7f5f30105410 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f2cff9640 1 -- 172.21.15.17:0/706888671 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003d80 con 0x7f5f30105410 2024-02-14T15:04:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.246+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f301a3770 con 0x7f5f30105410 2024-02-14T15:04:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2cff9640 1 -- 172.21.15.17:0/706888671 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5f24005440 con 0x7f5f30105410 2024-02-14T15:04:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 Processor -- start 2024-02-14T15:04:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 -- start start 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58db9720 0x55ab58db9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dc26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58dc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58d279e0 con 0x55ab58db9720 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58d24f10 con 0x55ab58dc2c20 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58d206f0 con 0x55ab58dba040 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58db9720 0x55ab58db9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58db9720 0x55ab58db9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55790/0 (socket says 172.21.15.17:55790) 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3636430541 learned_addr learned my addr 172.21.15.17:0/3636430541 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dc26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2effd640 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58dc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2cff9640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5f180a2220 0x7f5f180a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3636430541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 msgr2=0x55ab58dc26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dc26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3636430541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 msgr2=0x55ab58dc5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58dc5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3636430541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab58a31920 con 0x55ab58db9720 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2cff9640 1 -- 172.21.15.17:0/706888671 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f5f240f0410 con 0x7f5f30105410 2024-02-14T15:04:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.249+0000 7f5f2effd640 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58dc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.250+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5f180a2220 0x7f5f180a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.250+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58db9720 0x55ab58db9b00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5f28010440 tx=0x7f5f28010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d9aef255751f43 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.250+0000 7f5f15ffb640 1 -- 172.21.15.17:0/3636430541 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28022070 con 0x55ab58db9720 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.250+0000 7f5f15ffb640 1 -- 172.21.15.17:0/3636430541 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f280134c0 con 0x55ab58db9720 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.250+0000 7f5f15ffb640 1 -- 172.21.15.17:0/3636430541 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28011640 con 0x55ab58db9720 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3636430541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58db9720 msgr2=0x55ab58db9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5f180a2220 0x7f5f180a46e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f28021000 tx=0x7f5f2801e000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58db9720 0x55ab58db9b00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5f28010440 tx=0x7f5f28010900 comp rx=0 tx=0).stop 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3636430541 shutdown_connections 2024-02-14T15:04:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58dc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dc26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3636430541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58db9720 0x55ab58db9b00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3636430541 >> 172.21.15.17:0/3636430541 conn(0x55ab58db3e50 msgr2=0x55ab58db4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3636430541 shutdown_connections 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3636430541 wait complete. 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 -- start start 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.251+0000 7f5f38dbd740 1 Processor -- start 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dbf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dc2c20 0x55ab58dbffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58cbcfa0 con 0x55ab58dc2c20 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab589f4250 con 0x55ab58db9720 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58ca6200 con 0x55ab58dba040 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 10 client.0 _ll_register_callbacks cb 0x55ab58d15b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dc2c20 0x55ab58dbffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dc2c20 0x55ab58dbffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55792/0 (socket says 172.21.15.17:55792) 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3183599053 learned_addr learned my addr 172.21.15.17:0/3183599053 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2effd640 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dbf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3183599053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 msgr2=0x55ab58dbf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dbf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3183599053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 msgr2=0x55ab58dc2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 -- 172.21.15.17:0/3183599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f28009c20 con 0x55ab58dc2c20 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dc2c20 0x55ab58dbffa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f80 tx=0x7f5f28009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1722ac236e1c4625 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f0effd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f0effd640 1 -- 172.21.15.17:0/3183599053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28022040 con 0x55ab58dc2c20 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f0effd640 1 -- 172.21.15.17:0/3183599053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f5f28019030 con 0x55ab58dc2c20 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab58a20600 con 0x55ab58dc2c20 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f0effd640 1 -- 172.21.15.17:0/3183599053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f28011da0 con 0x55ab58dc2c20 2024-02-14T15:04:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 10 client.16227 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.252+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab58d1ed70 con 0x55ab58dc2c20 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f0effd640 1 -- 172.21.15.17:0/3183599053 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ab58a20600 con 0x55ab58dc2c20 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f0effd640 1 -- 172.21.15.17:0/3183599053 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5f28005070 con 0x55ab58dc2c20 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f38dbd740 10 client.16227 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f38dbd740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f38dbd740 20 client.16227 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58dd2890 0x55ab58dd4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f38dbd740 4 client.16227 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab58a711b0 con 0x55ab58dd2890 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58dd2890 0x55ab58dd4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58dd2890 0x55ab58dd4c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee6bb9220b1b564 server_cookie=52a06dcb7a903aad in_seq=0 out_seq=0 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.253+0000 7f5f0effd640 10 client.16227 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f0effd640 1 -- 172.21.15.17:0/3183599053 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab58a711b0 con 0x55ab58dd2890 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f0effd640 10 client.16227 handle_command_reply: tid=0 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 client.16227 shutdown 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dc2c20 msgr2=0x55ab58dbffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dc2c20 0x55ab58dbffa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f80 tx=0x7f5f28009ab0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 shutdown_connections 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58dd2890 0x55ab58dd4c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dc2c20 0x55ab58dbffa0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dba040 0x55ab58dbf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/3183599053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 >> 172.21.15.17:0/3183599053 conn(0x55ab58db3e50 msgr2=0x55ab58db5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f0effd640 0 client.16227 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 shutdown_connections 2024-02-14T15:04:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 1 -- 172.21.15.17:0/3183599053 wait complete. 2024-02-14T15:04:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.255+0000 7f5f38dbd740 20 client.16227 trim_cache size 0 max 16384 2024-02-14T15:04:59.269 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:58 smithi017 ceph-mon[32549]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-02-14T15:04:59.270 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:04:58 smithi017 ceph-mon[32549]: scrub summary: idle 2024-02-14T15:04:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 Processor -- start 2024-02-14T15:04:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 -- start start 2024-02-14T15:04:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58db9720 0x55ab58dc20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58db5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58db8ec0 con 0x55ab58dba040 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58daf090 con 0x55ab58dc2c20 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.271+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58dd2690 con 0x55ab58db9720 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58db9720 0x55ab58dc20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58db9720 0x55ab58dc20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:46096/0 (socket says 172.21.15.17:46096) 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2f7fe640 1 -- 172.21.15.17:0/1916909218 learned_addr learned my addr 172.21.15.17:0/1916909218 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58db5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2effd640 1 -- 172.21.15.17:0/1916909218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58db9720 msgr2=0x55ab58dc20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2effd640 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58db9720 0x55ab58dc20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2effd640 1 -- 172.21.15.17:0/1916909218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 msgr2=0x55ab58db5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2effd640 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58db5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2effd640 1 -- 172.21.15.17:0/1916909218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab58cbb820 con 0x55ab58dba040 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58db9720 0x55ab58dc20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f2effd640 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc2600 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5f240ed910 tx=0x7f5f2402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab3fa0830fab821 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f157fa640 1 -- 172.21.15.17:0/1916909218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240bc510 con 0x55ab58dba040 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f157fa640 1 -- 172.21.15.17:0/1916909218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f240bd2b0 con 0x55ab58dba040 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.272+0000 7f5f157fa640 1 -- 172.21.15.17:0/1916909218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240eda80 con 0x55ab58dba040 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 -- 172.21.15.17:0/1916909218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 msgr2=0x55ab58dc2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc2600 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5f240ed910 tx=0x7f5f2402bc30 comp rx=0 tx=0).stop 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 -- 172.21.15.17:0/1916909218 shutdown_connections 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc2600 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58dc2c20 0x55ab58db5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/1916909218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58db9720 0x55ab58dc20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 -- 172.21.15.17:0/1916909218 >> 172.21.15.17:0/1916909218 conn(0x55ab58db3e50 msgr2=0x55ab58db5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 -- 172.21.15.17:0/1916909218 shutdown_connections 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.273+0000 7f5f38dbd740 1 -- 172.21.15.17:0/1916909218 wait complete. 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 -- start start 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 Processor -- start 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dc2c20 0x55ab58dc8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58dc8bd0 con 0x55ab58dba040 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58dc8d40 con 0x55ab58db9720 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab58dc8eb0 con 0x55ab58dc2c20 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f38dbd740 10 client.0 _ll_register_callbacks cb 0x7f5f28009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55808/0 (socket says 172.21.15.17:55808) 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f2effd640 1 -- 172.21.15.17:0/4262841630 learned_addr learned my addr 172.21.15.17:0/4262841630 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.274+0000 7f5f2ffff640 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dc2c20 0x55ab58dc8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f2effd640 1 -- 172.21.15.17:0/4262841630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dc2c20 msgr2=0x55ab58dc8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f2effd640 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dc2c20 0x55ab58dc8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f2effd640 1 -- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 msgr2=0x55ab58dc9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f2effd640 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f2effd640 1 -- 172.21.15.17:0/4262841630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2001b040 con 0x55ab58dba040 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f2effd640 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc8150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005ec0 tx=0x7f5f240bc630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2cf030a3e6dfb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f0effd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f0effd640 1 -- 172.21.15.17:0/4262841630 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240bcbb0 con 0x55ab58dba040 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f0effd640 1 -- 172.21.15.17:0/4262841630 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f5f240d5570 con 0x55ab58dba040 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f280134c0 con 0x55ab58dba040 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f0effd640 1 -- 172.21.15.17:0/4262841630 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f240b9220 con 0x55ab58dba040 2024-02-14T15:04:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 10 client.16233 fetch_fsmap learned FSMap version 24 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f5f301a3770 con 0x55ab58dba040 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f0effd640 1 -- 172.21.15.17:0/4262841630 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f280134c0 con 0x55ab58dba040 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f0effd640 1 -- 172.21.15.17:0/4262841630 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5f2403c070 con 0x55ab58dba040 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 10 client.16233 fetch_fsmap finished waiting for FSMap version 24 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 20 client.16233 populate_metadata read hostname 'smithi017' 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58e29f30 0x55ab58e2c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 4 client.16233 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.275+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab58a5d550 con 0x55ab58e29f30 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.276+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58e29f30 0x55ab58e2c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:04:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.276+0000 7f5f2f7fe640 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58e29f30 0x55ab58e2c350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d284d633d405ef76 server_cookie=60188879d4ef4a6f in_seq=0 out_seq=0 2024-02-14T15:04:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.276+0000 7f5f0effd640 10 client.16233 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.327+0000 7f5f0effd640 1 -- 172.21.15.17:0/4262841630 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab58a5d550 con 0x55ab58e29f30 2024-02-14T15:04:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.327+0000 7f5f0effd640 10 client.16233 handle_command_reply: tid=0 2024-02-14T15:04:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.327+0000 7f5f38dbd740 1 client.16233 shutdown 2024-02-14T15:04:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.327+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 msgr2=0x55ab58dc8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.327+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc8150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005ec0 tx=0x7f5f240bc630 comp rx=0 tx=0).stop 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 shutdown_connections 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ab58e29f30 0x55ab58e2c350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ab58dba040 0x55ab58dc8150 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab58dc2c20 0x55ab58dc8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f0effd640 0 client.16233 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 --2- 172.21.15.17:0/4262841630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab58db9720 0x55ab58dc9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 >> 172.21.15.17:0/4262841630 conn(0x55ab58db3e50 msgr2=0x55ab58dbadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 shutdown_connections 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 1 -- 172.21.15.17:0/4262841630 wait complete. 2024-02-14T15:04:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f38dbd740 20 client.16233 trim_cache size 0 max 16384 2024-02-14T15:04:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:04:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:04:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:04:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:04:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5f180a2220 msgr2=0x7f5f180a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5f180a2220 0x7f5f180a46e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f28021000 tx=0x7f5f2801e000 comp rx=0 tx=0).stop 2024-02-14T15:04:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30105410 msgr2=0x7f5f3019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:04:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30105410 0x7f5f3019c940 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009230 tx=0x7f5f24004c00 comp rx=0 tx=0).stop 2024-02-14T15:04:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 shutdown_connections 2024-02-14T15:04:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5f180a2220 0x7f5f180a46e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f30105410 0x7f5f3019c940 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f301965b0 0x7f5f30196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 --2- 172.21.15.17:0/706888671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f30100340 0x7f5f3019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:04:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.328+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 >> 172.21.15.17:0/706888671 conn(0x7f5f30077220 msgr2=0x7f5f300fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:04:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.329+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 shutdown_connections 2024-02-14T15:04:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:04:59.329+0000 7f5f36119640 1 -- 172.21.15.17:0/706888671 wait complete. 2024-02-14T15:04:59.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 2926 33.48 MB/sec warmup 1 sec latency 20.967 ms 2024-02-14T15:05:00.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 4587 20.74 MB/sec warmup 2 sec latency 33.233 ms 2024-02-14T15:05:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:00 smithi050 ceph-mon[36512]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 7 op/s 2024-02-14T15:05:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:00 smithi076 ceph-mon[37057]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 7 op/s 2024-02-14T15:05:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:00 smithi017 ceph-mon[32549]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 7 op/s 2024-02-14T15:05:01.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 7315 19.63 MB/sec warmup 3 sec latency 44.193 ms 2024-02-14T15:05:02.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 9879 17.55 MB/sec warmup 4 sec latency 7.456 ms 2024-02-14T15:05:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:02 smithi050 ceph-mon[36512]: pgmap v51: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 340 KiB/s wr, 91 op/s 2024-02-14T15:05:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:02 smithi076 ceph-mon[37057]: pgmap v51: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 340 KiB/s wr, 91 op/s 2024-02-14T15:05:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:02 smithi017 ceph-mon[32549]: pgmap v51: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 340 KiB/s wr, 91 op/s 2024-02-14T15:05:03.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 11930 16.10 MB/sec warmup 5 sec latency 16.575 ms 2024-02-14T15:05:04.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 13.90 MB/sec warmup 6 sec latency 982.817 ms 2024-02-14T15:05:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:04 smithi050 ceph-mon[36512]: pgmap v52: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 339 KiB/s wr, 88 op/s 2024-02-14T15:05:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:04 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:04 smithi076 ceph-mon[37057]: pgmap v52: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 339 KiB/s wr, 88 op/s 2024-02-14T15:05:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:04 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:04 smithi017 ceph-mon[32549]: pgmap v52: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 339 KiB/s wr, 88 op/s 2024-02-14T15:05:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:04 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:05.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 11.91 MB/sec warmup 7 sec latency 1982.923 ms 2024-02-14T15:05:06.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 10.42 MB/sec warmup 8 sec latency 2983.035 ms 2024-02-14T15:05:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:06 smithi050 ceph-mon[36512]: pgmap v53: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 339 KiB/s wr, 88 op/s 2024-02-14T15:05:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:06 smithi076 ceph-mon[37057]: pgmap v53: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 339 KiB/s wr, 88 op/s 2024-02-14T15:05:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:06 smithi017 ceph-mon[32549]: pgmap v53: 129 pgs: 129 active+clean; 4.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 339 KiB/s wr, 88 op/s 2024-02-14T15:05:07.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 9.26 MB/sec warmup 9 sec latency 3983.158 ms 2024-02-14T15:05:08.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 8.34 MB/sec warmup 10 sec latency 4983.249 ms 2024-02-14T15:05:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:08 smithi050 ceph-mon[36512]: pgmap v54: 129 pgs: 129 active+clean; 9.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 776 KiB/s wr, 142 op/s 2024-02-14T15:05:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:08 smithi076 ceph-mon[37057]: pgmap v54: 129 pgs: 129 active+clean; 9.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 776 KiB/s wr, 142 op/s 2024-02-14T15:05:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:08 smithi017 ceph-mon[32549]: pgmap v54: 129 pgs: 129 active+clean; 9.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 776 KiB/s wr, 142 op/s 2024-02-14T15:05:09.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 7.58 MB/sec warmup 11 sec latency 5983.368 ms 2024-02-14T15:05:10.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 6.95 MB/sec warmup 12 sec latency 6983.453 ms 2024-02-14T15:05:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:10 smithi050 ceph-mon[36512]: pgmap v55: 129 pgs: 129 active+clean; 9.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 775 KiB/s wr, 137 op/s 2024-02-14T15:05:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:10 smithi076 ceph-mon[37057]: pgmap v55: 129 pgs: 129 active+clean; 9.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 775 KiB/s wr, 137 op/s 2024-02-14T15:05:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:10 smithi017 ceph-mon[32549]: pgmap v55: 129 pgs: 129 active+clean; 9.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 775 KiB/s wr, 137 op/s 2024-02-14T15:05:11.859 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 6.41 MB/sec warmup 13 sec latency 7983.562 ms 2024-02-14T15:05:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 -- 172.21.15.17:0/3715179459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2006a220 msgr2=0x7f7d2010e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 --2- 172.21.15.17:0/3715179459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2006a220 0x7f7d2010e090 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009ca0 tx=0x7f7d1802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 -- 172.21.15.17:0/3715179459 shutdown_connections 2024-02-14T15:05:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 --2- 172.21.15.17:0/3715179459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2006a220 0x7f7d2010e090 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 --2- 172.21.15.17:0/3715179459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d20069860 0x7f7d20069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 --2- 172.21.15.17:0/3715179459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d20106200 0x7f7d201065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 -- 172.21.15.17:0/3715179459 >> 172.21.15.17:0/3715179459 conn(0x7f7d20076f80 msgr2=0x7f7d20077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 -- 172.21.15.17:0/3715179459 shutdown_connections 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.426+0000 7f7d2570c640 1 -- 172.21.15.17:0/3715179459 wait complete. 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 Processor -- start 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 -- start start 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d20069860 0x7f7d20197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d2006a220 0x7f7d20197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d20106200 0x7f7d2019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019c620 con 0x7f7d20106200 2024-02-14T15:05:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019c790 con 0x7f7d2006a220 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d2570c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2019c950 con 0x7f7d20069860 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d20106200 0x7f7d2019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d20106200 0x7f7d2019c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55278/0 (socket says 172.21.15.17:55278) 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1709363507 learned_addr learned my addr 172.21.15.17:0/1709363507 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.427+0000 7f7d1effd640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d20069860 0x7f7d20197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d17fff640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d2006a220 0x7f7d20197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1709363507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d20069860 msgr2=0x7f7d20197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d20069860 0x7f7d20197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1709363507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d2006a220 msgr2=0x7f7d20197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d2006a220 0x7f7d20197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1709363507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18009900 con 0x7f7d20106200 2024-02-14T15:05:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d20106200 0x7f7d2019c020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7d1802c700 tx=0x7f7d18035d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1cff9640 1 -- 172.21.15.17:0/1709363507 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18032ab0 con 0x7f7d20106200 2024-02-14T15:05:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1cff9640 1 -- 172.21.15.17:0/1709363507 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d18032ea0 con 0x7f7d20106200 2024-02-14T15:05:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2019cbd0 con 0x7f7d20106200 2024-02-14T15:05:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d1cff9640 1 -- 172.21.15.17:0/1709363507 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18004880 con 0x7f7d20106200 2024-02-14T15:05:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.428+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2019cea0 con 0x7f7d20106200 2024-02-14T15:05:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.430+0000 7f7d1cff9640 1 -- 172.21.15.17:0/1709363507 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7d1803b040 con 0x7f7d20106200 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d1cff9640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7d0c0a2220 0x7f7d0c0a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d283b9740 1 Processor -- start 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d283b9740 1 -- start start 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d1cff9640 1 -- 172.21.15.17:0/1709363507 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f7d180f0db0 con 0x7f7d20106200 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d1effd640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7d0c0a2220 0x7f7d0c0a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158582cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582d040 0x5615858356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561585835c20 0x561585838010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d283b9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158579a9e0 con 0x561585835c20 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.431+0000 7f7d283b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561585797f10 con 0x56158582c720 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d283b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615857936f0 con 0x56158582d040 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561585835c20 0x561585838010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561585835c20 0x561585838010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55282/0 (socket says 172.21.15.17:55282) 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1233677925 learned_addr learned my addr 172.21.15.17:0/1233677925 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1effd640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7d0c0a2220 0x7f7d0c0a46e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005fd0 tx=0x7f7d1000b390 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d17fff640 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582d040 0x5615858356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1effd640 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158582cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1233677925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582d040 msgr2=0x5615858356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582d040 0x5615858356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1233677925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 msgr2=0x56158582cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158582cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 -- 172.21.15.17:0/1233677925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615854a4920 con 0x561585835c20 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561585835c20 0x561585838010 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b9f20 tx=0x7f7d180bd450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eeb852215369bd9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d157fa640 1 -- 172.21.15.17:0/1233677925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180bd550 con 0x561585835c20 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d157fa640 1 -- 172.21.15.17:0/1233677925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d180c16a0 con 0x561585835c20 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.432+0000 7f7d157fa640 1 -- 172.21.15.17:0/1233677925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180b9bc0 con 0x561585835c20 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 -- 172.21.15.17:0/1233677925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561585835c20 msgr2=0x561585838010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561585835c20 0x561585838010 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b9f20 tx=0x7f7d180bd450 comp rx=0 tx=0).stop 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 -- 172.21.15.17:0/1233677925 shutdown_connections 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561585835c20 0x561585838010 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582d040 0x5615858356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1233677925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158582cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 -- 172.21.15.17:0/1233677925 >> 172.21.15.17:0/1233677925 conn(0x561585826e50 msgr2=0x561585827230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 -- 172.21.15.17:0/1233677925 shutdown_connections 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.433+0000 7f7d283b9740 1 -- 172.21.15.17:0/1233677925 wait complete. 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 -- start start 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 Processor -- start 2024-02-14T15:05:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582c720 0x5615858355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582d040 0x561585832930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x561585832eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158572ffa0 con 0x56158582c720 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561585467250 con 0x56158582d040 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561585719200 con 0x561585835c20 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d283b9740 10 client.0 _ll_register_callbacks cb 0x561585788b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582c720 0x5615858355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582c720 0x5615858355a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55290/0 (socket says 172.21.15.17:55290) 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 -- 172.21.15.17:0/3110569539 learned_addr learned my addr 172.21.15.17:0/3110569539 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x561585832eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d17fff640 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582d040 0x561585832930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 -- 172.21.15.17:0/3110569539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 msgr2=0x561585832eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x561585832eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 -- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582d040 msgr2=0x561585832930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582d040 0x561585832930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.434+0000 7f7d1effd640 1 -- 172.21.15.17:0/3110569539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2019cea0 con 0x56158582c720 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d1effd640 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582c720 0x5615858355a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7d100116a0 tx=0x7f7d10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab78e4b06dfb2999 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d067fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d067fc640 1 -- 172.21.15.17:0/3110569539 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1001e780 con 0x56158582c720 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d067fc640 1 -- 172.21.15.17:0/3110569539 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f7d10017070 con 0x56158582c720 2024-02-14T15:05:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561585493600 con 0x56158582c720 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d067fc640 1 -- 172.21.15.17:0/3110569539 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d1001e8e0 con 0x56158582c720 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 10 client.16245 fetch_fsmap learned FSMap version 25 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561585791d70 con 0x56158582c720 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d067fc640 1 -- 172.21.15.17:0/3110569539 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561585493600 con 0x56158582c720 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d067fc640 1 -- 172.21.15.17:0/3110569539 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7d10028c20 con 0x56158582c720 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 10 client.16245 fetch_fsmap finished waiting for FSMap version 25 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 20 client.16245 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561585845860 0x561585847c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 4 client.16245 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.435+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615854e41b0 con 0x561585845860 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.436+0000 7f7d17fff640 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561585845860 0x561585847c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.436+0000 7f7d17fff640 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561585845860 0x561585847c80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f09878159538af server_cookie=c9bf4c4a35594ad2 in_seq=0 out_seq=0 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.436+0000 7f7d067fc640 10 client.16245 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d067fc640 1 -- 172.21.15.17:0/3110569539 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615854e41b0 con 0x561585845860 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d067fc640 10 client.16245 handle_command_reply: tid=0 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 client.16245 shutdown 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582c720 msgr2=0x5615858355a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582c720 0x5615858355a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7d100116a0 tx=0x7f7d10004730 comp rx=0 tx=0).stop 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 shutdown_connections 2024-02-14T15:05:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561585845860 0x561585847c80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x561585832eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582d040 0x561585832930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d067fc640 0 client.16245 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3110569539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582c720 0x5615858355a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 >> 172.21.15.17:0/3110569539 conn(0x561585826e50 msgr2=0x561585828ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 shutdown_connections 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 1 -- 172.21.15.17:0/3110569539 wait complete. 2024-02-14T15:05:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.437+0000 7f7d283b9740 20 client.16245 trim_cache size 0 max 16384 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 Processor -- start 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 -- start start 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582c720 0x561585835060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x5615858355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561585835c20 0x5615858321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615858220f0 con 0x56158582d040 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158584ceb0 con 0x561585835c20 2024-02-14T15:05:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d283b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561585828010 con 0x56158582c720 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x5615858355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x5615858355a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55304/0 (socket says 172.21.15.17:55304) 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d17fff640 1 -- 172.21.15.17:0/3133898070 learned_addr learned my addr 172.21.15.17:0/3133898070 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582c720 0x561585835060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.451+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561585835c20 0x5615858321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d17fff640 1 -- 172.21.15.17:0/3133898070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582c720 msgr2=0x561585835060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d17fff640 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582c720 0x561585835060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d17fff640 1 -- 172.21.15.17:0/3133898070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561585835c20 msgr2=0x5615858321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d17fff640 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561585835c20 0x5615858321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d17fff640 1 -- 172.21.15.17:0/3133898070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56158572e820 con 0x56158582d040 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d1effd640 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582c720 0x561585835060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d17fff640 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x5615858355a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x561585832190 tx=0x7f7d0800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bf2b3c7464947 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d14ff9640 1 -- 172.21.15.17:0/3133898070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0801e070 con 0x56158582d040 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d14ff9640 1 -- 172.21.15.17:0/3133898070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d08017040 con 0x56158582d040 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.452+0000 7f7d14ff9640 1 -- 172.21.15.17:0/3133898070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08011a90 con 0x56158582d040 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- 172.21.15.17:0/3133898070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 msgr2=0x5615858355a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x5615858355a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x561585832190 tx=0x7f7d0800b390 comp rx=0 tx=0).stop 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- 172.21.15.17:0/3133898070 shutdown_connections 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x5615858355a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561585835c20 0x5615858321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 --2- 172.21.15.17:0/3133898070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56158582c720 0x561585835060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- 172.21.15.17:0/3133898070 >> 172.21.15.17:0/3133898070 conn(0x561585826e50 msgr2=0x561585831200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- 172.21.15.17:0/3133898070 shutdown_connections 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- 172.21.15.17:0/3133898070 wait complete. 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- start start 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 Processor -- start 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158583c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x56158583c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x56158583aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158583b020 con 0x56158582d040 2024-02-14T15:05:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158583b190 con 0x56158582c720 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158583b350 con 0x561585835c20 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.453+0000 7f7d283b9740 10 client.0 _ll_register_callbacks cb 0x7f7d18005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x56158583c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d1effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158583c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x56158583c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55308/0 (socket says 172.21.15.17:55308) 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 -- 172.21.15.17:0/1330378016 learned_addr learned my addr 172.21.15.17:0/1330378016 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x56158583aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 -- 172.21.15.17:0/1330378016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 msgr2=0x56158583aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x56158583aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 -- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 msgr2=0x56158583c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158583c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 -- 172.21.15.17:0/1330378016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561585791d70 con 0x56158582d040 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d1effd640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158583c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d1f7fe640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x56158583aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d17fff640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x56158583c9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7d08004650 tx=0x7f7d08004030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd947bcdc873119 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d067fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d067fc640 1 -- 172.21.15.17:0/1330378016 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08005030 con 0x56158582d040 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d067fc640 1 -- 172.21.15.17:0/1330378016 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f7d08005190 con 0x56158582d040 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d180c16a0 con 0x56158582d040 2024-02-14T15:05:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d067fc640 1 -- 172.21.15.17:0/1330378016 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d08005350 con 0x56158582d040 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d283b9740 10 client.16251 fetch_fsmap learned FSMap version 25 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.454+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7d0800b900 con 0x56158582d040 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d067fc640 1 -- 172.21.15.17:0/1330378016 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d180c16a0 con 0x56158582d040 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d067fc640 1 -- 172.21.15.17:0/1330378016 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7d08011990 con 0x56158582d040 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d283b9740 10 client.16251 fetch_fsmap finished waiting for FSMap version 25 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d283b9740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d283b9740 20 client.16251 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56158589ce80 0x56158589f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d283b9740 4 client.16251 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615854d0550 con 0x56158589ce80 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d1effd640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56158589ce80 0x56158589f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d1effd640 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56158589ce80 0x56158589f2a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41717f377ee97645 server_cookie=4f56bc1ed15795f1 in_seq=0 out_seq=0 2024-02-14T15:05:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:12.455+0000 7f7d067fc640 10 client.16251 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:12.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 5.95 MB/sec warmup 14 sec latency 8983.660 ms 2024-02-14T15:05:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:12 smithi050 ceph-mon[36512]: pgmap v56: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 794 KiB/s wr, 143 op/s 2024-02-14T15:05:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:12 smithi076 ceph-mon[37057]: pgmap v56: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 794 KiB/s wr, 143 op/s 2024-02-14T15:05:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:12 smithi017 ceph-mon[32549]: pgmap v56: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 794 KiB/s wr, 143 op/s 2024-02-14T15:05:13.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 5.56 MB/sec warmup 15 sec latency 9983.756 ms 2024-02-14T15:05:14.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 5.21 MB/sec warmup 16 sec latency 10983.853 ms 2024-02-14T15:05:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:14 smithi050 ceph-mon[36512]: pgmap v57: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 455 KiB/s wr, 59 op/s 2024-02-14T15:05:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:14 smithi076 ceph-mon[37057]: pgmap v57: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 455 KiB/s wr, 59 op/s 2024-02-14T15:05:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:14 smithi017 ceph-mon[32549]: pgmap v57: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 455 KiB/s wr, 59 op/s 2024-02-14T15:05:15.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 4.90 MB/sec warmup 17 sec latency 11983.912 ms 2024-02-14T15:05:16.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 4.63 MB/sec warmup 18 sec latency 12983.996 ms 2024-02-14T15:05:16.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:16 smithi050 ceph-mon[36512]: pgmap v58: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 455 KiB/s wr, 59 op/s 2024-02-14T15:05:16.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:16 smithi076 ceph-mon[37057]: pgmap v58: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 455 KiB/s wr, 59 op/s 2024-02-14T15:05:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:16 smithi017 ceph-mon[32549]: pgmap v58: 129 pgs: 129 active+clean; 8.9 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 455 KiB/s wr, 59 op/s 2024-02-14T15:05:17.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 12061 4.39 MB/sec warmup 19 sec latency 13984.125 ms 2024-02-14T15:05:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d067fc640 1 -- 172.21.15.17:0/1330378016 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615854d0550 con 0x56158589ce80 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d067fc640 10 client.16251 handle_command_reply: tid=0 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 client.16251 shutdown 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 msgr2=0x56158583c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x56158583c9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7d08004650 tx=0x7f7d08004030 comp rx=0 tx=0).stop 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 shutdown_connections 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56158589ce80 0x56158589f2a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56158582d040 0x56158583c9c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561585835c20 0x56158583aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d067fc640 0 client.16251 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 --2- 172.21.15.17:0/1330378016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56158582c720 0x56158583c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 >> 172.21.15.17:0/1330378016 conn(0x561585826e50 msgr2=0x5615858321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 shutdown_connections 2024-02-14T15:05:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 1 -- 172.21.15.17:0/1330378016 wait complete. 2024-02-14T15:05:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.200+0000 7f7d283b9740 20 client.16251 trim_cache size 0 max 16384 2024-02-14T15:05:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:05:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:05:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:05:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7d0c0a2220 msgr2=0x7f7d0c0a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7d0c0a2220 0x7f7d0c0a46e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005fd0 tx=0x7f7d1000b390 comp rx=0 tx=0).stop 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d20106200 msgr2=0x7f7d2019c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d20106200 0x7f7d2019c020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7d1802c700 tx=0x7f7d18035d50 comp rx=0 tx=0).stop 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 shutdown_connections 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7d0c0a2220 0x7f7d0c0a46e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d20106200 0x7f7d2019c020 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d2006a220 0x7f7d20197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 --2- 172.21.15.17:0/1709363507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d20069860 0x7f7d20197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 >> 172.21.15.17:0/1709363507 conn(0x7f7d20076f80 msgr2=0x7f7d200fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 shutdown_connections 2024-02-14T15:05:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:18.201+0000 7f7d2570c640 1 -- 172.21.15.17:0/1709363507 wait complete. 2024-02-14T15:05:18.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 14431 4.86 MB/sec warmup 20 sec latency 14317.085 ms 2024-02-14T15:05:18.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:18 smithi050 ceph-mon[36512]: pgmap v59: 129 pgs: 129 active+clean; 8.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 470 KiB/s wr, 105 op/s 2024-02-14T15:05:18.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:18 smithi076 ceph-mon[37057]: pgmap v59: 129 pgs: 129 active+clean; 8.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 470 KiB/s wr, 105 op/s 2024-02-14T15:05:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:18 smithi017 ceph-mon[32549]: pgmap v59: 129 pgs: 129 active+clean; 8.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 470 KiB/s wr, 105 op/s 2024-02-14T15:05:19.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 17118 5.24 MB/sec warmup 21 sec latency 6.659 ms 2024-02-14T15:05:20.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 20118 5.61 MB/sec warmup 22 sec latency 5.723 ms 2024-02-14T15:05:20.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:20 smithi050 ceph-mon[36512]: pgmap v60: 129 pgs: 129 active+clean; 8.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 34 KiB/s wr, 52 op/s 2024-02-14T15:05:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:20 smithi076 ceph-mon[37057]: pgmap v60: 129 pgs: 129 active+clean; 8.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 34 KiB/s wr, 52 op/s 2024-02-14T15:05:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:20 smithi017 ceph-mon[32549]: pgmap v60: 129 pgs: 129 active+clean; 8.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 34 KiB/s wr, 52 op/s 2024-02-14T15:05:21.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 22303 5.94 MB/sec warmup 23 sec latency 14.662 ms 2024-02-14T15:05:22.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 25325 6.45 MB/sec warmup 24 sec latency 15.568 ms 2024-02-14T15:05:22.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:22 smithi050 ceph-mon[36512]: pgmap v61: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 432 KiB/s wr, 101 op/s 2024-02-14T15:05:22.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:22 smithi076 ceph-mon[37057]: pgmap v61: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 432 KiB/s wr, 101 op/s 2024-02-14T15:05:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:22 smithi017 ceph-mon[32549]: pgmap v61: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 432 KiB/s wr, 101 op/s 2024-02-14T15:05:23.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 28644 6.95 MB/sec warmup 25 sec latency 6.249 ms 2024-02-14T15:05:24.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:05:24.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:05:24.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 -- 172.21.15.17:0/3262463886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84106510 msgr2=0x7f1f8410e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 --2- 172.21.15.17:0/3262463886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84106510 0x7f1f8410e0e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009cd0 tx=0x7f1f7c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 -- 172.21.15.17:0/3262463886 shutdown_connections 2024-02-14T15:05:24.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 --2- 172.21.15.17:0/3262463886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84106510 0x7f1f8410e0e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 --2- 172.21.15.17:0/3262463886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f8406a620 0x7f1f84105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 --2- 172.21.15.17:0/3262463886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f84069c70 0x7f1f8406a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 -- 172.21.15.17:0/3262463886 >> 172.21.15.17:0/3262463886 conn(0x7f1f840772b0 msgr2=0x7f1f840776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 -- 172.21.15.17:0/3262463886 shutdown_connections 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.423+0000 7f1f8af25640 1 -- 172.21.15.17:0/3262463886 wait complete. 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 Processor -- start 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 -- start start 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84069c70 0x7f1f8419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f84106510 0x7f1f8419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f841976a0 0x7f1f84197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84198170 con 0x7f1f84069c70 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841982e0 con 0x7f1f841976a0 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f8af25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841984a0 con 0x7f1f84106510 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84069c70 0x7f1f8419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84069c70 0x7f1f8419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49254/0 (socket says 172.21.15.17:49254) 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 -- 172.21.15.17:0/2570662964 learned_addr learned my addr 172.21.15.17:0/2570662964 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f83fff640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f84106510 0x7f1f8419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f894c2640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f841976a0 0x7f1f84197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 -- 172.21.15.17:0/2570662964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f84106510 msgr2=0x7f1f8419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f84106510 0x7f1f8419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 -- 172.21.15.17:0/2570662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f841976a0 msgr2=0x7f1f84197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f841976a0 0x7f1f84197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.424+0000 7f1f88cc1640 1 -- 172.21.15.17:0/2570662964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009490 con 0x7f1f84069c70 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.425+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84069c70 0x7f1f8419d590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e9f0 tx=0x7f1f7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.425+0000 7f1f81ffb640 1 -- 172.21.15.17:0/2570662964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78009720 con 0x7f1f84069c70 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.425+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c009900 con 0x7f1f84069c70 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.425+0000 7f1f81ffb640 1 -- 172.21.15.17:0/2570662964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f7800cd80 con 0x7f1f84069c70 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.425+0000 7f1f81ffb640 1 -- 172.21.15.17:0/2570662964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78010690 con 0x7f1f84069c70 2024-02-14T15:05:24.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.425+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f841a4740 con 0x7f1f84069c70 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.427+0000 7f1f81ffb640 1 -- 172.21.15.17:0/2570662964 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1f780046c0 con 0x7f1f84069c70 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 Processor -- start 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 -- start start 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f81ffb640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1f740a2230 0x7f1f740a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a64aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a6535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f81ffb640 1 -- 172.21.15.17:0/2570662964 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f1f78014070 con 0x7f1f84069c70 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 0x555f8a655f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a6055b0 con 0x555f8a653b30 2024-02-14T15:05:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a5b92b0 con 0x555f8a64af50 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a5b67e0 con 0x555f8a64a630 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f83fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a6535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f83fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a6535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49380/0 (socket says 172.21.15.17:49380) 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.428+0000 7f1f83fff640 1 -- 172.21.15.17:0/3670732293 learned_addr learned my addr 172.21.15.17:0/3670732293 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a64aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f894c2640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1f740a2230 0x7f1f740a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f83fff640 1 -- 172.21.15.17:0/3670732293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 msgr2=0x555f8a64aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f83fff640 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a64aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f83fff640 1 -- 172.21.15.17:0/3670732293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 msgr2=0x555f8a655f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f83fff640 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 0x555f8a655f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f83fff640 1 -- 172.21.15.17:0/3670732293 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8a574020 con 0x555f8a64af50 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f83fff640 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a6535f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009db0 tx=0x7f1f70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ec1f31943b17d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f894c2640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1f740a2230 0x7f1f740a46f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005cc0 tx=0x7f1f7c005c50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f6effd640 1 -- 172.21.15.17:0/3670732293 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70019410 con 0x555f8a64af50 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f6effd640 1 -- 172.21.15.17:0/3670732293 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f70019a50 con 0x555f8a64af50 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.429+0000 7f1f6effd640 1 -- 172.21.15.17:0/3670732293 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70018a90 con 0x555f8a64af50 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/3670732293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 msgr2=0x555f8a6535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a6535f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009db0 tx=0x7f1f70011810 comp rx=0 tx=0).stop 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/3670732293 shutdown_connections 2024-02-14T15:05:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 0x555f8a655f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a6535f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/3670732293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a64aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/3670732293 >> 172.21.15.17:0/3670732293 conn(0x555f8a644d60 msgr2=0x555f8a645140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/3670732293 shutdown_connections 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.430+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/3670732293 wait complete. 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 -- start start 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 Processor -- start 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a650b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64af50 0x555f8a6510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a653b30 0x555f8a651610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a5b1fc0 con 0x555f8a64af50 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a285430 con 0x555f8a653b30 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a241e00 con 0x555f8a64a630 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f8dbd1740 10 client.0 _ll_register_callbacks cb 0x555f8a533870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64af50 0x555f8a6510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a650b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a650b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38734/0 (socket says 172.21.15.17:38734) 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f88cc1640 1 -- 172.21.15.17:0/4030213162 learned_addr learned my addr 172.21.15.17:0/4030213162 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64af50 0x555f8a6510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49262/0 (socket says 172.21.15.17:49262) 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.431+0000 7f1f894c2640 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a653b30 0x555f8a651610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f88cc1640 1 -- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a653b30 msgr2=0x555f8a651610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a653b30 0x555f8a651610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f88cc1640 1 -- 172.21.15.17:0/4030213162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64af50 msgr2=0x555f8a6510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64af50 0x555f8a6510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f88cc1640 1 -- 172.21.15.17:0/4030213162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f700099e0 con 0x555f8a64a630 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a650b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f780977e0 tx=0x7f1f7809ae10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1a82ccf09c8afd server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f63fff640 1 -- 172.21.15.17:0/4030213162 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780a3120 con 0x555f8a64a630 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f63fff640 1 -- 172.21.15.17:0/4030213162 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f1f780a3280 con 0x555f8a64a630 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f8a2b17e0 con 0x555f8a64a630 2024-02-14T15:05:24.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f63fff640 1 -- 172.21.15.17:0/4030213162 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f780b9250 con 0x555f8a64a630 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f8dbd1740 10 client.25792 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555f8a5b0640 con 0x555f8a64a630 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.432+0000 7f1f63fff640 1 -- 172.21.15.17:0/4030213162 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555f8a2b17e0 con 0x555f8a64a630 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f63fff640 1 -- 172.21.15.17:0/4030213162 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1f7801f060 con 0x555f8a64a630 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f8dbd1740 10 client.25792 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f8dbd1740 10 client.25792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f8dbd1740 20 client.25792 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a663e60 0x555f8a666250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f8dbd1740 4 client.25792 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f8a2ee720 con 0x555f8a663e60 2024-02-14T15:05:24.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f83fff640 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a663e60 0x555f8a666250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f63fff640 10 client.25792 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.433+0000 7f1f83fff640 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a663e60 0x555f8a666250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace73d8b4f58eb5f server_cookie=c3e98c90a473ecb7 in_seq=0 out_seq=0 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f63fff640 1 -- 172.21.15.17:0/4030213162 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f8a2ee720 con 0x555f8a663e60 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f63fff640 10 client.25792 handle_command_reply: tid=0 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 client.25792 shutdown 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 msgr2=0x555f8a650b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a650b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f780977e0 tx=0x7f1f7809ae10 comp rx=0 tx=0).stop 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 shutdown_connections 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a663e60 0x555f8a666250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64af50 0x555f8a6510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a653b30 0x555f8a651610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f63fff640 0 client.25792 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:24.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/4030213162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a650b90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 >> 172.21.15.17:0/4030213162 conn(0x555f8a644d60 msgr2=0x555f8a646ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 shutdown_connections 2024-02-14T15:05:24.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/4030213162 wait complete. 2024-02-14T15:05:24.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.434+0000 7f1f8dbd1740 20 client.25792 trim_cache size 0 max 16384 2024-02-14T15:05:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 Processor -- start 2024-02-14T15:05:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 -- start start 2024-02-14T15:05:24.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64a630 0x555f8a643ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a651430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a653b30 0x555f8a65a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a536fd0 con 0x555f8a64a630 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a660a70 con 0x555f8a64af50 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a6627b0 con 0x555f8a653b30 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64a630 0x555f8a643ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f894c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a653b30 0x555f8a65a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64a630 0x555f8a643ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49270/0 (socket says 172.21.15.17:49270) 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f83fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a651430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.450+0000 7f1f88cc1640 1 -- 172.21.15.17:0/1022234656 learned_addr learned my addr 172.21.15.17:0/1022234656 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f88cc1640 1 -- 172.21.15.17:0/1022234656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a653b30 msgr2=0x555f8a65a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a653b30 0x555f8a65a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f88cc1640 1 -- 172.21.15.17:0/1022234656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 msgr2=0x555f8a651430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a651430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f88cc1640 1 -- 172.21.15.17:0/1022234656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8a54cb30 con 0x555f8a64a630 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f894c2640 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a653b30 0x555f8a65a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f83fff640 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a651430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64a630 0x555f8a643ce0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1f7809ab10 tx=0x7f1f780971c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d202535b655752db server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f6e7fc640 1 -- 172.21.15.17:0/1022234656 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7809a770 con 0x555f8a64a630 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f6e7fc640 1 -- 172.21.15.17:0/1022234656 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f7809d040 con 0x555f8a64a630 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.451+0000 7f1f6e7fc640 1 -- 172.21.15.17:0/1022234656 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780ce5a0 con 0x555f8a64a630 2024-02-14T15:05:24.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/1022234656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64a630 msgr2=0x555f8a643ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64a630 0x555f8a643ce0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1f7809ab10 tx=0x7f1f780971c0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/1022234656 shutdown_connections 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a653b30 0x555f8a65a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a651430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/1022234656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a64a630 0x555f8a643ce0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/1022234656 >> 172.21.15.17:0/1022234656 conn(0x555f8a644d60 msgr2=0x555f8a6435d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/1022234656 shutdown_connections 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/1022234656 wait complete. 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 -- start start 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.452+0000 7f1f8dbd1740 1 Processor -- start 2024-02-14T15:05:24.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a659d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a65a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f8dbd1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 0x555f8a6b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a6b48e0 con 0x555f8a653b30 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a6b4a50 con 0x555f8a64af50 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f8dbd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a6b4c10 con 0x555f8a64a630 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f8dbd1740 10 client.0 _ll_register_callbacks cb 0x7f1f70009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a659d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f83fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a65a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a659d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38748/0 (socket says 172.21.15.17:38748) 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 -- 172.21.15.17:0/583533850 learned_addr learned my addr 172.21.15.17:0/583533850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f83fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a65a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49416/0 (socket says 172.21.15.17:49416) 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f894c2640 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 0x555f8a6b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 -- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 msgr2=0x555f8a65a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a65a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 -- 172.21.15.17:0/583533850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 msgr2=0x555f8a6b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 0x555f8a6b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 -- 172.21.15.17:0/583533850 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f8a574020 con 0x555f8a64a630 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f83fff640 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a65a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f88cc1640 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a659d20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f78097270 tx=0x7f1f7809a5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d1011addfb2eed3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.453+0000 7f1f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f63fff640 1 -- 172.21.15.17:0/583533850 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7809a770 con 0x555f8a64a630 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f63fff640 1 -- 172.21.15.17:0/583533850 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f1f7809ef60 con 0x555f8a64a630 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f70019a50 con 0x555f8a64a630 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f63fff640 1 -- 172.21.15.17:0/583533850 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f780b3c70 con 0x555f8a64a630 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 10 client.25798 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f1f8410e8f0 con 0x555f8a64a630 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f63fff640 1 -- 172.21.15.17:0/583533850 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f70019a50 con 0x555f8a64a630 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f63fff640 1 -- 172.21.15.17:0/583533850 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1f7801f070 con 0x555f8a64a630 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 10 client.25798 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:24.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 20 client.25798 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a6bc010 0x555f8a6be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 4 client.25798 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f8a33d890 con 0x555f8a6bc010 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.454+0000 7f1f83fff640 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a6bc010 0x555f8a6be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f63fff640 10 client.25798 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f83fff640 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a6bc010 0x555f8a6be400 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7cd13b8f2951fe server_cookie=5e53669cf19f67bc in_seq=0 out_seq=0 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f63fff640 1 -- 172.21.15.17:0/583533850 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555f8a33d890 con 0x555f8a6bc010 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f63fff640 10 client.25798 handle_command_reply: tid=0 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 client.25798 shutdown 2024-02-14T15:05:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 msgr2=0x555f8a659d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a659d20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f78097270 tx=0x7f1f7809a5e0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 shutdown_connections 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555f8a6bc010 0x555f8a6be400 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f8a653b30 0x555f8a6b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555f8a64af50 0x555f8a65a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 --2- 172.21.15.17:0/583533850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f8a64a630 0x555f8a659d20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f63fff640 0 client.25798 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 >> 172.21.15.17:0/583533850 conn(0x555f8a644d60 msgr2=0x555f8a643e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 shutdown_connections 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 1 -- 172.21.15.17:0/583533850 wait complete. 2024-02-14T15:05:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.455+0000 7f1f8dbd1740 20 client.25798 trim_cache size 0 max 16384 2024-02-14T15:05:24.466 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:05:24.466 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "no active scrubs running", 2024-02-14T15:05:24.466 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": {} 2024-02-14T15:05:24.466 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1f740a2230 msgr2=0x7f1f740a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1f740a2230 0x7f1f740a46f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005cc0 tx=0x7f1f7c005c50 comp rx=0 tx=0).stop 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84069c70 msgr2=0x7f1f8419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84069c70 0x7f1f8419d590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e9f0 tx=0x7f1f7800edf0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 shutdown_connections 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1f740a2230 0x7f1f740a46f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f841976a0 0x7f1f84197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f84106510 0x7f1f8419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 --2- 172.21.15.17:0/2570662964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f84069c70 0x7f1f8419d590 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 >> 172.21.15.17:0/2570662964 conn(0x7f1f840772b0 msgr2=0x7f1f840fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 shutdown_connections 2024-02-14T15:05:24.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.456+0000 7f1f8af25640 1 -- 172.21.15.17:0/2570662964 wait complete. 2024-02-14T15:05:24.469 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-14T15:05:24.469 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 -- 172.21.15.17:0/3919174850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 msgr2=0x7f9d3c10b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3919174850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 0x7f9d3c10b250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d2402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 -- 172.21.15.17:0/3919174850 shutdown_connections 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3919174850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 0x7f9d3c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3919174850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 0x7f9d3c10b250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3919174850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 0x7f9d3c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 -- 172.21.15.17:0/3919174850 >> 172.21.15.17:0/3919174850 conn(0x7f9d3c0772b0 msgr2=0x7f9d3c0776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 -- 172.21.15.17:0/3919174850 shutdown_connections 2024-02-14T15:05:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.582+0000 7f9d411eb640 1 -- 172.21.15.17:0/3919174850 wait complete. 2024-02-14T15:05:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 Processor -- start 2024-02-14T15:05:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 -- start start 2024-02-14T15:05:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 0x7f9d3c195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 0x7f9d3c195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 0x7f9d3c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c19a520 con 0x7f9d3c10b820 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c19a690 con 0x7f9d3c102d70 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d411eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c19a850 con 0x7f9d3c102450 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.583+0000 7f9d3b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 0x7f9d3c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 0x7f9d3c195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 0x7f9d3c195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38762/0 (socket says 172.21.15.17:38762) 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 0x7f9d3c199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49286/0 (socket says 172.21.15.17:49286) 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3ad9d640 1 -- 172.21.15.17:0/3638796183 learned_addr learned my addr 172.21.15.17:0/3638796183 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3259c640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 0x7f9d3c195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3b59e640 1 -- 172.21.15.17:0/3638796183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 msgr2=0x7f9d3c195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3b59e640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 0x7f9d3c195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3b59e640 1 -- 172.21.15.17:0/3638796183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 msgr2=0x7f9d3c195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3b59e640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 0x7f9d3c195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3b59e640 1 -- 172.21.15.17:0/3638796183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24009560 con 0x7f9d3c10b820 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3ad9d640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 0x7f9d3c195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d3b59e640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 0x7f9d3c199f20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d34012f90 tx=0x7f9d34010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d38d99640 1 -- 172.21.15.17:0/3638796183 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3401c070 con 0x7f9d3c10b820 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d38d99640 1 -- 172.21.15.17:0/3638796183 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d34013040 con 0x7f9d3c10b820 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c19aad0 con 0x7f9d3c10b820 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d38d99640 1 -- 172.21.15.17:0/3638796183 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3400ba70 con 0x7f9d3c10b820 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.584+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c19ad80 con 0x7f9d3c10b820 2024-02-14T15:05:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.585+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3c0680b0 con 0x7f9d3c10b820 2024-02-14T15:05:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.587+0000 7f9d38d99640 1 -- 172.21.15.17:0/3638796183 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9d3400bc30 con 0x7f9d3c10b820 2024-02-14T15:05:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.587+0000 7f9d38d99640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d280a2230 0x7f9d280a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.587+0000 7f9d3ad9d640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d280a2230 0x7f9d280a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.587+0000 7f9d38d99640 1 -- 172.21.15.17:0/3638796183 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f9d340d2760 con 0x7f9d3c10b820 2024-02-14T15:05:24.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.588+0000 7f9d3ad9d640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d280a2230 0x7f9d280a46f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c011500 tx=0x7f9d2c005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.591+0000 7f9d38d99640 1 -- 172.21.15.17:0/3638796183 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d3409b090 con 0x7f9d3c10b820 2024-02-14T15:05:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.832+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d3c1969b0 con 0x7f9d3c10b820 2024-02-14T15:05:24.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.833+0000 7f9d38d99640 1 -- 172.21.15.17:0/3638796183 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11320 (secure 0 0 0) 0x7f9d3409e750 con 0x7f9d3c10b820 2024-02-14T15:05:24.834 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:05:24.834 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:05:23.165200+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:05:24.835 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 26 2024-02-14T15:05:24.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d280a2230 msgr2=0x7f9d280a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d280a2230 0x7f9d280a46f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c011500 tx=0x7f9d2c005f90 comp rx=0 tx=0).stop 2024-02-14T15:05:24.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 msgr2=0x7f9d3c199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 0x7f9d3c199f20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d34012f90 tx=0x7f9d34010490 comp rx=0 tx=0).stop 2024-02-14T15:05:24.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 shutdown_connections 2024-02-14T15:05:24.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d280a2230 0x7f9d280a46f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d3c10b820 0x7f9d3c199f20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c102d70 0x7f9d3c195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 --2- 172.21.15.17:0/3638796183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d3c102450 0x7f9d3c195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.835+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 >> 172.21.15.17:0/3638796183 conn(0x7f9d3c0772b0 msgr2=0x7f9d3c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.836+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 shutdown_connections 2024-02-14T15:05:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.836+0000 7f9d411eb640 1 -- 172.21.15.17:0/3638796183 wait complete. 2024-02-14T15:05:24.849 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:05:24.849 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-14T15:05:24.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 31774 7.25 MB/sec warmup 26 sec latency 6.108 ms 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/1067051881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd00fffd0 msgr2=0x7f9fd01003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/1067051881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd00fffd0 0x7f9fd01003b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009900 tx=0x7f9fb802bcd0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/1067051881 shutdown_connections 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/1067051881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fd010db80 0x7f9fd010ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/1067051881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fd01008f0 0x7f9fd010d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/1067051881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd00fffd0 0x7f9fd01003b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/1067051881 >> 172.21.15.17:0/1067051881 conn(0x7f9fd00772e0 msgr2=0x7f9fd00776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/1067051881 shutdown_connections 2024-02-14T15:05:24.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/1067051881 wait complete. 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.959+0000 7f9fd6c9f640 1 Processor -- start 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd6c9f640 1 -- start start 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd6c9f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fd00fffd0 0x7f9fd019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd6c9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd01008f0 0x7f9fd019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd6c9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fd010db80 0x7f9fd01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd6c9f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0199ff0 con 0x7f9fd01008f0 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd6c9f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019a160 con 0x7f9fd010db80 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd6c9f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019a320 con 0x7f9fd00fffd0 2024-02-14T15:05:24.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd01008f0 0x7f9fd019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd4a3b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fd00fffd0 0x7f9fd019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd01008f0 0x7f9fd019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49300/0 (socket says 172.21.15.17:49300) 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 -- 172.21.15.17:0/3883057001 learned_addr learned my addr 172.21.15.17:0/3883057001 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd4a3b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fd00fffd0 0x7f9fd019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38772/0 (socket says 172.21.15.17:38772) 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fd523c640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fd010db80 0x7f9fd01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 -- 172.21.15.17:0/3883057001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fd00fffd0 msgr2=0x7f9fd019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fd00fffd0 0x7f9fd019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 -- 172.21.15.17:0/3883057001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fd010db80 msgr2=0x7f9fd01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fd010db80 0x7f9fd01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 -- 172.21.15.17:0/3883057001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb8009560 con 0x7f9fd01008f0 2024-02-14T15:05:24.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.960+0000 7f9fcffff640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd01008f0 0x7f9fd019fd30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9f0 tx=0x7f9fc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.961+0000 7f9fcdffb640 1 -- 172.21.15.17:0/3883057001 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0009700 con 0x7f9fd01008f0 2024-02-14T15:05:24.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.961+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd019a600 con 0x7f9fd01008f0 2024-02-14T15:05:24.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.961+0000 7f9fcdffb640 1 -- 172.21.15.17:0/3883057001 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc000cd80 con 0x7f9fd01008f0 2024-02-14T15:05:24.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.961+0000 7f9fcdffb640 1 -- 172.21.15.17:0/3883057001 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0018920 con 0x7f9fd01008f0 2024-02-14T15:05:24.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.961+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd01a6890 con 0x7f9fd01008f0 2024-02-14T15:05:24.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.963+0000 7f9fcdffb640 1 -- 172.21.15.17:0/3883057001 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9fc0018a80 con 0x7f9fd01008f0 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 Processor -- start 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 -- start start 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd49298c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd49299210 0x55bd4929b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd491eec80 con 0x55bd49290690 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd491fe320 con 0x55bd49299210 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd9861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd491fb850 con 0x55bd4928fce0 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fcdffb640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9fbc0a2230 0x7f9fbc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd4a3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd49298c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd4a3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd49298c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49306/0 (socket says 172.21.15.17:49306) 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd4a3b640 1 -- 172.21.15.17:0/1276280605 learned_addr learned my addr 172.21.15.17:0/1276280605 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd523c640 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd523c640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9fbc0a2230 0x7f9fbc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fcffff640 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd49299210 0x55bd4929b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd523c640 1 -- 172.21.15.17:0/1276280605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd49299210 msgr2=0x55bd4929b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fcdffb640 1 -- 172.21.15.17:0/3883057001 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f9fc0014070 con 0x7f9fd01008f0 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd523c640 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd49299210 0x55bd4929b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd523c640 1 -- 172.21.15.17:0/1276280605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 msgr2=0x55bd49298c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd523c640 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd49298c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd523c640 1 -- 172.21.15.17:0/1276280605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd491bde30 con 0x55bd4928fce0 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.964+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd49298c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.965+0000 7f9fd523c640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9fbc0a2230 0x7f9fbc0a46f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005fd0 tx=0x7f9fc801e000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.965+0000 7f9fd523c640 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492900c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9fd01021b0 tx=0x7f9fc800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db4b6526f8b483f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.965+0000 7f9fc6ffd640 1 -- 172.21.15.17:0/1276280605 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80114d0 con 0x55bd4928fce0 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.965+0000 7f9fc6ffd640 1 -- 172.21.15.17:0/1276280605 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8009cf0 con 0x55bd4928fce0 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.965+0000 7f9fc6ffd640 1 -- 172.21.15.17:0/1276280605 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8005690 con 0x55bd4928fce0 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- 172.21.15.17:0/1276280605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 msgr2=0x55bd492900c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492900c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9fd01021b0 tx=0x7f9fc800f790 comp rx=0 tx=0).stop 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- 172.21.15.17:0/1276280605 shutdown_connections 2024-02-14T15:05:24.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd49298c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd49299210 0x55bd4929b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1276280605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492900c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- 172.21.15.17:0/1276280605 >> 172.21.15.17:0/1276280605 conn(0x55bd4928a3b0 msgr2=0x55bd4928a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- 172.21.15.17:0/1276280605 shutdown_connections 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- 172.21.15.17:0/1276280605 wait complete. 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- start start 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 Processor -- start 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928fce0 0x55bd49296140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd49290690 0x55bd49296680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49299210 0x55bd49296bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd491f7030 con 0x55bd49299210 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd48eca430 con 0x55bd4928fce0 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd48e86e00 con 0x55bd49290690 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.966+0000 7f9fd9861740 10 client.0 _ll_register_callbacks cb 0x55bd48f07be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49299210 0x55bd49296bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd49290690 0x55bd49296680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49299210 0x55bd49296bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49310/0 (socket says 172.21.15.17:49310) 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 -- 172.21.15.17:0/469579525 learned_addr learned my addr 172.21.15.17:0/469579525 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd49290690 0x55bd49296680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38792/0 (socket says 172.21.15.17:38792) 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928fce0 0x55bd49296140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 -- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd49290690 msgr2=0x55bd49296680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd49290690 0x55bd49296680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 -- 172.21.15.17:0/469579525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928fce0 msgr2=0x55bd49296140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928fce0 0x55bd49296140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 -- 172.21.15.17:0/469579525 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc8019040 con 0x55bd49299210 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fcffff640 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd49290690 0x55bd49296680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd523c640 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49299210 0x55bd49296bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9fc801df30 tx=0x7f9fc8004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d6ea2debf8a40e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9faffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9faffff640 1 -- 172.21.15.17:0/469579525 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8011050 con 0x55bd49299210 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9faffff640 1 -- 172.21.15.17:0/469579525 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f9fc80111b0 con 0x55bd49299210 2024-02-14T15:05:24.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.967+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd48ef67e0 con 0x55bd49299210 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9faffff640 1 -- 172.21.15.17:0/469579525 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc8011370 con 0x55bd49299210 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 10 client.16284 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bd491f56b0 con 0x55bd49299210 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9faffff640 1 -- 172.21.15.17:0/469579525 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bd48ef67e0 con 0x55bd49299210 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9faffff640 1 -- 172.21.15.17:0/469579525 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9fc8024070 con 0x55bd49299210 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 10 client.16284 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 10 client.16284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 20 client.16284 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd492a9230 0x55bd492ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 4 client.16284 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:24.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd48f33720 con 0x55bd492a9230 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.968+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd492a9230 0x55bd492ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.969+0000 7f9faffff640 10 client.16284 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.969+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd492a9230 0x55bd492ab620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1cac09ce52f6c11 server_cookie=13a9998cf899df94 in_seq=0 out_seq=0 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.970+0000 7f9faffff640 1 -- 172.21.15.17:0/469579525 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd48f33720 con 0x55bd492a9230 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.970+0000 7f9faffff640 10 client.16284 handle_command_reply: tid=0 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.970+0000 7f9fd9861740 1 client.16284 shutdown 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.970+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49299210 msgr2=0x55bd49296bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.970+0000 7f9fd9861740 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49299210 0x55bd49296bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9fc801df30 tx=0x7f9fc8004ef0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 shutdown_connections 2024-02-14T15:05:24.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd492a9230 0x55bd492ab620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9faffff640 0 client.16284 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49299210 0x55bd49296bc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd49290690 0x55bd49296680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 --2- 172.21.15.17:0/469579525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928fce0 0x55bd49296140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 >> 172.21.15.17:0/469579525 conn(0x55bd4928a3b0 msgr2=0x55bd4928c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 shutdown_connections 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 1 -- 172.21.15.17:0/469579525 wait complete. 2024-02-14T15:05:24.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.971+0000 7f9fd9861740 20 client.16284 trim_cache size 0 max 16384 2024-02-14T15:05:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:24 smithi076 ceph-mon[37057]: pgmap v62: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 95 op/s 2024-02-14T15:05:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:24 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:24.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.984+0000 7f9fd9861740 1 Processor -- start 2024-02-14T15:05:24.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.984+0000 7f9fd9861740 1 -- start start 2024-02-14T15:05:24.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd49298870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd492933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd9861740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4917c130 con 0x55bd49290690 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd9861740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd49285c70 con 0x55bd4928a3b0 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd9861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd4928f630 con 0x55bd4928fce0 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd49298870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd523c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd492933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd49298870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49484/0 (socket says 172.21.15.17:49484) 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 -- 172.21.15.17:0/1038024375 learned_addr learned my addr 172.21.15.17:0/1038024375 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 -- 172.21.15.17:0/1038024375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 msgr2=0x55bd492969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/1038024375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 -- 172.21.15.17:0/1038024375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 msgr2=0x55bd492933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/1038024375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd492933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 -- 172.21.15.17:0/1038024375 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd49191c90 con 0x55bd4928a3b0 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/1038024375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd49298870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005ec0 tx=0x7f9fb802e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=782c08c64539e064 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fc67fc640 1 -- 172.21.15.17:0/1038024375 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8046070 con 0x55bd4928a3b0 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fc67fc640 1 -- 172.21.15.17:0/1038024375 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fb804b070 con 0x55bd4928a3b0 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.985+0000 7f9fc67fc640 1 -- 172.21.15.17:0/1038024375 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb802cb40 con 0x55bd4928a3b0 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 -- 172.21.15.17:0/1038024375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 msgr2=0x55bd49298870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1038024375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd49298870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005ec0 tx=0x7f9fb802e040 comp rx=0 tx=0).stop 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 -- 172.21.15.17:0/1038024375 shutdown_connections 2024-02-14T15:05:24.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1038024375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd492933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1038024375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd492969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1038024375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd49298870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 -- 172.21.15.17:0/1038024375 >> 172.21.15.17:0/1038024375 conn(0x55bd49290de0 msgr2=0x55bd492aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 -- 172.21.15.17:0/1038024375 shutdown_connections 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 -- 172.21.15.17:0/1038024375 wait complete. 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.986+0000 7f9fd9861740 1 -- start start 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 1 Processor -- start 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd4929ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd4929f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd4929f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd492ff7b0 con 0x55bd49290690 2024-02-14T15:05:24.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd492ff8f0 con 0x55bd4928a3b0 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd492ffab0 con 0x55bd4928fce0 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd9861740 10 client.0 _ll_register_callbacks cb 0x7f9fc8009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd4929f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd4929f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49324/0 (socket says 172.21.15.17:49324) 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 -- 172.21.15.17:0/1588490406 learned_addr learned my addr 172.21.15.17:0/1588490406 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fcffff640 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd4929f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd4929ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 -- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 msgr2=0x55bd4929f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd4929f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 -- 172.21.15.17:0/1588490406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 msgr2=0x55bd4929ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd4929ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.987+0000 7f9fd523c640 1 -- 172.21.15.17:0/1588490406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb8030070 con 0x55bd49290690 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd523c640 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd4929f7a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8011fd0 tx=0x7f9fc8026680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee6d4384d5305a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9faffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9faffff640 1 -- 172.21.15.17:0/1588490406 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc801ddd0 con 0x55bd49290690 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9faffff640 1 -- 172.21.15.17:0/1588490406 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f9fc8011240 con 0x55bd49290690 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fc8009cf0 con 0x55bd49290690 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9faffff640 1 -- 172.21.15.17:0/1588490406 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc8011400 con 0x55bd49290690 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 10 client.16290 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bd491b7aa0 con 0x55bd49290690 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9faffff640 1 -- 172.21.15.17:0/1588490406 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8009cf0 con 0x55bd49290690 2024-02-14T15:05:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9faffff640 1 -- 172.21.15.17:0/1588490406 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9fc8024040 con 0x55bd49290690 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 10 client.16290 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 10 client.16290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 20 client.16290 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd49299210 0x55bd49302eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 4 client.16290 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd48f82890 con 0x55bd49299210 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.988+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd49299210 0x55bd49302eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9fd4a3b640 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd49299210 0x55bd49302eb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ce818683f95fe04 server_cookie=80f0251e286b6479 in_seq=0 out_seq=0 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9faffff640 10 client.16290 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9faffff640 1 -- 172.21.15.17:0/1588490406 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd48f82890 con 0x55bd49299210 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9faffff640 10 client.16290 handle_command_reply: tid=0 2024-02-14T15:05:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9fd9861740 1 client.16290 shutdown 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 msgr2=0x55bd4929f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd4929f7a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8011fd0 tx=0x7f9fc8026680 comp rx=0 tx=0).stop 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.989+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 shutdown_connections 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x55bd49299210 0x55bd49302eb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bd49290690 0x55bd4929f7a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9faffff640 0 client.16290 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd4928fce0 0x55bd4929f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 1 --2- 172.21.15.17:0/1588490406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd4928a3b0 0x55bd4929ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 >> 172.21.15.17:0/1588490406 conn(0x55bd49290de0 msgr2=0x55bd49288c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 shutdown_connections 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 1 -- 172.21.15.17:0/1588490406 wait complete. 2024-02-14T15:05:24.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd9861740 20 client.16290 trim_cache size 0 max 16384 2024-02-14T15:05:25.000 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:05:25.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9fbc0a2230 msgr2=0x7f9fbc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9fbc0a2230 0x7f9fbc0a46f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005fd0 tx=0x7f9fc801e000 comp rx=0 tx=0).stop 2024-02-14T15:05:25.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd01008f0 msgr2=0x7f9fd019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.990+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd01008f0 0x7f9fd019fd30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9f0 tx=0x7f9fc000eeb0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 shutdown_connections 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9fbc0a2230 0x7f9fbc0a46f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd01008f0 0x7f9fd019fd30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fd010db80 0x7f9fd01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 --2- 172.21.15.17:0/3883057001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fd00fffd0 0x7f9fd019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 >> 172.21.15.17:0/3883057001 conn(0x7f9fd00772e0 msgr2=0x7f9fd010f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 shutdown_connections 2024-02-14T15:05:25.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:24.991+0000 7f9fd6c9f640 1 -- 172.21.15.17:0/3883057001 wait complete. 2024-02-14T15:05:25.002 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:24 smithi017 ceph-mon[32549]: pgmap v62: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 95 op/s 2024-02-14T15:05:25.003 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:24 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:25.004 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:05:25.004 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-14T15:05:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:24 smithi050 ceph-mon[36512]: pgmap v62: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 95 op/s 2024-02-14T15:05:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:24 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:25.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/2128867297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f8100cf0 msgr2=0x7fe8f8104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/2128867297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f8100cf0 0x7fe8f8104d30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009930 tx=0x7fe8f402bc30 comp rx=0 tx=0).stop 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/2128867297 shutdown_connections 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/2128867297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f8100cf0 0x7fe8f8104d30 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/2128867297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8f8105410 0x7fe8f810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/2128867297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f8100340 0x7fe8f8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/2128867297 >> 172.21.15.17:0/2128867297 conn(0x7fe8f80772e0 msgr2=0x7fe8f80776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/2128867297 shutdown_connections 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.154+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/2128867297 wait complete. 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 Processor -- start 2024-02-14T15:05:25.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 -- start start 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8f8100340 0x7fe8f819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f8105410 0x7fe8f819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f81965f0 0x7fe8f8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8196fe0 con 0x7fe8f81965f0 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8197150 con 0x7fe8f8100340 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ff7f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8197310 con 0x7fe8f8105410 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f81965f0 0x7fe8f8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f81965f0 0x7fe8f8196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49340/0 (socket says 172.21.15.17:49340) 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 -- 172.21.15.17:0/604903161 learned_addr learned my addr 172.21.15.17:0/604903161 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fd590640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8f8100340 0x7fe8f819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f8105410 0x7fe8f819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 -- 172.21.15.17:0/604903161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f8105410 msgr2=0x7fe8f819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f8105410 0x7fe8f819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 -- 172.21.15.17:0/604903161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8f8100340 msgr2=0x7fe8f819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8f8100340 0x7fe8f819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 -- 172.21.15.17:0/604903161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f4009560 con 0x7fe8f81965f0 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8fdd91640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f81965f0 0x7fe8f8196aa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000ebf0 tx=0x7fe8f000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.155+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/604903161 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0013070 con 0x7fe8f81965f0 2024-02-14T15:05:25.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.156+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/604903161 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8f000be40 con 0x7fe8f81965f0 2024-02-14T15:05:25.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.156+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f8068500 con 0x7fe8f81965f0 2024-02-14T15:05:25.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.156+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/604903161 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0017410 con 0x7fe8f81965f0 2024-02-14T15:05:25.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.156+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f80689e0 con 0x7fe8f81965f0 2024-02-14T15:05:25.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.158+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/604903161 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe8f0004a40 con 0x7fe8f81965f0 2024-02-14T15:05:25.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 Processor -- start 2024-02-14T15:05:25.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 -- start start 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f8690 0x55aaa7800c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7803660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa7756c80 con 0x55aaa7801210 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa7766320 con 0x55aaa77f7ce0 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8ea7fc640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe8e40a22b0 0x7fe8e40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe9023c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa7763850 con 0x55aaa77f8690 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8fd590640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f8690 0x55aaa7800c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8fcd8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7803660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8fdd91640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe8e40a22b0 0x7fe8e40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8fcd8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8fd590640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f8690 0x55aaa7800c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38834/0 (socket says 172.21.15.17:38834) 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8fd590640 1 -- 172.21.15.17:0/3303975581 learned_addr learned my addr 172.21.15.17:0/3303975581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.159+0000 7fe8ea7fc640 1 -- 172.21.15.17:0/604903161 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe8f00cfc50 con 0x7fe8f81965f0 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/3303975581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f8690 msgr2=0x55aaa7800c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/3303975581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f8690 0x55aaa7800c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/3303975581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 msgr2=0x55aaa7803660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/3303975581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7803660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/3303975581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa7725e30 con 0x55aaa77f7ce0 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8fdd91640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe8e40a22b0 0x7fe8e40a4770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8f009bf90 tx=0x7fe8f0010040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/3303975581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77f80c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402c170 tx=0x7fe8f4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1273e7b3a51228f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8e37fe640 1 -- 172.21.15.17:0/3303975581 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004740 con 0x55aaa77f7ce0 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8e37fe640 1 -- 172.21.15.17:0/3303975581 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8f4004030 con 0x55aaa77f7ce0 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.160+0000 7fe8e37fe640 1 -- 172.21.15.17:0/3303975581 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4003a70 con 0x55aaa77f7ce0 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 -- 172.21.15.17:0/3303975581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 msgr2=0x55aaa77f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 --2- 172.21.15.17:0/3303975581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77f80c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402c170 tx=0x7fe8f4005fd0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 -- 172.21.15.17:0/3303975581 shutdown_connections 2024-02-14T15:05:25.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 --2- 172.21.15.17:0/3303975581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7803660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 --2- 172.21.15.17:0/3303975581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f8690 0x55aaa7800c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 --2- 172.21.15.17:0/3303975581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77f80c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 -- 172.21.15.17:0/3303975581 >> 172.21.15.17:0/3303975581 conn(0x55aaa77f23b0 msgr2=0x55aaa77f2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 -- 172.21.15.17:0/3303975581 shutdown_connections 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 -- 172.21.15.17:0/3303975581 wait complete. 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 -- start start 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.161+0000 7fe9023c0740 1 Processor -- start 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f8690 0x55aaa77fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe9023c0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa775f030 con 0x55aaa77f8690 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe9023c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa7432430 con 0x55aaa77f7ce0 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe9023c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa73eee00 con 0x55aaa7801210 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe9023c0740 10 client.0 _ll_register_callbacks cb 0x55aaa746fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f8690 0x55aaa77fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f8690 0x55aaa77fe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49366/0 (socket says 172.21.15.17:49366) 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 -- 172.21.15.17:0/202654937 learned_addr learned my addr 172.21.15.17:0/202654937 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 -- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 msgr2=0x55aaa77fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 -- 172.21.15.17:0/202654937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 msgr2=0x55aaa77fe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77fe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 -- 172.21.15.17:0/202654937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f810d7f0 con 0x55aaa77f8690 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8fd590640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f8690 0x55aaa77fe790 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec012720 tx=0x7fe8ec012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b64c7abfd4f1978 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.162+0000 7fe8e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/202654937 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec017070 con 0x55aaa77f8690 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaa745e7e0 con 0x55aaa77f8690 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/202654937 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe8ec004050 con 0x55aaa77f8690 2024-02-14T15:05:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/202654937 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8ec004230 con 0x55aaa77f8690 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 10 client.16302 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aaa775d6b0 con 0x55aaa77f8690 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/202654937 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aaa745e7e0 con 0x55aaa77f8690 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/202654937 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe8ec003a60 con 0x55aaa77f8690 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 10 client.16302 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 10 client.16302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 20 client.16302 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa7813900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 4 client.16302 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:25.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaa749b720 con 0x55aaa78114e0 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.163+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa7813900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.164+0000 7fe8e0ff9640 10 client.16302 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.164+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa7813900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0449e9c510354dc server_cookie=d364cc83afcdd1d6 in_seq=0 out_seq=0 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.164+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/202654937 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aaa749b720 con 0x55aaa78114e0 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.164+0000 7fe8e0ff9640 10 client.16302 handle_command_reply: tid=0 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.164+0000 7fe9023c0740 1 client.16302 shutdown 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.164+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f8690 msgr2=0x55aaa77fe790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.164+0000 7fe9023c0740 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f8690 0x55aaa77fe790 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec012720 tx=0x7fe8ec012be0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 shutdown_connections 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa7813900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f8690 0x55aaa77fe790 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe8e0ff9640 0 client.16302 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 --2- 172.21.15.17:0/202654937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f7ce0 0x55aaa77fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 >> 172.21.15.17:0/202654937 conn(0x55aaa77f23b0 msgr2=0x55aaa77f4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 shutdown_connections 2024-02-14T15:05:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 1 -- 172.21.15.17:0/202654937 wait complete. 2024-02-14T15:05:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.165+0000 7fe9023c0740 20 client.16302 trim_cache size 0 max 16384 2024-02-14T15:05:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.177+0000 7fe9023c0740 1 Processor -- start 2024-02-14T15:05:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.177+0000 7fe9023c0740 1 -- start start 2024-02-14T15:05:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f7ce0 0x55aaa7800980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa77feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe9023c0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa76e4130 con 0x55aaa77f7ce0 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe9023c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa77f90e0 con 0x55aaa77f8690 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe9023c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa780df40 con 0x55aaa7801210 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f7ce0 0x55aaa7800980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fd590640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa77feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fd590640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa77feaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49544/0 (socket says 172.21.15.17:49544) 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fd590640 1 -- 172.21.15.17:0/977295147 learned_addr learned my addr 172.21.15.17:0/977295147 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f7ce0 0x55aaa7800980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49378/0 (socket says 172.21.15.17:49378) 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/977295147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 msgr2=0x55aaa77fcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/977295147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 msgr2=0x55aaa77feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa77feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/977295147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa76f9c90 con 0x55aaa77f7ce0 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f7ce0 0x55aaa7800980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55aaa7814230 tx=0x7fe8f402c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdb7483e40ffbe1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.178+0000 7fe8e2ffd640 1 -- 172.21.15.17:0/977295147 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f405f070 con 0x55aaa77f7ce0 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe8e2ffd640 1 -- 172.21.15.17:0/977295147 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8f405a8d0 con 0x55aaa77f7ce0 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe8e2ffd640 1 -- 172.21.15.17:0/977295147 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4003bc0 con 0x55aaa77f7ce0 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 -- 172.21.15.17:0/977295147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f7ce0 msgr2=0x55aaa7800980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f7ce0 0x55aaa7800980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55aaa7814230 tx=0x7fe8f402c6d0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 -- 172.21.15.17:0/977295147 shutdown_connections 2024-02-14T15:05:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa7801210 0x55aaa77fcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa77feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 --2- 172.21.15.17:0/977295147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa77f7ce0 0x55aaa7800980 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 -- 172.21.15.17:0/977295147 >> 172.21.15.17:0/977295147 conn(0x55aaa77f23b0 msgr2=0x55aaa77f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 -- 172.21.15.17:0/977295147 shutdown_connections 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.179+0000 7fe9023c0740 1 -- 172.21.15.17:0/977295147 wait complete. 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 -- start start 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 Processor -- start 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f7ce0 0x55aaa7806390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa78068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7866a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa7866f40 con 0x55aaa7801210 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa7867080 con 0x55aaa77f8690 2024-02-14T15:05:25.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa78671c0 con 0x55aaa77f7ce0 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe9023c0740 10 client.0 _ll_register_callbacks cb 0x7fe8ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe8fcd8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f7ce0 0x55aaa7806390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe8fcd8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7866a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe8fcd8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7866a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49380/0 (socket says 172.21.15.17:49380) 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/1445196450 learned_addr learned my addr 172.21.15.17:0/1445196450 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.180+0000 7fe8fd590640 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa78068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/1445196450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f7ce0 msgr2=0x55aaa7806390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f7ce0 0x55aaa7806390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/1445196450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 msgr2=0x55aaa78068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa78068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8fcd8f640 1 -- 172.21.15.17:0/1445196450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaa7725e30 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8fcd8f640 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7866a00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4004810 tx=0x7fe8f4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e969fb29342c5669 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/1445196450 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f405f040 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/1445196450 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe8f405aa40 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8f4004030 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/1445196450 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f405ac00 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe9023c0740 10 client.16308 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aaa771faa0 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.181+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/1445196450 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8f4073020 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/1445196450 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe8f4003a70 con 0x55aaa7801210 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 10 client.16308 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:25.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 10 client.16308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 20 client.16308 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa786a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 4 client.16308 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aaa74ea890 con 0x55aaa78114e0 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe8fd590640 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa786a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe8fd590640 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa786a5b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba622f381e8e5c35 server_cookie=4025991f483a685b in_seq=0 out_seq=0 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe8e0ff9640 10 client.16308 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe8e0ff9640 1 -- 172.21.15.17:0/1445196450 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aaa74ea890 con 0x55aaa78114e0 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe8e0ff9640 10 client.16308 handle_command_reply: tid=0 2024-02-14T15:05:25.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 1 client.16308 shutdown 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 msgr2=0x55aaa7866a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.182+0000 7fe9023c0740 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7866a00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4004810 tx=0x7fe8f4005fd0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 shutdown_connections 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55aaa78114e0 0x55aaa786a5b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55aaa7801210 0x55aaa7866a00 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8e0ff9640 0 client.16308 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa77f8690 0x55aaa78068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 --2- 172.21.15.17:0/1445196450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55aaa77f7ce0 0x55aaa7806390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 >> 172.21.15.17:0/1445196450 conn(0x55aaa77f23b0 msgr2=0x55aaa77fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 shutdown_connections 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 1 -- 172.21.15.17:0/1445196450 wait complete. 2024-02-14T15:05:25.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe9023c0740 20 client.16308 trim_cache size 0 max 16384 2024-02-14T15:05:25.194 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe8e40a22b0 msgr2=0x7fe8e40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe8e40a22b0 0x7fe8e40a4770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8f009bf90 tx=0x7fe8f0010040 comp rx=0 tx=0).stop 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f81965f0 msgr2=0x7fe8f8196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f81965f0 0x7fe8f8196aa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000ebf0 tx=0x7fe8f000b690 comp rx=0 tx=0).stop 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 shutdown_connections 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe8e40a22b0 0x7fe8e40a4770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f81965f0 0x7fe8f8196aa0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f8105410 0x7fe8f819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 --2- 172.21.15.17:0/604903161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8f8100340 0x7fe8f819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 >> 172.21.15.17:0/604903161 conn(0x7fe8f80772e0 msgr2=0x7fe8f81065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 shutdown_connections 2024-02-14T15:05:25.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.183+0000 7fe8ff7f4640 1 -- 172.21.15.17:0/604903161 wait complete. 2024-02-14T15:05:25.197 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:05:25.197 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-14T15:05:25.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.305+0000 7f27c49a0640 1 -- 172.21.15.17:0/1548975866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 msgr2=0x7f27c010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.305+0000 7f27c49a0640 1 --2- 172.21.15.17:0/1548975866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c010df70 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009cd0 tx=0x7f27b802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.305+0000 7f27c49a0640 1 -- 172.21.15.17:0/1548975866 shutdown_connections 2024-02-14T15:05:25.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.305+0000 7f27c49a0640 1 --2- 172.21.15.17:0/1548975866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c010df70 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.305+0000 7f27c49a0640 1 --2- 172.21.15.17:0/1548975866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27c0100570 0x7f27c0105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.305+0000 7f27c49a0640 1 --2- 172.21.15.17:0/1548975866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27c00ffbc0 0x7f27c00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.305+0000 7f27c49a0640 1 -- 172.21.15.17:0/1548975866 >> 172.21.15.17:0/1548975866 conn(0x7f27c0075200 msgr2=0x7f27c0075610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.306+0000 7f27c49a0640 1 -- 172.21.15.17:0/1548975866 shutdown_connections 2024-02-14T15:05:25.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.306+0000 7f27c49a0640 1 -- 172.21.15.17:0/1548975866 wait complete. 2024-02-14T15:05:25.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.306+0000 7f27c49a0640 1 Processor -- start 2024-02-14T15:05:25.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.306+0000 7f27c49a0640 1 -- start start 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27c49a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27c00ffbc0 0x7f27c01976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27c49a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c0197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27c49a0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27c019bf90 0x7f27c019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27c49a0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c019c980 con 0x7f27c0106380 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27c49a0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c019caf0 con 0x7f27c019bf90 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27c49a0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c019cc60 con 0x7f27c00ffbc0 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c0197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27c00ffbc0 0x7f27c01976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27bffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27c019bf90 0x7f27c019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c0197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49404/0 (socket says 172.21.15.17:49404) 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 -- 172.21.15.17:0/4272637913 learned_addr learned my addr 172.21.15.17:0/4272637913 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 -- 172.21.15.17:0/4272637913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27c00ffbc0 msgr2=0x7f27c01976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27c00ffbc0 0x7f27c01976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 -- 172.21.15.17:0/4272637913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27c019bf90 msgr2=0x7f27c019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27c019bf90 0x7f27c019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 -- 172.21.15.17:0/4272637913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8009900 con 0x7f27c0106380 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27bffff640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27c019bf90 0x7f27c019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27beffd640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c0197c10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f27b400e9c0 tx=0x7f27b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27bcff9640 1 -- 172.21.15.17:0/4272637913 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4009700 con 0x7f27c0106380 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27bcff9640 1 -- 172.21.15.17:0/4272637913 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27b400cd80 con 0x7f27c0106380 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.307+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c01a4240 con 0x7f27c0106380 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.308+0000 7f27bcff9640 1 -- 172.21.15.17:0/4272637913 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4010430 con 0x7f27c0106380 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.308+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c01a46f0 con 0x7f27c0106380 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27bcff9640 1 -- 172.21.15.17:0/4272637913 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f27b4005000 con 0x7f27c0106380 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 Processor -- start 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27bcff9640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27a80a2230 0x7f27a80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 -- start start 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27bffff640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27a80a2230 0x7f27a80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27bcff9640 1 -- 172.21.15.17:0/4272637913 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f27b4014070 con 0x7f27c0106380 2024-02-14T15:05:25.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225aceb0 0x55df225ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 0x55df225b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225b63e0 0x55df225b8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2250bc80 con 0x55df225aceb0 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2251b320 con 0x55df225b63e0 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.310+0000 7f27c87d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df22518850 con 0x55df225ad860 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225aceb0 0x55df225ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225aceb0 0x55df225ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49412/0 (socket says 172.21.15.17:49412) 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1318177187 learned_addr learned my addr 172.21.15.17:0/1318177187 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bffff640 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225b63e0 0x55df225b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1318177187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 msgr2=0x55df225b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 0x55df225b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1318177187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225b63e0 msgr2=0x55df225b8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225b63e0 0x55df225b8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1318177187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df224dae30 con 0x55df225aceb0 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bffff640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27a80a2230 0x7f27a80a46f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005cc0 tx=0x7f27b8032e50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225aceb0 0x55df225ad290 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d20 tx=0x7f27b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7e5e7e92e5df25 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27a5ffb640 1 -- 172.21.15.17:0/1318177187 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0019630 con 0x55df225aceb0 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.311+0000 7f27a5ffb640 1 -- 172.21.15.17:0/1318177187 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27b0019c70 con 0x55df225aceb0 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27a5ffb640 1 -- 172.21.15.17:0/1318177187 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000ec70 con 0x55df225aceb0 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 -- 172.21.15.17:0/1318177187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225aceb0 msgr2=0x55df225ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225aceb0 0x55df225ad290 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d20 tx=0x7f27b0011810 comp rx=0 tx=0).stop 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 -- 172.21.15.17:0/1318177187 shutdown_connections 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225b63e0 0x55df225b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 0x55df225b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1318177187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225aceb0 0x55df225ad290 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 -- 172.21.15.17:0/1318177187 >> 172.21.15.17:0/1318177187 conn(0x55df225a7580 msgr2=0x55df225a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 -- 172.21.15.17:0/1318177187 shutdown_connections 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.312+0000 7f27c87d3740 1 -- 172.21.15.17:0/1318177187 wait complete. 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 -- start start 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 Processor -- start 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225ad860 0x55df225b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225b63e0 0x55df225b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df22514030 con 0x55df225b63e0 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df221e7430 con 0x55df225ad860 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df221a3e00 con 0x55df225aceb0 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27c87d3740 10 client.0 _ll_register_callbacks cb 0x55df22224be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27bffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225b63e0 0x55df225b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27beffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225ad860 0x55df225b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38894/0 (socket says 172.21.15.17:38894) 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27bffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225b63e0 0x55df225b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49418/0 (socket says 172.21.15.17:49418) 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27beffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225ad860 0x55df225b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49606/0 (socket says 172.21.15.17:49606) 2024-02-14T15:05:25.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.313+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1024982729 learned_addr learned my addr 172.21.15.17:0/1024982729 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27bffff640 1 -- 172.21.15.17:0/1024982729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 msgr2=0x55df225b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27bffff640 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27bffff640 1 -- 172.21.15.17:0/1024982729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225ad860 msgr2=0x55df225b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27bffff640 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225ad860 0x55df225b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27bffff640 1 -- 172.21.15.17:0/1024982729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b0009950 con 0x55df225b63e0 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27bffff640 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225b63e0 0x55df225b3dd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f27b802c630 tx=0x7f27b802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa78df03a111fd90 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f279effd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f279effd640 1 -- 172.21.15.17:0/1024982729 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8003970 con 0x55df225b63e0 2024-02-14T15:05:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f279effd640 1 -- 172.21.15.17:0/1024982729 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f27b8003ad0 con 0x55df225b63e0 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df222137e0 con 0x55df225b63e0 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f279effd640 1 -- 172.21.15.17:0/1024982729 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b8003cd0 con 0x55df225b63e0 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27c87d3740 10 client.16320 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.314+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55df225126b0 con 0x55df225b63e0 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f279effd640 1 -- 172.21.15.17:0/1024982729 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55df222137e0 con 0x55df225b63e0 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f279effd640 1 -- 172.21.15.17:0/1024982729 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f27b80416b0 con 0x55df225b63e0 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27c87d3740 10 client.16320 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27c87d3740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27c87d3740 20 client.16320 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225c6540 0x55df225c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27c87d3740 4 client.16320 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df22250720 con 0x55df225c6540 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225c6540 0x55df225c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f279effd640 10 client.16320 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.315+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225c6540 0x55df225c8930 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d58c16d46d607f server_cookie=9765b7665042ddf8 in_seq=0 out_seq=0 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.316+0000 7f279effd640 1 -- 172.21.15.17:0/1024982729 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df22250720 con 0x55df225c6540 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.316+0000 7f279effd640 10 client.16320 handle_command_reply: tid=0 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.316+0000 7f27c87d3740 1 client.16320 shutdown 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.316+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225b63e0 msgr2=0x55df225b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.316+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225b63e0 0x55df225b3dd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f27b802c630 tx=0x7f27b802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 shutdown_connections 2024-02-14T15:05:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225c6540 0x55df225c8930 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225b63e0 0x55df225b3dd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225ad860 0x55df225b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f279effd640 0 client.16320 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1024982729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 >> 172.21.15.17:0/1024982729 conn(0x55df225a7580 msgr2=0x55df225a92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 shutdown_connections 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 1 -- 172.21.15.17:0/1024982729 wait complete. 2024-02-14T15:05:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.317+0000 7f27c87d3740 20 client.16320 trim_cache size 0 max 16384 2024-02-14T15:05:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 Processor -- start 2024-02-14T15:05:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 -- start start 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225a7580 0x55df225b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225ad860 0x55df225bd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df22499130 con 0x55df225ad860 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df225a31f0 con 0x55df225a7580 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27c87d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df225c4de0 con 0x55df225aceb0 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27bffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225ad860 0x55df225bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27bffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225ad860 0x55df225bd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49422/0 (socket says 172.21.15.17:49422) 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225a7580 0x55df225b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.330+0000 7f27bffff640 1 -- 172.21.15.17:0/2936727353 learned_addr learned my addr 172.21.15.17:0/2936727353 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27beffd640 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27bffff640 1 -- 172.21.15.17:0/2936727353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 msgr2=0x55df225a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27bffff640 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225a90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27bffff640 1 -- 172.21.15.17:0/2936727353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225a7580 msgr2=0x55df225b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27bffff640 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225a7580 0x55df225b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27bffff640 1 -- 172.21.15.17:0/2936727353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df224aec90 con 0x55df225ad860 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27bffff640 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225ad860 0x55df225bd410 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006010 tx=0x7f27b80347f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb32b02c4d06bd server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27a57fa640 1 -- 172.21.15.17:0/2936727353 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8041bd0 con 0x55df225ad860 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27a57fa640 1 -- 172.21.15.17:0/2936727353 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27b8041d30 con 0x55df225ad860 2024-02-14T15:05:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.331+0000 7f27a57fa640 1 -- 172.21.15.17:0/2936727353 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b80042a0 con 0x55df225ad860 2024-02-14T15:05:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 -- 172.21.15.17:0/2936727353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225ad860 msgr2=0x55df225bd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225ad860 0x55df225bd410 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006010 tx=0x7f27b80347f0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 -- 172.21.15.17:0/2936727353 shutdown_connections 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225ad860 0x55df225bd410 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225aceb0 0x55df225a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 --2- 172.21.15.17:0/2936727353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225a7580 0x55df225b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 -- 172.21.15.17:0/2936727353 >> 172.21.15.17:0/2936727353 conn(0x55df225adfb0 msgr2=0x55df225a89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 -- 172.21.15.17:0/2936727353 shutdown_connections 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.332+0000 7f27c87d3740 1 -- 172.21.15.17:0/2936727353 wait complete. 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 -- start start 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 Processor -- start 2024-02-14T15:05:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225a7580 0x55df225bbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225aceb0 0x55df225bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 0x55df225bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2261ca20 con 0x55df225a7580 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2261cb60 con 0x55df225aceb0 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2261cd20 con 0x55df225ad860 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27c87d3740 10 client.0 _ll_register_callbacks cb 0x7f27b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225a7580 0x55df225bbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225a7580 0x55df225bbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49438/0 (socket says 172.21.15.17:49438) 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1064542251 learned_addr learned my addr 172.21.15.17:0/1064542251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bffff640 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 0x55df225bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27beffd640 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225aceb0 0x55df225bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1064542251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 msgr2=0x55df225bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 0x55df225bca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225aceb0 msgr2=0x55df225bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.333+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225aceb0 0x55df225bc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27bf7fe640 1 -- 172.21.15.17:0/1064542251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df225126b0 con 0x55df225a7580 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27bf7fe640 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225a7580 0x55df225bbfd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d20 tx=0x7f27b000e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88244160e7f1e03b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f279effd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f279effd640 1 -- 172.21.15.17:0/1064542251 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000ea40 con 0x55df225a7580 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f279effd640 1 -- 172.21.15.17:0/1064542251 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f27b000eba0 con 0x55df225a7580 2024-02-14T15:05:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27b0019c70 con 0x55df225a7580 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f279effd640 1 -- 172.21.15.17:0/1064542251 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b000ed80 con 0x55df225a7580 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 10 client.16326 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55df224d4aa0 con 0x55df225a7580 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f279effd640 1 -- 172.21.15.17:0/1064542251 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27b0019c70 con 0x55df225a7580 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f279effd640 1 -- 172.21.15.17:0/1064542251 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f27b00197e0 con 0x55df225a7580 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 10 client.16326 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 20 client.16326 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225b63e0 0x55df226204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 4 client.16326 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:05:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.334+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df2229f890 con 0x55df225b63e0 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27beffd640 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225b63e0 0x55df226204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27beffd640 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225b63e0 0x55df226204e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b178bd7e1605f11 server_cookie=7b9f3a4436d9ac5a in_seq=0 out_seq=0 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f279effd640 10 client.16326 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f279effd640 1 -- 172.21.15.17:0/1064542251 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df2229f890 con 0x55df225b63e0 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f279effd640 10 client.16326 handle_command_reply: tid=0 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 client.16326 shutdown 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225a7580 msgr2=0x55df225bbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225a7580 0x55df225bbfd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d20 tx=0x7f27b000e8c0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 shutdown_connections 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55df225b63e0 0x55df226204e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df225ad860 0x55df225bca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df225aceb0 0x55df225bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 --2- 172.21.15.17:0/1064542251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55df225a7580 0x55df225bbfd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f279effd640 0 client.16326 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 >> 172.21.15.17:0/1064542251 conn(0x55df225adfb0 msgr2=0x55df225a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 shutdown_connections 2024-02-14T15:05:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.335+0000 7f27c87d3740 1 -- 172.21.15.17:0/1064542251 wait complete. 2024-02-14T15:05:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c87d3740 20 client.16326 trim_cache size 0 max 16384 2024-02-14T15:05:25.345 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:05:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27a80a2230 msgr2=0x7f27a80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27a80a2230 0x7f27a80a46f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005cc0 tx=0x7f27b8032e50 comp rx=0 tx=0).stop 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 msgr2=0x7f27c0197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c0197c10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f27b400e9c0 tx=0x7f27b400ee80 comp rx=0 tx=0).stop 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 shutdown_connections 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27a80a2230 0x7f27a80a46f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27c0106380 0x7f27c0197c10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27c019bf90 0x7f27c019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 --2- 172.21.15.17:0/4272637913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27c00ffbc0 0x7f27c01976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 >> 172.21.15.17:0/4272637913 conn(0x7f27c0075200 msgr2=0x7f27c00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 shutdown_connections 2024-02-14T15:05:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:25.336+0000 7f27c49a0640 1 -- 172.21.15.17:0/4272637913 wait complete. 2024-02-14T15:05:25.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 34413 7.42 MB/sec warmup 27 sec latency 20.931 ms 2024-02-14T15:05:25.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:25 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3638796183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:05:26.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:25 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3638796183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:05:26.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:25 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3638796183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:05:26.349 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-14T15:05:26.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:05:26.350 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-14T15:05:26.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 -- 172.21.15.17:0/248391106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee04102450 msgr2=0x7fee04102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 --2- 172.21.15.17:0/248391106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee04102450 0x7fee04102830 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009ca0 tx=0x7fedf402c180 comp rx=0 tx=0).stop 2024-02-14T15:05:26.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 -- 172.21.15.17:0/248391106 shutdown_connections 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 --2- 172.21.15.17:0/248391106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee0410b820 0x7fee0410dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 --2- 172.21.15.17:0/248391106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102d70 0x7fee0410b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 --2- 172.21.15.17:0/248391106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee04102450 0x7fee04102830 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 -- 172.21.15.17:0/248391106 >> 172.21.15.17:0/248391106 conn(0x7fee040772e0 msgr2=0x7fee040776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 -- 172.21.15.17:0/248391106 shutdown_connections 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 -- 172.21.15.17:0/248391106 wait complete. 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.462+0000 7fee0957a640 1 Processor -- start 2024-02-14T15:05:26.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee0957a640 1 -- start start 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee0957a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee04102450 0x7fee0419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee0957a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102d70 0x7fee0419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee0957a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0410b820 0x7fee04197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee0957a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee04197eb0 con 0x7fee0410b820 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee0957a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee04198020 con 0x7fee04102450 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee0957a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee041981e0 con 0x7fee04102d70 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0410b820 0x7fee04197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee027fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102d70 0x7fee0419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0410b820 0x7fee04197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49450/0 (socket says 172.21.15.17:49450) 2024-02-14T15:05:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee027fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102d70 0x7fee0419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38924/0 (socket says 172.21.15.17:38924) 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 -- 172.21.15.17:0/2460755810 learned_addr learned my addr 172.21.15.17:0/2460755810 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee02ffd640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee04102450 0x7fee0419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 -- 172.21.15.17:0/2460755810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102d70 msgr2=0x7fee0419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102d70 0x7fee0419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 -- 172.21.15.17:0/2460755810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee04102450 msgr2=0x7fee0419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee04102450 0x7fee0419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.463+0000 7fee037fe640 1 -- 172.21.15.17:0/2460755810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf4009900 con 0x7fee0410b820 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.464+0000 7fee02ffd640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee04102450 0x7fee0419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.464+0000 7fee037fe640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0410b820 0x7fee04197720 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d700 tx=0x7fedfc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.464+0000 7fedebfff640 1 -- 172.21.15.17:0/2460755810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc004020 con 0x7fee0410b820 2024-02-14T15:05:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.464+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee041984c0 con 0x7fee0410b820 2024-02-14T15:05:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.464+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee041a45e0 con 0x7fee0410b820 2024-02-14T15:05:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.464+0000 7fedebfff640 1 -- 172.21.15.17:0/2460755810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedfc009d00 con 0x7fee0410b820 2024-02-14T15:05:26.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.464+0000 7fedebfff640 1 -- 172.21.15.17:0/2460755810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc003cf0 con 0x7fee0410b820 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.466+0000 7fedebfff640 1 -- 172.21.15.17:0/2460755810 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fedfc010460 con 0x7fee0410b820 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.466+0000 7fee0c227740 1 Processor -- start 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.466+0000 7fee0c227740 1 -- start start 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee0c227740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe336d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee0c227740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe33760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee0c227740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe33789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee0c227740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe3328bd0 con 0x55afe336d0f0 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee0c227740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe32db030 con 0x55afe33765f0 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee0c227740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe32d8560 con 0x55afe336da10 2024-02-14T15:05:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee027fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe336d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe33760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee027fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe336d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49452/0 (socket says 172.21.15.17:49452) 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe33760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38940/0 (socket says 172.21.15.17:38940) 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee027fc640 1 -- 172.21.15.17:0/132963788 learned_addr learned my addr 172.21.15.17:0/132963788 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee027fc640 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe33789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fedebfff640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fedec0a2230 0x7fedec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 -- 172.21.15.17:0/132963788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 msgr2=0x55afe33789e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe33789e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 -- 172.21.15.17:0/132963788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 msgr2=0x55afe336d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe336d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 -- 172.21.15.17:0/132963788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe3296290 con 0x55afe336da10 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee027fc640 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe33789e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee037fe640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fedec0a2230 0x7fedec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fedebfff640 1 -- 172.21.15.17:0/2460755810 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fedfc0ce0b0 con 0x7fee0410b820 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.467+0000 7fee02ffd640 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe33760b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fedf4006010 tx=0x7fedf40323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f617e1475b457dc server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fede97fa640 1 -- 172.21.15.17:0/132963788 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4034640 con 0x55afe336da10 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fede97fa640 1 -- 172.21.15.17:0/132963788 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf4003e50 con 0x55afe336da10 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fede97fa640 1 -- 172.21.15.17:0/132963788 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4032ed0 con 0x55afe336da10 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee037fe640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fedec0a2230 0x7fedec0a46f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0cb550 tx=0x7fedfc085040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 -- 172.21.15.17:0/132963788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 msgr2=0x55afe33760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe33760b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fedf4006010 tx=0x7fedf40323c0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 -- 172.21.15.17:0/132963788 shutdown_connections 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe33789e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe33760b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 --2- 172.21.15.17:0/132963788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe336d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 -- 172.21.15.17:0/132963788 >> 172.21.15.17:0/132963788 conn(0x55afe3367820 msgr2=0x55afe3367c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 -- 172.21.15.17:0/132963788 shutdown_connections 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.468+0000 7fee0c227740 1 -- 172.21.15.17:0/132963788 wait complete. 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 -- start start 2024-02-14T15:05:26.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 Processor -- start 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe3373580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe3373ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe3374000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe32d3d40 con 0x55afe336d0f0 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe3278e60 con 0x55afe33765f0 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe2fa74b0 con 0x55afe336da10 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee0c227740 10 client.0 _ll_register_callbacks cb 0x55afe328f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe3373580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee02ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe3373ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe3373580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49466/0 (socket says 172.21.15.17:49466) 2024-02-14T15:05:26.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 -- 172.21.15.17:0/153682943 learned_addr learned my addr 172.21.15.17:0/153682943 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe3374000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 -- 172.21.15.17:0/153682943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 msgr2=0x55afe3373ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe3373ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 -- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 msgr2=0x55afe3374000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe3374000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee027fc640 1 -- 172.21.15.17:0/153682943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0410e8d0 con 0x55afe336d0f0 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.469+0000 7fee02ffd640 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe3373ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fee027fc640 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe3373580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fedf8011bb0 tx=0x7fedf8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6847cc606258289b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fede6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fede6ff5640 1 -- 172.21.15.17:0/153682943 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf80112f0 con 0x55afe336d0f0 2024-02-14T15:05:26.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fede6ff5640 1 -- 172.21.15.17:0/153682943 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fedf8004050 con 0x55afe336d0f0 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afe32f9dd0 con 0x55afe336d0f0 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fede6ff5640 1 -- 172.21.15.17:0/153682943 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedf8004210 con 0x55afe336d0f0 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fee0c227740 10 client.16338 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55afe32d23c0 con 0x55afe336d0f0 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fede6ff5640 1 -- 172.21.15.17:0/153682943 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55afe32f9dd0 con 0x55afe336d0f0 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fede6ff5640 1 -- 172.21.15.17:0/153682943 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fedf8010550 con 0x55afe336d0f0 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fee0c227740 10 client.16338 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.470+0000 7fee0c227740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.471+0000 7fee0c227740 20 client.16338 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.471+0000 7fee0c227740 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe3386660 0x55afe3388a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.471+0000 7fee0c227740 4 client.16338 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.471+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afe32a6cb0 con 0x55afe3386660 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.471+0000 7fee02ffd640 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe3386660 0x55afe3388a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.471+0000 7fee02ffd640 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe3386660 0x55afe3388a80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0a91650f766b36 server_cookie=eb49ea72565a2015 in_seq=0 out_seq=0 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.471+0000 7fede6ff5640 10 client.16338 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fede6ff5640 1 -- 172.21.15.17:0/153682943 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55afe32a6cb0 con 0x55afe3386660 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fede6ff5640 10 client.16338 handle_command_reply: tid=0 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 client.16338 shutdown 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 msgr2=0x55afe3373580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe3373580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fedf8011bb0 tx=0x7fedf8005f90 comp rx=0 tx=0).stop 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 shutdown_connections 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe3386660 0x55afe3388a80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe33765f0 0x55afe3374000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe336da10 0x55afe3373ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fede6ff5640 0 client.16338 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 --2- 172.21.15.17:0/153682943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336d0f0 0x55afe3373580 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 >> 172.21.15.17:0/153682943 conn(0x55afe3367820 msgr2=0x55afe3369570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 shutdown_connections 2024-02-14T15:05:26.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 1 -- 172.21.15.17:0/153682943 wait complete. 2024-02-14T15:05:26.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.472+0000 7fee0c227740 20 client.16338 trim_cache size 0 max 16384 2024-02-14T15:05:26.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 Processor -- start 2024-02-14T15:05:26.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 -- start start 2024-02-14T15:05:26.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe3375cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe3369310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe337d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe3259670 con 0x55afe336da10 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe3368620 con 0x55afe336d0f0 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee0c227740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe33789e0 con 0x55afe33765f0 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe3369310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe3369310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49482/0 (socket says 172.21.15.17:49482) 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 -- 172.21.15.17:0/3996355471 learned_addr learned my addr 172.21.15.17:0/3996355471 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee027fc640 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe337d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee027fc640 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe3375cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 -- 172.21.15.17:0/3996355471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 msgr2=0x55afe337d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe337d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 -- 172.21.15.17:0/3996355471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 msgr2=0x55afe3375cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe3375cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.489+0000 7fee02ffd640 1 -- 172.21.15.17:0/3996355471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe326ec70 con 0x55afe336da10 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.490+0000 7fee02ffd640 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe3369310 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fedf402c1c0 tx=0x7fedf4040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b8b96216467e34 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.490+0000 7fede8ff9640 1 -- 172.21.15.17:0/3996355471 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4032ed0 con 0x55afe336da10 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.490+0000 7fede8ff9640 1 -- 172.21.15.17:0/3996355471 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf4032aa0 con 0x55afe336da10 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.490+0000 7fede8ff9640 1 -- 172.21.15.17:0/3996355471 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4055740 con 0x55afe336da10 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 -- 172.21.15.17:0/3996355471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 msgr2=0x55afe3369310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe3369310 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fedf402c1c0 tx=0x7fedf4040040 comp rx=0 tx=0).stop 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 -- 172.21.15.17:0/3996355471 shutdown_connections 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe3369310 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe337d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 --2- 172.21.15.17:0/3996355471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe3375cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 -- 172.21.15.17:0/3996355471 >> 172.21.15.17:0/3996355471 conn(0x55afe3367820 msgr2=0x55afe3368c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 -- 172.21.15.17:0/3996355471 shutdown_connections 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 -- 172.21.15.17:0/3996355471 wait complete. 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 -- start start 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 Processor -- start 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe337c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe337c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.491+0000 7fee0c227740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe33dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee0c227740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe337d080 con 0x55afe336da10 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee0c227740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe33dd070 con 0x55afe336d0f0 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee0c227740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe33dd1b0 con 0x55afe33765f0 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe337c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee0c227740 10 client.0 _ll_register_callbacks cb 0x7fedf8009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe337c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49488/0 (socket says 172.21.15.17:49488) 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 -- 172.21.15.17:0/389024427 learned_addr learned my addr 172.21.15.17:0/389024427 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee027fc640 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe337c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee027fc640 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe33dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 -- 172.21.15.17:0/389024427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 msgr2=0x55afe33dcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe33dcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 -- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 msgr2=0x55afe337c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe337c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 -- 172.21.15.17:0/389024427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afe3296290 con 0x55afe336da10 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee02ffd640 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe337c7b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009ca0 tx=0x7fedf4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c51ea141a768621 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fede6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fede6ff5640 1 -- 172.21.15.17:0/389024427 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4032540 con 0x55afe336da10 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fede6ff5640 1 -- 172.21.15.17:0/389024427 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fedf404e040 con 0x55afe336da10 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.492+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedf4003e50 con 0x55afe336da10 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fede6ff5640 1 -- 172.21.15.17:0/389024427 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedf4034440 con 0x55afe336da10 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 10 client.16344 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55afe3294af0 con 0x55afe336da10 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fede6ff5640 1 -- 172.21.15.17:0/389024427 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf4003e50 con 0x55afe336da10 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fede6ff5640 1 -- 172.21.15.17:0/389024427 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fedf4052070 con 0x55afe336da10 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 10 client.16344 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:26.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 20 client.16344 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe33de670 0x55afe33e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 4 client.16344 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55afe2fd3860 con 0x55afe33de670 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.493+0000 7fee027fc640 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe33de670 0x55afe33e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fede6ff5640 10 client.16344 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee027fc640 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe33de670 0x55afe33e0a90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8420070694dcae73 server_cookie=a8c170f5c4bb01c1 in_seq=0 out_seq=0 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fede6ff5640 1 -- 172.21.15.17:0/389024427 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55afe2fd3860 con 0x55afe33de670 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fede6ff5640 10 client.16344 handle_command_reply: tid=0 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 client.16344 shutdown 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 msgr2=0x55afe337c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe337c7b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009ca0 tx=0x7fedf4003970 comp rx=0 tx=0).stop 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 shutdown_connections 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55afe33de670 0x55afe33e0a90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55afe336da10 0x55afe337c7b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afe33765f0 0x55afe33dcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 --2- 172.21.15.17:0/389024427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afe336d0f0 0x55afe337c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fede6ff5640 0 client.16344 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.494+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 >> 172.21.15.17:0/389024427 conn(0x55afe3367820 msgr2=0x55afe3366a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 shutdown_connections 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0c227740 1 -- 172.21.15.17:0/389024427 wait complete. 2024-02-14T15:05:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0c227740 20 client.16344 trim_cache size 0 max 16384 2024-02-14T15:05:26.504 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:05:26.504 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2024-02-14T15:05:26.504 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_tag": "905c2bd6-5caf-4851-bcf5-58412e92832f", 2024-02-14T15:05:26.504 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": "asynchronous" 2024-02-14T15:05:26.504 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:05:26.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fedec0a2230 msgr2=0x7fedec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fedec0a2230 0x7fedec0a46f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0cb550 tx=0x7fedfc085040 comp rx=0 tx=0).stop 2024-02-14T15:05:26.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0410b820 msgr2=0x7fee04197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0410b820 0x7fee04197720 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d700 tx=0x7fedfc00dbc0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 shutdown_connections 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fedec0a2230 0x7fedec0a46f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee0410b820 0x7fee04197720 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee04102d70 0x7fee0419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 --2- 172.21.15.17:0/2460755810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee04102450 0x7fee0419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 >> 172.21.15.17:0/2460755810 conn(0x7fee040772e0 msgr2=0x7fee040fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.495+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 shutdown_connections 2024-02-14T15:05:26.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.496+0000 7fee0957a640 1 -- 172.21.15.17:0/2460755810 wait complete. 2024-02-14T15:05:26.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:05:26.508 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:05:26.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.625+0000 7f6aa498a640 1 -- 172.21.15.17:0/983794043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102d70 msgr2=0x7f6aa010b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.625+0000 7f6aa498a640 1 --2- 172.21.15.17:0/983794043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102d70 0x7f6aa010b250 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009900 tx=0x7f6a9402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.625+0000 7f6aa498a640 1 -- 172.21.15.17:0/983794043 shutdown_connections 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.625+0000 7f6aa498a640 1 --2- 172.21.15.17:0/983794043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102d70 0x7f6aa010b250 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.625+0000 7f6aa498a640 1 --2- 172.21.15.17:0/983794043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6aa010b820 0x7f6aa010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.625+0000 7f6aa498a640 1 --2- 172.21.15.17:0/983794043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102450 0x7f6aa0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.625+0000 7f6aa498a640 1 -- 172.21.15.17:0/983794043 >> 172.21.15.17:0/983794043 conn(0x7f6aa00772e0 msgr2=0x7f6aa00776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 -- 172.21.15.17:0/983794043 shutdown_connections 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 -- 172.21.15.17:0/983794043 wait complete. 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 Processor -- start 2024-02-14T15:05:26.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 -- start start 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102450 0x7f6aa019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102d70 0x7f6aa019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6aa010b820 0x7f6aa01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0197e40 con 0x7f6aa0102450 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0197fb0 con 0x7f6aa010b820 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.626+0000 7f6aa498a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0198170 con 0x7f6aa0102d70 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102450 0x7f6aa019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102d70 0x7f6aa019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102450 0x7f6aa019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49514/0 (socket says 172.21.15.17:49514) 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 -- 172.21.15.17:0/549959539 learned_addr learned my addr 172.21.15.17:0/549959539 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102d70 0x7f6aa019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38992/0 (socket says 172.21.15.17:38992) 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9ed9d640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6aa010b820 0x7f6aa01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 -- 172.21.15.17:0/549959539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102d70 msgr2=0x7f6aa019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102d70 0x7f6aa019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 -- 172.21.15.17:0/549959539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6aa010b820 msgr2=0x7f6aa01977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6aa010b820 0x7f6aa01977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 -- 172.21.15.17:0/549959539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a90009490 con 0x7f6aa0102450 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102d70 0x7f6aa019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:26.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102450 0x7f6aa019d630 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000ea60 tx=0x7f6a9000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a8f7fe640 1 -- 172.21.15.17:0/549959539 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000cde0 con 0x7f6aa0102450 2024-02-14T15:05:26.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a94009560 con 0x7f6aa0102450 2024-02-14T15:05:26.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a8f7fe640 1 -- 172.21.15.17:0/549959539 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a900035f0 con 0x7f6aa0102450 2024-02-14T15:05:26.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6a8f7fe640 1 -- 172.21.15.17:0/549959539 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90010690 con 0x7f6aa0102450 2024-02-14T15:05:26.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.627+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01a4480 con 0x7f6aa0102450 2024-02-14T15:05:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.630+0000 7f6a8f7fe640 1 -- 172.21.15.17:0/549959539 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6a90003af0 con 0x7f6aa0102450 2024-02-14T15:05:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.630+0000 7f6a8f7fe640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6a880a2230 0x7f6a880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6a8f7fe640 1 -- 172.21.15.17:0/549959539 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6a90014070 con 0x7f6aa0102450 2024-02-14T15:05:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6a880a2230 0x7f6a880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 Processor -- start 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 -- start start 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583630 0x55997a583a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583f50 0x55997a58c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a58ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a53e5b0 con 0x55997a583630 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a4f22b0 con 0x55997a583f50 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6aa7631740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a4ef7e0 con 0x55997a58cb30 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6a880a2230 0x7f6a880a46f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006010 tx=0x7f6a94005e40 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6a9e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583630 0x55997a583a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6a9e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583630 0x55997a583a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49518/0 (socket says 172.21.15.17:49518) 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6a9e59c640 1 -- 172.21.15.17:0/720147099 learned_addr learned my addr 172.21.15.17:0/720147099 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.631+0000 7f6a9ed9d640 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a58ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583f50 0x55997a58c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9e59c640 1 -- 172.21.15.17:0/720147099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 msgr2=0x55997a58ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a58ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9e59c640 1 -- 172.21.15.17:0/720147099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583f50 msgr2=0x55997a58c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583f50 0x55997a58c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9e59c640 1 -- 172.21.15.17:0/720147099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997a4ad020 con 0x55997a583630 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9ed9d640 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a58ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583630 0x55997a583a10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6a9009af40 tx=0x7f6a9009a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538e41bcf2c413f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a8cff9640 1 -- 172.21.15.17:0/720147099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900a3750 con 0x55997a583630 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a8cff9640 1 -- 172.21.15.17:0/720147099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a900cbf90 con 0x55997a583630 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.632+0000 7f6a8cff9640 1 -- 172.21.15.17:0/720147099 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90003960 con 0x55997a583630 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 -- 172.21.15.17:0/720147099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583630 msgr2=0x55997a583a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583630 0x55997a583a10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6a9009af40 tx=0x7f6a9009a9d0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 -- 172.21.15.17:0/720147099 shutdown_connections 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a58ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583f50 0x55997a58c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 --2- 172.21.15.17:0/720147099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583630 0x55997a583a10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 -- 172.21.15.17:0/720147099 >> 172.21.15.17:0/720147099 conn(0x55997a57dd60 msgr2=0x55997a57e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 -- 172.21.15.17:0/720147099 shutdown_connections 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 -- 172.21.15.17:0/720147099 wait complete. 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 -- start start 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.633+0000 7f6aa7631740 1 Processor -- start 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a589bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a58a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a58a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6aa7631740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a4eafc0 con 0x55997a58cb30 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6aa7631740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a1be430 con 0x55997a583630 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6aa7631740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a17ae00 con 0x55997a583f50 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6aa7631740 10 client.0 _ll_register_callbacks cb 0x55997a46c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a58a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a589bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a58a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:39002/0 (socket says 172.21.15.17:39002) 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a58a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 -- 172.21.15.17:0/2160103572 learned_addr learned my addr 172.21.15.17:0/2160103572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 -- 172.21.15.17:0/2160103572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 msgr2=0x55997a589bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a589bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 -- 172.21.15.17:0/2160103572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 msgr2=0x55997a58a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a58a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 -- 172.21.15.17:0/2160103572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01a4480 con 0x55997a583f50 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a58a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009900 tx=0x7f6a94009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb07acccddc200b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.634+0000 7f6a85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6a85ffb640 1 -- 172.21.15.17:0/2160103572 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94004520 con 0x55997a583f50 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55997a1ea7e0 con 0x55997a583f50 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6a85ffb640 1 -- 172.21.15.17:0/2160103572 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6a94047070 con 0x55997a583f50 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6a85ffb640 1 -- 172.21.15.17:0/2160103572 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a94005000 con 0x55997a583f50 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 10 client.25885 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55997a4e9640 con 0x55997a583f50 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6a85ffb640 1 -- 172.21.15.17:0/2160103572 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55997a1ea7e0 con 0x55997a583f50 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6a85ffb640 1 -- 172.21.15.17:0/2160103572 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6a9403f440 con 0x55997a583f50 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 10 client.25885 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 20 client.25885 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a59cea0 0x55997a59f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 4 client.25885 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.635+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55997a227720 con 0x55997a59cea0 2024-02-14T15:05:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.636+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a59cea0 0x55997a59f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.636+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a59cea0 0x55997a59f2c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe59b10aebb01c8 server_cookie=55be22a894b39cc1 in_seq=0 out_seq=0 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.636+0000 7f6a85ffb640 10 client.25885 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6a85ffb640 1 -- 172.21.15.17:0/2160103572 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55997a227720 con 0x55997a59cea0 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6a85ffb640 10 client.25885 handle_command_reply: tid=0 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 client.25885 shutdown 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 msgr2=0x55997a58a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a58a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009900 tx=0x7f6a94009260 comp rx=0 tx=0).stop 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 shutdown_connections 2024-02-14T15:05:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a59cea0 0x55997a59f2c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a58a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a58a110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 --2- 172.21.15.17:0/2160103572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a589bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6a85ffb640 0 client.25885 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 >> 172.21.15.17:0/2160103572 conn(0x55997a57dd60 msgr2=0x55997a57fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 shutdown_connections 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 1 -- 172.21.15.17:0/2160103572 wait complete. 2024-02-14T15:05:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.637+0000 7f6aa7631740 20 client.25885 trim_cache size 0 max 16384 2024-02-14T15:05:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 Processor -- start 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 -- start start 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a58c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a584f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a585450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a46ffd0 con 0x55997a58cb30 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a227720 con 0x55997a583630 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6aa7631740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a1ea7e0 con 0x55997a583f50 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6a9ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a585450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6a9ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a585450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49540/0 (socket says 172.21.15.17:49540) 2024-02-14T15:05:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6a9ed9d640 1 -- 172.21.15.17:0/4213513731 learned_addr learned my addr 172.21.15.17:0/4213513731 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a584f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.657+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a58c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a9ed9d640 1 -- 172.21.15.17:0/4213513731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 msgr2=0x55997a584f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a9ed9d640 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a584f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a9ed9d640 1 -- 172.21.15.17:0/4213513731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 msgr2=0x55997a58c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a9ed9d640 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a58c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a9ed9d640 1 -- 172.21.15.17:0/4213513731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997a485b30 con 0x55997a58cb30 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a9ed9d640 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a585450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6a980124d0 tx=0x7f6a980128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba87d2036bbd4c09 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a87fff640 1 -- 172.21.15.17:0/4213513731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98017070 con 0x55997a58cb30 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a87fff640 1 -- 172.21.15.17:0/4213513731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a98003970 con 0x55997a58cb30 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.658+0000 7f6a87fff640 1 -- 172.21.15.17:0/4213513731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004730 con 0x55997a58cb30 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 -- 172.21.15.17:0/4213513731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 msgr2=0x55997a585450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a585450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6a980124d0 tx=0x7f6a980128d0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 -- 172.21.15.17:0/4213513731 shutdown_connections 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a58cb30 0x55997a585450 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a583f50 0x55997a584f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 --2- 172.21.15.17:0/4213513731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a58c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 -- 172.21.15.17:0/4213513731 >> 172.21.15.17:0/4213513731 conn(0x55997a57dd60 msgr2=0x55997a5a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 -- 172.21.15.17:0/4213513731 shutdown_connections 2024-02-14T15:05:26.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 -- 172.21.15.17:0/4213513731 wait complete. 2024-02-14T15:05:26.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 -- start start 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.659+0000 7f6aa7631740 1 Processor -- start 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a592490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583f50 0x55997a5929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6aa7631740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a592f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6aa7631740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a5f33d0 con 0x55997a583f50 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6aa7631740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a5f3510 con 0x55997a583630 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6aa7631740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997a5f3680 con 0x55997a58cb30 2024-02-14T15:05:26.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6aa7631740 10 client.0 _ll_register_callbacks cb 0x7f6a90096b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a592490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583f50 0x55997a5929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a592490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49742/0 (socket says 172.21.15.17:49742) 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583f50 0x55997a5929d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49552/0 (socket says 172.21.15.17:49552) 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9dd9b640 1 -- 172.21.15.17:0/3734122140 learned_addr learned my addr 172.21.15.17:0/3734122140 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a592f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 -- 172.21.15.17:0/3734122140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 msgr2=0x55997a592f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a592f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 -- 172.21.15.17:0/3734122140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583f50 msgr2=0x55997a5929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583f50 0x55997a5929d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 -- 172.21.15.17:0/3734122140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a98009d20 con 0x55997a583630 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583f50 0x55997a5929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9ed9d640 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a592f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a9e59c640 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a592490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a900a3220 tx=0x7f6a90097380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0cab365546894f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.660+0000 7f6a85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6a85ffb640 1 -- 172.21.15.17:0/3734122140 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900cc100 con 0x55997a583630 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6a85ffb640 1 -- 172.21.15.17:0/3734122140 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6a900a3750 con 0x55997a583630 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a900cbf90 con 0x55997a583630 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6a85ffb640 1 -- 172.21.15.17:0/3734122140 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a9009f100 con 0x55997a583630 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 10 client.25766 fetch_fsmap learned FSMap version 26 2024-02-14T15:05:26.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55997a4ab880 con 0x55997a583630 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6a85ffb640 1 -- 172.21.15.17:0/3734122140 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a900cbf90 con 0x55997a583630 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6a85ffb640 1 -- 172.21.15.17:0/3734122140 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6a9009a400 con 0x55997a583630 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 10 client.25766 fetch_fsmap finished waiting for FSMap version 26 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 20 client.25766 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a5f4bf0 0x55997a5f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 4 client.25766 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55997a276890 con 0x55997a5f4bf0 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.661+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a5f4bf0 0x55997a5f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6a9dd9b640 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a5f4bf0 0x55997a5f7010 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717c387e5ed2ccb8 server_cookie=fa64c359f11b9ab in_seq=0 out_seq=0 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6a85ffb640 10 client.25766 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6a85ffb640 1 -- 172.21.15.17:0/3734122140 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55997a276890 con 0x55997a5f4bf0 2024-02-14T15:05:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6a85ffb640 10 client.25766 handle_command_reply: tid=0 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 client.25766 shutdown 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 msgr2=0x55997a592490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a592490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a900a3220 tx=0x7f6a90097380 comp rx=0 tx=0).stop 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 shutdown_connections 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55997a5f4bf0 0x55997a5f7010 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55997a583f50 0x55997a5929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6a85ffb640 0 client.25766 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55997a58cb30 0x55997a592f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 --2- 172.21.15.17:0/3734122140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55997a583630 0x55997a592490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 >> 172.21.15.17:0/3734122140 conn(0x55997a57dd60 msgr2=0x55997a584f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 shutdown_connections 2024-02-14T15:05:26.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 1 -- 172.21.15.17:0/3734122140 wait complete. 2024-02-14T15:05:26.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.662+0000 7f6aa7631740 20 client.25766 trim_cache size 0 max 16384 2024-02-14T15:05:26.675 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:05:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-14T15:05:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": { 2024-02-14T15:05:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "905c2bd6-5caf-4851-bcf5-58412e92832f": { 2024-02-14T15:05:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/", 2024-02-14T15:05:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "tag": "905c2bd6-5caf-4851-bcf5-58412e92832f", 2024-02-14T15:05:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "options": "recursive,force" 2024-02-14T15:05:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:05:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:05:26.676 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:05:26.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6a880a2230 msgr2=0x7f6a880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6a880a2230 0x7f6a880a46f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006010 tx=0x7f6a94005e40 comp rx=0 tx=0).stop 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102450 msgr2=0x7f6aa019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102450 0x7f6aa019d630 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000ea60 tx=0x7f6a9000ef20 comp rx=0 tx=0).stop 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 shutdown_connections 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6a880a2230 0x7f6a880a46f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6aa010b820 0x7f6aa01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa0102d70 0x7f6aa019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 --2- 172.21.15.17:0/549959539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aa0102450 0x7f6aa019d630 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 >> 172.21.15.17:0/549959539 conn(0x7f6aa00772e0 msgr2=0x7f6aa00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:26.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 shutdown_connections 2024-02-14T15:05:26.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:26.663+0000 7f6aa498a640 1 -- 172.21.15.17:0/549959539 wait complete. 2024-02-14T15:05:26.679 INFO:tasks.cephfs.filesystem:scrub status for tag:905c2bd6-5caf-4851-bcf5-58412e92832f - {'path': '/', 'tag': '905c2bd6-5caf-4851-bcf5-58412e92832f', 'options': 'recursive,force'} 2024-02-14T15:05:26.860 INFO:tasks.workunit.client.0.smithi017.stdout: 1 36943 7.76 MB/sec warmup 28 sec latency 8.716 ms 2024-02-14T15:05:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-mon[37057]: pgmap v63: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 95 op/s 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.498+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/ auth{0=1,1=1} v33 ap=1 f(v0 m2024-02-14T15:04:58.872471+0000 1=0+1) n(v1 rc2024-02-14T15:05:26.207188+0000 b541755 94=82+12) old_inodes=1 (isnap sync r=13) (inest mix dirty) (ipolicy sync r=7) (ifile excl) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@1},l=25037 | dirtyscattered=1 lock=2 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x564dbfd39180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000007:[<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000007:[<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:04:58.872471+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:26.207188+0000 b541755 94=82+12)","memory_value.dirstat":"f(v0 m2024-02-14T15:04:58.872471+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-02-14T15:05:26.207188+0000 b541755 94=82+12)","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.498+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000008 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/ auth{0=1,1=1} v43 ap=1 f(v0 m2024-02-14T15:04:58.907471+0000 6=5+1) n(v1 rc2024-02-14T15:05:26.207188+0000 b541755 93=82+11) old_inodes=1 (isnap sync r=13) (inest mix dirty) (ipolicy sync r=7) (ifile excl) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@1},l=25037 | dirtyscattered=1 lock=2 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfd39700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000008:[<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000008:[<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:04:58.907471+0000 6=5+1)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:26.207188+0000 b541755 93=82+11)","memory_value.dirstat":"f(v0 m2024-02-14T15:04:58.907471+0000 6=5+1)","memory_value.rstat":"n(v1 rc2024-02-14T15:05:26.207188+0000 b541755 93=82+11)","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.499+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000009 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ auth{0=1,1=1} v61 ap=1 f(v0 m2024-02-14T15:04:59.133469+0000 9=0+9) n(v2 rc2024-02-14T15:05:26.207188+0000 b541755 87=77+10) old_inodes=1 (isnap sync r=13) (inest mix w=2 dirty) (ipolicy sync r=7) (ifile excl) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@1},l=25037 | dirtyscattered=1 lock=3 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000009:[<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000009:[<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:04:59.133469+0000 9=0+9)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:26.207188+0000 b541755 87=77+10)","memory_value.dirstat":"f(v0 m2024-02-14T15:04:59.133469+0000 9=0+9)","memory_value.rstat":"n(v2 rc2024-02-14T15:05:26.207188+0000 b541755 87=77+10)","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.499+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.001 auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbfed9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000204:[<0x10000000008/filler.001 v6>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000204:[<0x10000000008/filler.001 v6>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.499+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.002 auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff56000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000205:[<0x10000000008/filler.002 v8>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000205:[<0x10000000008/filler.002 v8>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.500+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.004 auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff56580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000207:[<0x10000000008/filler.004 v12>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000207:[<0x10000000008/filler.004 v12>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.500+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.003 auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff56b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000206:[<0x10000000008/filler.003 v10>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000206:[<0x10000000008/filler.003 v10>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.501+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000a [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.000 auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff57080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000000a:[<0x10000000008/filler.000 v4>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000000a:[<0x10000000008/filler.000 v4>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.990 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.501+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{0=1,1=1} v6787 ap=3 f(v0 m2024-02-14T15:05:26.230188+0000 15=15+0) n(v0 rc2024-02-14T15:05:26.230188+0000 16=15+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@565},l=25037 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:26.230188+0000 15=15+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:26.230188+0000 16=15+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:26.230188+0000 15=15+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:05:26.230188+0000 16=15+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.501+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{0=1,1=1} v6627 ap=2 f(v0 m2024-02-14T15:05:25.936191+0000 8=8+0) n(v0 rc2024-02-14T15:05:25.936191+0000 b88384 9=8+1) old_inodes=1 (iauth sync r=1) (isnap sync r=1) (inest mix w=1) (ipolicy sync r=1) (ifile excl w=1) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@777},l=25037 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v14>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v6627>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:25.936191+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:25.936191+0000 b88384 9=8+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:25.936191+0000 8=8+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:05:25.936191+0000 b88384 9=8+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.502+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{0=1,1=1} v6467 ap=3 f(v0 m2024-02-14T15:05:25.563195+0000 5=5+0) n(v1 rc2024-02-14T15:05:25.563195+0000 6=5+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@446},l=25037 | dirtyscattered=0 request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v8>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v6467>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:25.563195+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:25.563195+0000 6=5+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:25.563195+0000 5=5+0)","memory_value.rstat":"n(v1 rc2024-02-14T15:05:25.563195+0000 6=5+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.502+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000240 [...3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/ auth{1=1} v120 ap=1 f(v0 m2024-02-14T15:04:59.146469+0000 3=3+0) n(v0 rc2024-02-14T15:04:59.146469+0000 4=3+1) (inest mix) (ifile excl) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@1},l=25037 | lock=0 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000240:[<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000240:[<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:04:59.146469+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:04:59.146469+0000 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:04:59.146469+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:04:59.146469+0000 4=3+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.503+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{0=1,1=1} v6705 ap=1 f(v0 m2024-02-14T15:05:26.079190+0000 9=9+0) n(v0 rc2024-02-14T15:05:26.079190+0000 b31173 10=9+1) old_inodes=1 (inest mix) (ifile excl) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@624},l=25037 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v58>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v6705>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:26.079190+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:26.079190+0000 b31173 10=9+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:26.079190+0000 9=9+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:05:26.079190+0000 b31173 10=9+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.503+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{0=1,1=1} v6543 ap=3 f(v0 m2024-02-14T15:05:25.769193+0000 24=24+0) n(v0 rc2024-02-14T15:05:25.769193+0000 25=24+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@914},l=25037 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:25.769193+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:25.769193+0000 25=24+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:25.769193+0000 24=24+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:05:25.769193+0000 25=24+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.503+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/ auth{0=1,1=1} v6929 pv6933 ap=7 f(v0 m2024-02-14T15:05:26.489185+0000 4=4+0)->f(v0 m2024-02-14T15:05:26.496185+0000 4=4+0) n(v0 rc2024-02-14T15:05:26.489185+0000 5=4+1)->n(v0 rc2024-02-14T15:05:26.496185+0000 5=4+1) old_inodes=1 (iauth sync r=1) (isnap sync r=3) (inest mix w=2) (ipolicy sync r=1) (ifile excl->sync w=2) (iversion lock w=2 last_client=25037) caps={25037=pAsLsXsFs/pAsLsXsFsx@1358},l=25037 | request=1 lock=6 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 waiter=1 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v66>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v6929>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v22>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:26.489185+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:26.489185+0000 5=4+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:26.489185+0000 4=4+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:05:26.489185+0000 5=4+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.505+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{0=1,1=1} v6411 ap=3 f(v0 m2024-02-14T15:05:25.471196+0000 7=7+0) n(v0 rc2024-02-14T15:05:25.471196+0000 b162102 8=7+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@453},l=25037 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v186>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v6411>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:25.471196+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:25.471196+0000 b162102 8=7+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:25.471196+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:05:25.471196+0000 b162102 8=7+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.506+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{0=1,1=1} v6735 ap=3 f(v0 m2024-02-14T15:05:26.131189+0000 4=4+0) n(v0 rc2024-02-14T15:05:26.131189+0000 b260096 5=4+1) old_inodes=1 (iauth sync r=1) (isnap sync r=2) (inest mix w=2) (ipolicy sync r=1) (ifile excl w=2) (iversion lock) caps={25037=pAsLsXsFsx/pAsLsXsFsx@236},l=25037 | request=0 lock=5 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x564dbfed9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v112>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v6735>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-14T15:05:26.131189+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-02-14T15:05:26.131189+0000 b260096 5=4+1)","memory_value.dirstat":"f(v0 m2024-02-14T15:05:26.131189+0000 4=4+0)","memory_value.rstat":"n(v0 rc2024-02-14T15:05:26.131189+0000 b260096 5=4+1)","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.506+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@119},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff57600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.991 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.506+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@119},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff57b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.506+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@119},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff58100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.507+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@132},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff58680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.507+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@126},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff58c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.508+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@119},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff59180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.509+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@119},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff59700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.509+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@126},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff64000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.510+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@119},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff64580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.510+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@125},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff64b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.992 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.510+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@126},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff65080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v50>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v6787>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.510+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@168},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff65600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v14>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v6627>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.511+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@168},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff65b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v14>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v6627>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.512+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@97},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff66680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020b:[<0x10000000208/FACTS.DOC v6>,<0x10000000009/WORD v8>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000020b:[<0x10000000208/FACTS.DOC v6>,<0x10000000009/WORD v6467>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.513+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@97},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff66100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020a:[<0x10000000208/BASEMACH.DOC v4>,<0x10000000009/WORD v8>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000020a:[<0x10000000208/BASEMACH.DOC v4>,<0x10000000009/WORD v6467>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.514+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@97},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff67180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000209:[<0x10000000208/CHAP10.DOC v2>,<0x10000000009/WORD v8>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000209:[<0x10000000208/CHAP10.DOC v2>,<0x10000000009/WORD v6467>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.514+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000241 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff74000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000241:[<0x10000000240/SMALL.FIL v2>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000241:[<0x10000000240/SMALL.FIL v2>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.514+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000242 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff74580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000242:[<0x10000000240/MEDIUM.FIL v4>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000242:[<0x10000000240/MEDIUM.FIL v4>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.514+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000243 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff74b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000243:[<0x10000000240/LARGE.FIL v6>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000243:[<0x10000000240/LARGE.FIL v6>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.515+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000222 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@148},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff75080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v58>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v6705>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.515+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000224 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@148},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff75b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v58>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v6705>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.993 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.515+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000223 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@148},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff75600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v58>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v6705>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.515+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022b [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff76100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022b:[<0x10000000229/COURSES.PX v4>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000022b:[<0x10000000229/COURSES.PX v4>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.515+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022c [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04 auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff76680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022c:[<0x10000000229/COURSES.X04 v6>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000022c:[<0x10000000229/COURSES.X04 v6>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.515+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000231 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0 auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff77180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000231:[<0x10000000229/STUDENTS.YG0 v16>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000231:[<0x10000000229/STUDENTS.YG0 v16>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.516+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000237 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff77700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000237:[<0x10000000229/ENROLL.PX v28>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000237:[<0x10000000229/ENROLL.PX v28>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.516+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022f [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff80000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022f:[<0x10000000229/STUDENTS.PX v12>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000022f:[<0x10000000229/STUDENTS.PX v12>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.516+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023b [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff80580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023b:[<0x10000000229/STUDENTS.TV v36>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000023b:[<0x10000000229/STUDENTS.TV v36>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.516+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023c [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff80b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023c:[<0x10000000229/STUDENTS.FSL v38>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000023c:[<0x10000000229/STUDENTS.FSL v38>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.517+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000233 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1 auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff81080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000233:[<0x10000000229/STUDENTS.YG1 v20>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000233:[<0x10000000229/STUDENTS.YG1 v20>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.517+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000239 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03 auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff81600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000239:[<0x10000000229/ENROLL.Y03 v32>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000239:[<0x10000000229/ENROLL.Y03 v32>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.994 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.517+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023d [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff81b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023d:[<0x10000000229/COURSES.FSL v40>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000023d:[<0x10000000229/COURSES.FSL v40>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.517+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000234 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff82100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000234:[<0x10000000229/STUDENTS.VAL v22>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000234:[<0x10000000229/STUDENTS.VAL v22>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.518+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022a [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff82680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022a:[<0x10000000229/COURSES.DB v2>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000022a:[<0x10000000229/COURSES.DB v2>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.518+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022d [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04 auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff82c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022d:[<0x10000000229/COURSES.Y04 v8>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000022d:[<0x10000000229/COURSES.Y04 v8>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.519+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000238 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03 auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff83180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000238:[<0x10000000229/ENROLL.X03 v30>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000238:[<0x10000000229/ENROLL.X03 v30>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.519+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000235 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff83700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000235:[<0x10000000229/STUDENTS.MB v24>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000235:[<0x10000000229/STUDENTS.MB v24>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.520+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023a [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023a:[<0x10000000229/ENROLL.VAL v34>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000023a:[<0x10000000229/ENROLL.VAL v34>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.521+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000236 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000236:[<0x10000000229/ENROLL.DB v26>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000236:[<0x10000000229/ENROLL.DB v26>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.521+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000232 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1 auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000232:[<0x10000000229/STUDENTS.XG1 v18>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000232:[<0x10000000229/STUDENTS.XG1 v18>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.523+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000230 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0 auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000230:[<0x10000000229/STUDENTS.XG0 v14>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000230:[<0x10000000229/STUDENTS.XG0 v14>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.523+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022e [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@180},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022e:[<0x10000000229/STUDENTS.DB v10>,<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000022e:[<0x10000000229/STUDENTS.DB v10>,<0x10000000009/PARADOX v6543>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.995 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.525+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@267},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v66>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v6933>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.525+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@267},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v66>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v6933>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.527+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000228 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@267},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff8f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v66>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v6933>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.529+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@103},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff9e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v186>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v6411>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.530+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@109},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff9f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v186>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v6411>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.532+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@103},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbff9fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v186>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v6411>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.533+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@103},l=25037 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbffa0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v186>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v6411>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.534+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [3,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@2} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@109},l=25037 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x564dbffa0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v186>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v6411>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:26.996 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:05:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:05:26.539+0000 7fb84fb3b700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [4,head] /volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB auth v338 ap=1 s=260096 n(v0 rc2024-02-14T15:05:26.094190+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25037=0-4194304@3} caps={25037=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@56},l=25037 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x564dbffa1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023f:[<0x1000000023e/FASTENER.MDB v2>,<0x10000000009/ACCESS v112>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v33>]//[3]","memoryvalue":"(4)0x1000000023f:[<0x1000000023e/FASTENER.MDB v338>,<0x10000000009/ACCESS v6735>,<0x10000000008/~dmtmp v61>,<0x10000000007/client0 v43>,<0x10000000006/clients v33>,<0x10000000005/tmp v24>,<0x100000001fb/client.0 v23>,<0x100000001fa/5a1981ee-3c8b-4fe5-965e-923670054a4f v39>,<0x10000000001/sv_1 v55>,<0x10000000000/qa v29>,<0x1/volumes v23>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-14T15:05:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:26 smithi017 ceph-mon[32549]: pgmap v63: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 95 op/s 2024-02-14T15:05:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:26 smithi050 ceph-mon[36512]: pgmap v63: 129 pgs: 129 active+clean; 9.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 95 op/s 2024-02-14T15:05:27.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 39422 7.98 MB/sec warmup 29 sec latency 158.955 ms 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: scrub queued for path: / 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: scrub summary: active paths [/] 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: scrub summary: active paths [/] 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.985 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.001) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.002) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.004) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.003) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000000a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.000) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.986 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.987 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.988 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.989 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.990 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.991 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.992 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.992 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.992 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.992 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.992 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.992 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:27.992 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:27 smithi076 ceph-mon[37057]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: scrub queued for path: / 2024-02-14T15:05:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: scrub summary: active paths [/] 2024-02-14T15:05:28.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: scrub summary: active paths [/] 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.001) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.002) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.004) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.003) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000000a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.000) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.018 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.019 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.022 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.023 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.024 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.025 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.025 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:27 smithi017 ceph-mon[32549]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: scrub queued for path: / 2024-02-14T15:05:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: scrub summary: active paths [/] 2024-02-14T15:05:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: scrub summary: active paths [/] 2024-02-14T15:05:28.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.001) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.002) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.004) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.003) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000000a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.000) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.147 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.149 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.c log and `damage ls` output for details 2024-02-14T15:05:28.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:27 smithi050 ceph-mon[36512]: scrub summary: idle+waiting paths [/] 2024-02-14T15:05:28.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 42589 8.29 MB/sec warmup 30 sec latency 7.019 ms 2024-02-14T15:05:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:28 smithi076 ceph-mon[37057]: pgmap v64: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-02-14T15:05:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:28 smithi017 ceph-mon[32549]: pgmap v64: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-02-14T15:05:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:28 smithi050 ceph-mon[36512]: pgmap v64: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-02-14T15:05:29.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 44967 8.30 MB/sec warmup 31 sec latency 22.401 ms 2024-02-14T15:05:30.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 47050 8.45 MB/sec warmup 32 sec latency 17.263 ms 2024-02-14T15:05:30.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:30 smithi076 ceph-mon[37057]: pgmap v65: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-02-14T15:05:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:30 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32545]: 2024-02-14T15:05:30.654+0000 7ff3e6861700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-14T15:05:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:30 smithi017 ceph-mon[32549]: pgmap v65: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-02-14T15:05:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:30 smithi050 ceph-mon[36512]: pgmap v65: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-02-14T15:05:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 -- 172.21.15.17:0/834046882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 msgr2=0x7faa3410e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 --2- 172.21.15.17:0/834046882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3410e100 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009cd0 tx=0x7faa2c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 -- 172.21.15.17:0/834046882 shutdown_connections 2024-02-14T15:05:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 --2- 172.21.15.17:0/834046882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3410e100 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 --2- 172.21.15.17:0/834046882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa34101e10 0x7faa34105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 --2- 172.21.15.17:0/834046882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa34101460 0x7faa34101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 -- 172.21.15.17:0/834046882 >> 172.21.15.17:0/834046882 conn(0x7faa340fcce0 msgr2=0x7faa340ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 -- 172.21.15.17:0/834046882 shutdown_connections 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.296+0000 7faa39415640 1 -- 172.21.15.17:0/834046882 wait complete. 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 Processor -- start 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 -- start start 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa34101460 0x7faa3419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa34197780 0x7faa34197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34198250 con 0x7faa34106530 2024-02-14T15:05:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa341983c0 con 0x7faa34101460 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa39415640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34198580 con 0x7faa34197780 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3419dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39380/0 (socket says 172.21.15.17:39380) 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa34101460 0x7faa3419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 -- 172.21.15.17:0/427188110 learned_addr learned my addr 172.21.15.17:0/427188110 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa337fe640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa34197780 0x7faa34197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 -- 172.21.15.17:0/427188110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa34197780 msgr2=0x7faa34197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa34197780 0x7faa34197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 -- 172.21.15.17:0/427188110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa34101460 msgr2=0x7faa3419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa34101460 0x7faa3419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa2bfff640 1 -- 172.21.15.17:0/427188110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c009900 con 0x7faa34106530 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.297+0000 7faa32ffd640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa34101460 0x7faa3419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.298+0000 7faa2bfff640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3419dbb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00e9c0 tx=0x7faa1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.298+0000 7faa30ff9640 1 -- 172.21.15.17:0/427188110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c009700 con 0x7faa34106530 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.298+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa341a44d0 con 0x7faa34106530 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.298+0000 7faa30ff9640 1 -- 172.21.15.17:0/427188110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa1c00cd80 con 0x7faa34106530 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.298+0000 7faa30ff9640 1 -- 172.21.15.17:0/427188110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c010430 con 0x7faa34106530 2024-02-14T15:05:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.298+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa341a49b0 con 0x7faa34106530 2024-02-14T15:05:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.301+0000 7faa30ff9640 1 -- 172.21.15.17:0/427188110 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7faa1c0105f0 con 0x7faa34106530 2024-02-14T15:05:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.301+0000 7faa3bfe1740 1 Processor -- start 2024-02-14T15:05:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.301+0000 7faa3bfe1740 1 -- start start 2024-02-14T15:05:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831b9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa30ff9640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7faa200a2220 0x7faa200a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831cb350 0x561b831cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b831279e0 con 0x561b831b9190 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b83124f10 con 0x561b831b9b40 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b831206f0 con 0x561b831cb350 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa2bfff640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7faa200a2220 0x7faa200a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831b9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39392/0 (socket says 172.21.15.17:39392) 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 -- 172.21.15.17:0/2139753251 learned_addr learned my addr 172.21.15.17:0/2139753251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831b9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:51318/0 (socket says 172.21.15.17:51318) 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831cb350 0x561b831cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa30ff9640 1 -- 172.21.15.17:0/427188110 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7faa1c014070 con 0x7faa34106530 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 -- 172.21.15.17:0/2139753251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 msgr2=0x561b831b9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831b9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 -- 172.21.15.17:0/2139753251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 msgr2=0x561b831b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831b9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa337fe640 1 -- 172.21.15.17:0/2139753251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b82e31920 con 0x561b831cb350 2024-02-14T15:05:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.302+0000 7faa32ffd640 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831b9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.303+0000 7faa2bfff640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7faa200a2220 0x7faa200a46e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa1c09ada0 tx=0x7faa1c004a50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.303+0000 7faa337fe640 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831cb350 0x561b831cd740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa2c004af0 tx=0x7faa2c005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f702a17dbf69f7ee server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.303+0000 7faa297fa640 1 -- 172.21.15.17:0/2139753251 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c034530 con 0x561b831cb350 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.303+0000 7faa297fa640 1 -- 172.21.15.17:0/2139753251 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa2c034690 con 0x561b831cb350 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.303+0000 7faa297fa640 1 -- 172.21.15.17:0/2139753251 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c003bb0 con 0x561b831cb350 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2139753251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831cb350 msgr2=0x561b831cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831cb350 0x561b831cd740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa2c004af0 tx=0x7faa2c005c50 comp rx=0 tx=0).stop 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2139753251 shutdown_connections 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831cb350 0x561b831cd740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831b9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2139753251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2139753251 >> 172.21.15.17:0/2139753251 conn(0x561b831b3e50 msgr2=0x561b831b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2139753251 shutdown_connections 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.304+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2139753251 wait complete. 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 -- start start 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 Processor -- start 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831cb350 0x561b831bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b830bcfa0 con 0x561b831b9190 2024-02-14T15:05:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b82df4250 con 0x561b831cb350 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b830a6200 con 0x561b831b9b40 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa3bfe1740 10 client.0 _ll_register_callbacks cb 0x561b83115b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831be0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39402/0 (socket says 172.21.15.17:39402) 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:34784/0 (socket says 172.21.15.17:34784) 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa337fe640 1 -- 172.21.15.17:0/2279017664 learned_addr learned my addr 172.21.15.17:0/2279017664 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa337fe640 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831cb350 0x561b831bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.305+0000 7faa32ffd640 1 -- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831cb350 msgr2=0x561b831bba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa32ffd640 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831cb350 0x561b831bba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa32ffd640 1 -- 172.21.15.17:0/2279017664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 msgr2=0x561b831be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa32ffd640 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831be0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa32ffd640 1 -- 172.21.15.17:0/2279017664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3410e910 con 0x561b831b9b40 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa32ffd640 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bb470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa24012cf0 tx=0x7faa2400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e6dc637f85bd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa1a7fc640 1 -- 172.21.15.17:0/2279017664 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24017070 con 0x561b831b9b40 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa1a7fc640 1 -- 172.21.15.17:0/2279017664 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7faa240120d0 con 0x561b831b9b40 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b82e20600 con 0x561b831b9b40 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa1a7fc640 1 -- 172.21.15.17:0/2279017664 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa24012290 con 0x561b831b9b40 2024-02-14T15:05:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa3bfe1740 10 client.25903 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b8311ed70 con 0x561b831b9b40 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa1a7fc640 1 -- 172.21.15.17:0/2279017664 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561b82e20600 con 0x561b831b9b40 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.306+0000 7faa1a7fc640 1 -- 172.21.15.17:0/2279017664 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faa24010790 con 0x561b831b9b40 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa3bfe1740 10 client.25903 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa3bfe1740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa3bfe1740 20 client.25903 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b831db270 0x561b831dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa3bfe1740 4 client.25903 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b82e711b0 con 0x561b831db270 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa337fe640 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b831db270 0x561b831dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa1a7fc640 10 client.25903 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.307+0000 7faa337fe640 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b831db270 0x561b831dd690 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64fd96d1f57cbe8a server_cookie=5980fd0ee0cccce in_seq=0 out_seq=0 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa1a7fc640 1 -- 172.21.15.17:0/2279017664 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7faa2c057a90 con 0x561b831db270 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa1a7fc640 10 client.25903 handle_command_reply: tid=0 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 client.25903 shutdown 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 msgr2=0x561b831bb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bb470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa24012cf0 tx=0x7faa2400f790 comp rx=0 tx=0).stop 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 shutdown_connections 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b831db270 0x561b831dd690 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831cb350 0x561b831bba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bb470 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/2279017664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831b9190 0x561b831be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa1a7fc640 0 client.25903 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 >> 172.21.15.17:0/2279017664 conn(0x561b831b3e50 msgr2=0x561b831b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 shutdown_connections 2024-02-14T15:05:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 1 -- 172.21.15.17:0/2279017664 wait complete. 2024-02-14T15:05:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.308+0000 7faa3bfe1740 20 client.25903 trim_cache size 0 max 16384 2024-02-14T15:05:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.322+0000 7faa3bfe1740 1 Processor -- start 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.322+0000 7faa3bfe1740 1 -- start start 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9190 0x561b831e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b831af0f0 con 0x561b831cb350 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b831d7c40 con 0x561b831b9190 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b831ca600 con 0x561b831b9b40 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9190 0x561b831e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9190 0x561b831e2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:51336/0 (socket says 172.21.15.17:51336) 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:34794/0 (socket says 172.21.15.17:34794) 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 -- 172.21.15.17:0/1538494809 learned_addr learned my addr 172.21.15.17:0/1538494809 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 -- 172.21.15.17:0/1538494809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 msgr2=0x561b831bdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 --2- 172.21.15.17:0/1538494809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 -- 172.21.15.17:0/1538494809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 msgr2=0x561b831be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 --2- 172.21.15.17:0/1538494809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831be0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.323+0000 7faa337fe640 1 -- 172.21.15.17:0/1538494809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b830bb820 con 0x561b831b9190 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.324+0000 7faa337fe640 1 --2- 172.21.15.17:0/1538494809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9190 0x561b831e2010 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x561b831db050 tx=0x7faa2c032bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d05f3d2477d4c047 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.324+0000 7faa28ff9640 1 -- 172.21.15.17:0/1538494809 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c032e30 con 0x561b831b9190 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.324+0000 7faa28ff9640 1 -- 172.21.15.17:0/1538494809 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa2c053040 con 0x561b831b9190 2024-02-14T15:05:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.324+0000 7faa28ff9640 1 -- 172.21.15.17:0/1538494809 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0043d0 con 0x561b831b9190 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.324+0000 7faa3bfe1740 1 -- 172.21.15.17:0/1538494809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9190 msgr2=0x561b831e2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.324+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/1538494809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9190 0x561b831e2010 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x561b831db050 tx=0x7faa2c032bb0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- 172.21.15.17:0/1538494809 shutdown_connections 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/1538494809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/1538494809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9b40 0x561b831bdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/1538494809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9190 0x561b831e2010 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- 172.21.15.17:0/1538494809 >> 172.21.15.17:0/1538494809 conn(0x561b831b3e50 msgr2=0x561b831d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- 172.21.15.17:0/1538494809 shutdown_connections 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- 172.21.15.17:0/1538494809 wait complete. 2024-02-14T15:05:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- start start 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 Processor -- start 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9190 0x561b831cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b83231860 con 0x561b831cb350 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b832319d0 con 0x561b831b9b40 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b83231b90 con 0x561b831b9190 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa3bfe1740 10 client.0 _ll_register_callbacks cb 0x7faa24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9190 0x561b831cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9190 0x561b831cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:34806/0 (socket says 172.21.15.17:34806) 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa337fe640 1 -- 172.21.15.17:0/582733864 learned_addr learned my addr 172.21.15.17:0/582733864 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa32ffd640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:51352/0 (socket says 172.21.15.17:51352) 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.325+0000 7faa337fe640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa337fe640 1 -- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 msgr2=0x561b831cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa337fe640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa337fe640 1 -- 172.21.15.17:0/582733864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 msgr2=0x561b831cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa337fe640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa337fe640 1 -- 172.21.15.17:0/582733864 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b82e31920 con 0x561b831b9190 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa32ffd640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa337fe640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9190 0x561b831cee90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa2c04e040 tx=0x7faa2c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1217ff33028731f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa1a7fc640 1 -- 172.21.15.17:0/582733864 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0328c0 con 0x561b831b9190 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa1a7fc640 1 -- 172.21.15.17:0/582733864 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7faa2c034a90 con 0x561b831b9190 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa2c034690 con 0x561b831b9190 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa1a7fc640 1 -- 172.21.15.17:0/582733864 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa2c032a20 con 0x561b831b9190 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa3bfe1740 10 client.25909 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b830e1720 con 0x561b831b9190 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.326+0000 7faa1a7fc640 1 -- 172.21.15.17:0/582733864 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa2c034690 con 0x561b831b9190 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa1a7fc640 1 -- 172.21.15.17:0/582733864 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faa2c0594c0 con 0x561b831b9190 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa3bfe1740 10 client.25909 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa3bfe1740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa3bfe1740 20 client.25909 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b83232f90 0x561b832353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa3bfe1740 4 client.25909 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:05:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b82e5d550 con 0x561b83232f90 2024-02-14T15:05:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa32ffd640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b83232f90 0x561b832353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa1a7fc640 10 client.25909 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.327+0000 7faa32ffd640 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b83232f90 0x561b832353b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a4ea21dd7e294d server_cookie=b82bd66bab68096 in_seq=0 out_seq=0 2024-02-14T15:05:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.341+0000 7faa1a7fc640 1 -- 172.21.15.17:0/582733864 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b82e5d550 con 0x561b83232f90 2024-02-14T15:05:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.341+0000 7faa1a7fc640 10 client.25909 handle_command_reply: tid=0 2024-02-14T15:05:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.341+0000 7faa3bfe1740 1 client.25909 shutdown 2024-02-14T15:05:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9190 msgr2=0x561b831cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9190 0x561b831cee90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa2c04e040 tx=0x7faa2c005fd0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 shutdown_connections 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561b83232f90 0x561b832353b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561b831cb350 0x561b831cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b831b9b40 0x561b831cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa1a7fc640 0 client.25909 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 --2- 172.21.15.17:0/582733864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b831b9190 0x561b831cee90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 >> 172.21.15.17:0/582733864 conn(0x561b831b3e50 msgr2=0x561b831d6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 shutdown_connections 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 1 -- 172.21.15.17:0/582733864 wait complete. 2024-02-14T15:05:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa3bfe1740 20 client.25909 trim_cache size 0 max 16384 2024-02-14T15:05:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:05:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:05:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:05:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:05:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7faa200a2220 msgr2=0x7faa200a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7faa200a2220 0x7faa200a46e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa1c09ada0 tx=0x7faa1c004a50 comp rx=0 tx=0).stop 2024-02-14T15:05:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 msgr2=0x7faa3419dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3419dbb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00e9c0 tx=0x7faa1c00ee80 comp rx=0 tx=0).stop 2024-02-14T15:05:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 shutdown_connections 2024-02-14T15:05:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7faa200a2220 0x7faa200a46e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa34106530 0x7faa3419dbb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa34197780 0x7faa34197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 --2- 172.21.15.17:0/427188110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa34101460 0x7faa3419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.342+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 >> 172.21.15.17:0/427188110 conn(0x7faa340fcce0 msgr2=0x7faa340ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.343+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 shutdown_connections 2024-02-14T15:05:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:31.343+0000 7faa39415640 1 -- 172.21.15.17:0/427188110 wait complete. 2024-02-14T15:05:31.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 49919 8.74 MB/sec warmup 33 sec latency 104.207 ms 2024-02-14T15:05:31.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:31 smithi076 ceph-mon[37057]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-14T15:05:31.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:31 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:active 2024-02-14T15:05:31.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:31 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:31.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:31 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:05:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:31 smithi017 ceph-mon[32549]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-14T15:05:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:31 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:active 2024-02-14T15:05:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:31 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:32.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:31 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:05:32.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:31 smithi050 ceph-mon[36512]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-14T15:05:32.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:31 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] up:active 2024-02-14T15:05:32.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:31 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:05:32.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:31 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:05:32.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 52266 8.73 MB/sec warmup 34 sec latency 42.045 ms 2024-02-14T15:05:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:32 smithi076 ceph-mon[37057]: pgmap v66: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-02-14T15:05:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:32 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:05:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:32 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:05:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:32 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:05:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:32 smithi017 ceph-mon[32549]: pgmap v66: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-02-14T15:05:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:32 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:05:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:32 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:05:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:32 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:05:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:32 smithi050 ceph-mon[36512]: pgmap v66: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-02-14T15:05:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:32 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:05:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:32 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:05:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:32 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:05:33.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 54638 8.94 MB/sec warmup 35 sec latency 14.080 ms 2024-02-14T15:05:33.917 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:33 smithi017 ceph-mon[32549]: scrub summary: idle 2024-02-14T15:05:33.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:33 smithi076 ceph-mon[37057]: scrub summary: idle 2024-02-14T15:05:34.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:33 smithi050 ceph-mon[36512]: scrub summary: idle 2024-02-14T15:05:34.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 57250 9.08 MB/sec warmup 36 sec latency 58.387 ms 2024-02-14T15:05:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:34 smithi076 ceph-mon[37057]: pgmap v67: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-02-14T15:05:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:34 smithi076 ceph-mon[37057]: scrub summary: idle 2024-02-14T15:05:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:34 smithi017 ceph-mon[32549]: pgmap v67: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-02-14T15:05:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:34 smithi017 ceph-mon[32549]: scrub summary: idle 2024-02-14T15:05:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:34 smithi050 ceph-mon[36512]: pgmap v67: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-02-14T15:05:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:34 smithi050 ceph-mon[36512]: scrub summary: idle 2024-02-14T15:05:35.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 60431 9.32 MB/sec warmup 37 sec latency 9.410 ms 2024-02-14T15:05:36.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 62759 9.29 MB/sec warmup 38 sec latency 41.758 ms 2024-02-14T15:05:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:36 smithi076 ceph-mon[37057]: pgmap v68: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-02-14T15:05:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:36 smithi017 ceph-mon[32549]: pgmap v68: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-02-14T15:05:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:36 smithi050 ceph-mon[36512]: pgmap v68: 129 pgs: 129 active+clean; 42 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-02-14T15:05:37.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 64696 9.38 MB/sec warmup 39 sec latency 64.624 ms 2024-02-14T15:05:38.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 67646 9.60 MB/sec warmup 40 sec latency 38.279 ms 2024-02-14T15:05:38.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:38 smithi076 ceph-mon[37057]: pgmap v69: 129 pgs: 129 active+clean; 57 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 298 op/s 2024-02-14T15:05:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:38 smithi017 ceph-mon[32549]: pgmap v69: 129 pgs: 129 active+clean; 57 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 298 op/s 2024-02-14T15:05:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:38 smithi050 ceph-mon[36512]: pgmap v69: 129 pgs: 129 active+clean; 57 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 298 op/s 2024-02-14T15:05:39.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 70339 9.68 MB/sec warmup 41 sec latency 17.394 ms 2024-02-14T15:05:40.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 72451 9.73 MB/sec warmup 42 sec latency 10.868 ms 2024-02-14T15:05:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:40 smithi076 ceph-mon[37057]: pgmap v70: 129 pgs: 129 active+clean; 57 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-02-14T15:05:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:40 smithi017 ceph-mon[32549]: pgmap v70: 129 pgs: 129 active+clean; 57 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-02-14T15:05:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:40 smithi050 ceph-mon[36512]: pgmap v70: 129 pgs: 129 active+clean; 57 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 208 op/s 2024-02-14T15:05:41.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 75148 9.83 MB/sec warmup 43 sec latency 14.639 ms 2024-02-14T15:05:42.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 78315 10.03 MB/sec warmup 44 sec latency 8.094 ms 2024-02-14T15:05:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:42 smithi076 ceph-mon[37057]: pgmap v71: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-02-14T15:05:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:42 smithi017 ceph-mon[32549]: pgmap v71: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-02-14T15:05:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:42 smithi050 ceph-mon[36512]: pgmap v71: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 326 op/s 2024-02-14T15:05:43.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 80963 10.07 MB/sec warmup 45 sec latency 16.238 ms 2024-02-14T15:05:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 -- 172.21.15.17:0/1353701004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25481024a0 msgr2=0x7f2548102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 --2- 172.21.15.17:0/1353701004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25481024a0 0x7f2548102880 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2538009900 tx=0x7f253802bcd0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 -- 172.21.15.17:0/1353701004 shutdown_connections 2024-02-14T15:05:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 --2- 172.21.15.17:0/1353701004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254810b7e0 0x7f254810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 --2- 172.21.15.17:0/1353701004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2548102dc0 0x7f254810b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 --2- 172.21.15.17:0/1353701004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25481024a0 0x7f2548102880 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 -- 172.21.15.17:0/1353701004 >> 172.21.15.17:0/1353701004 conn(0x7f2548077340 msgr2=0x7f2548077750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 -- 172.21.15.17:0/1353701004 shutdown_connections 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.432+0000 7f254da84640 1 -- 172.21.15.17:0/1353701004 wait complete. 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 Processor -- start 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 -- start start 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25481024a0 0x7f254819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2548102dc0 0x7f254819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254810b7e0 0x7f2548197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2548197c70 con 0x7f2548102dc0 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2548197de0 con 0x7f254810b7e0 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f254da84640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2548197fa0 con 0x7f25481024a0 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f2546ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2548102dc0 0x7f254819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f2546ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2548102dc0 0x7f254819d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40658/0 (socket says 172.21.15.17:40658) 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f2546ffd640 1 -- 172.21.15.17:0/2038134074 learned_addr learned my addr 172.21.15.17:0/2038134074 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f25477fe640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25481024a0 0x7f254819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.433+0000 7f2547fff640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254810b7e0 0x7f2548197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2546ffd640 1 -- 172.21.15.17:0/2038134074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25481024a0 msgr2=0x7f254819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2546ffd640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25481024a0 0x7f254819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2546ffd640 1 -- 172.21.15.17:0/2038134074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254810b7e0 msgr2=0x7f2548197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2546ffd640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254810b7e0 0x7f2548197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2546ffd640 1 -- 172.21.15.17:0/2038134074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538009560 con 0x7f2548102dc0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f25477fe640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25481024a0 0x7f254819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2546ffd640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2548102dc0 0x7f254819d9b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f253c00e9f0 tx=0x7f253c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2544ff9640 1 -- 172.21.15.17:0/2038134074 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c009700 con 0x7f2548102dc0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2548198280 con 0x7f2548102dc0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2544ff9640 1 -- 172.21.15.17:0/2038134074 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f253c00cd80 con 0x7f2548102dc0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f2544ff9640 1 -- 172.21.15.17:0/2038134074 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c018920 con 0x7f2548102dc0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.434+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25481a4510 con 0x7f2548102dc0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.436+0000 7f2544ff9640 1 -- 172.21.15.17:0/2038134074 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f253c018a80 con 0x7f2548102dc0 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 Processor -- start 2024-02-14T15:05:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 -- start start 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9510720 0x5578a9510b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a95196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a951c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a947e9e0 con 0x5578a9511040 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a947bf10 con 0x5578a9510720 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2550655740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a94776f0 con 0x5578a9519c20 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2544ff9640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25300a2220 0x7f25300a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f25477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a95196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f25477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a95196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40662/0 (socket says 172.21.15.17:40662) 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2546ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a951c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f25477fe640 1 -- 172.21.15.17:0/4153716755 learned_addr learned my addr 172.21.15.17:0/4153716755 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2547fff640 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9510720 0x5578a9510b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2547fff640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25300a2220 0x7f25300a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f2544ff9640 1 -- 172.21.15.17:0/2038134074 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f253c014070 con 0x7f2548102dc0 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f25477fe640 1 -- 172.21.15.17:0/4153716755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 msgr2=0x5578a951c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.437+0000 7f25477fe640 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a951c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f25477fe640 1 -- 172.21.15.17:0/4153716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9510720 msgr2=0x5578a9510b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f25477fe640 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9510720 0x5578a9510b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f25477fe640 1 -- 172.21.15.17:0/4153716755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578a9188920 con 0x5578a9511040 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f2547fff640 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9510720 0x5578a9510b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f2546ffd640 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a951c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f25477fe640 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a95196e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2538009900 tx=0x7f253802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c10ffceafd1f1d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f2547fff640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25300a2220 0x7f25300a46e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2540005fd0 tx=0x7f2540009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f252dffb640 1 -- 172.21.15.17:0/4153716755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253802c880 con 0x5578a9511040 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f252dffb640 1 -- 172.21.15.17:0/4153716755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f253802c9e0 con 0x5578a9511040 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.438+0000 7f252dffb640 1 -- 172.21.15.17:0/4153716755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380054f0 con 0x5578a9511040 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- 172.21.15.17:0/4153716755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 msgr2=0x5578a95196e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a95196e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2538009900 tx=0x7f253802c720 comp rx=0 tx=0).stop 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- 172.21.15.17:0/4153716755 shutdown_connections 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a95196e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a951c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 --2- 172.21.15.17:0/4153716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9510720 0x5578a9510b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- 172.21.15.17:0/4153716755 >> 172.21.15.17:0/4153716755 conn(0x5578a950ae50 msgr2=0x5578a950b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- 172.21.15.17:0/4153716755 shutdown_connections 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- 172.21.15.17:0/4153716755 wait complete. 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- start start 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 Processor -- start 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a95195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9516950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9519c20 0x5578a9516ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a9413fa0 con 0x5578a9519c20 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a914b250 con 0x5578a9511040 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a93fd200 con 0x5578a9510720 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.439+0000 7f2550655740 10 client.0 _ll_register_callbacks cb 0x5578a946cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9519c20 0x5578a9516ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f25477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a95195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9519c20 0x5578a9516ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40666/0 (socket says 172.21.15.17:40666) 2024-02-14T15:05:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 -- 172.21.15.17:0/4199808912 learned_addr learned my addr 172.21.15.17:0/4199808912 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2546ffd640 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9516950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 -- 172.21.15.17:0/4199808912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 msgr2=0x5578a95195c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a95195c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 -- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 msgr2=0x5578a9516950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9516950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 -- 172.21.15.17:0/4199808912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254810e900 con 0x5578a9519c20 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f25477fe640 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a95195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2547fff640 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9519c20 0x5578a9516ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2540011840 tx=0x7f2540004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd79c5f415924da server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2526ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2526ffd640 1 -- 172.21.15.17:0/4199808912 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540020db0 con 0x5578a9519c20 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2526ffd640 1 -- 172.21.15.17:0/4199808912 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f254001e410 con 0x5578a9519c20 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578a9177600 con 0x5578a9519c20 2024-02-14T15:05:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2526ffd640 1 -- 172.21.15.17:0/4199808912 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2540030670 con 0x5578a9519c20 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2550655740 10 client.16392 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.440+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5578a9475d70 con 0x5578a9519c20 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2526ffd640 1 -- 172.21.15.17:0/4199808912 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5578a9177600 con 0x5578a9519c20 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2526ffd640 1 -- 172.21.15.17:0/4199808912 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f254001e5d0 con 0x5578a9519c20 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2550655740 10 client.16392 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2550655740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2550655740 20 client.16392 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2550655740 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9529960 0x5578a952bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2550655740 4 client.16392 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578a91c81b0 con 0x5578a9529960 2024-02-14T15:05:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.441+0000 7f25477fe640 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9529960 0x5578a952bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f25477fe640 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9529960 0x5578a952bd80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aa32a8f11f664a server_cookie=30a1bc785232d984 in_seq=0 out_seq=0 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2526ffd640 10 client.16392 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2526ffd640 1 -- 172.21.15.17:0/4199808912 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578a91c81b0 con 0x5578a9529960 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2526ffd640 10 client.16392 handle_command_reply: tid=0 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 client.16392 shutdown 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9519c20 msgr2=0x5578a9516ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9519c20 0x5578a9516ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2540011840 tx=0x7f2540004ab0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 shutdown_connections 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9529960 0x5578a952bd80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9519c20 0x5578a9516ef0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9516950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 --2- 172.21.15.17:0/4199808912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a95195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2526ffd640 0 client.16392 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.442+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 >> 172.21.15.17:0/4199808912 conn(0x5578a950ae50 msgr2=0x5578a950cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.443+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 shutdown_connections 2024-02-14T15:05:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.443+0000 7f2550655740 1 -- 172.21.15.17:0/4199808912 wait complete. 2024-02-14T15:05:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.443+0000 7f2550655740 20 client.16392 trim_cache size 0 max 16384 2024-02-14T15:05:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 Processor -- start 2024-02-14T15:05:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 -- start start 2024-02-14T15:05:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9510720 0x5578a950c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9519080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a95195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a9526330 con 0x5578a9510720 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a9528070 con 0x5578a9511040 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2550655740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a950bbe0 con 0x5578a9519c20 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9510720 0x5578a950c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9510720 0x5578a950c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40674/0 (socket says 172.21.15.17:40674) 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2547fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a95195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 -- 172.21.15.17:0/3860474252 learned_addr learned my addr 172.21.15.17:0/3860474252 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f2546ffd640 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9519080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 -- 172.21.15.17:0/3860474252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 msgr2=0x5578a95195c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a95195c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 -- 172.21.15.17:0/3860474252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 msgr2=0x5578a9519080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9519080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 -- 172.21.15.17:0/3860474252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578a9412820 con 0x5578a9510720 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.455+0000 7f25477fe640 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9510720 0x5578a950c990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2538006010 tx=0x7f2538004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a6673e00f4aad server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f252d7fa640 1 -- 172.21.15.17:0/3860474252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253802c880 con 0x5578a9510720 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f252d7fa640 1 -- 172.21.15.17:0/3860474252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2538052070 con 0x5578a9510720 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f252d7fa640 1 -- 172.21.15.17:0/3860474252 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538004af0 con 0x5578a9510720 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 -- 172.21.15.17:0/3860474252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9510720 msgr2=0x5578a950c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9510720 0x5578a950c990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2538006010 tx=0x7f2538004df0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 -- 172.21.15.17:0/3860474252 shutdown_connections 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9519c20 0x5578a95195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9511040 0x5578a9519080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 --2- 172.21.15.17:0/3860474252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9510720 0x5578a950c990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 -- 172.21.15.17:0/3860474252 >> 172.21.15.17:0/3860474252 conn(0x5578a950ae50 msgr2=0x5578a950c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 -- 172.21.15.17:0/3860474252 shutdown_connections 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.456+0000 7f2550655740 1 -- 172.21.15.17:0/3860474252 wait complete. 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 -- start start 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 Processor -- start 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a9520ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a951eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9519c20 0x5578a951f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a951f920 con 0x5578a9511040 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a951fa90 con 0x5578a9519c20 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578a951fc50 con 0x5578a9510720 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2550655740 10 client.0 _ll_register_callbacks cb 0x7f25400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a951eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a951eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40684/0 (socket says 172.21.15.17:40684) 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 -- 172.21.15.17:0/1495574945 learned_addr learned my addr 172.21.15.17:0/1495574945 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f25477fe640 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a9520ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2547fff640 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9519c20 0x5578a951f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 -- 172.21.15.17:0/1495574945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 msgr2=0x5578a9520ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a9520ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 -- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9519c20 msgr2=0x5578a951f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9519c20 0x5578a951f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.457+0000 7f2546ffd640 1 -- 172.21.15.17:0/1495574945 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578a9188920 con 0x5578a9511040 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2546ffd640 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a951eea0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f253c010f00 tx=0x7f253c06a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8fc34f5e81d908 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2526ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2526ffd640 1 -- 172.21.15.17:0/1495574945 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c09d070 con 0x5578a9511040 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2526ffd640 1 -- 172.21.15.17:0/1495574945 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f253c06ae40 con 0x5578a9511040 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f253802c9e0 con 0x5578a9511040 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2526ffd640 1 -- 172.21.15.17:0/1495574945 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f253c097520 con 0x5578a9511040 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 10 client.16398 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5578a9438720 con 0x5578a9511040 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2526ffd640 1 -- 172.21.15.17:0/1495574945 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f253802c9e0 con 0x5578a9511040 2024-02-14T15:05:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2526ffd640 1 -- 172.21.15.17:0/1495574945 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f253c0cd070 con 0x5578a9511040 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 10 client.16398 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 20 client.16398 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9581350 0x5578a9583770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 4 client.16398 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578a91b4550 con 0x5578a9581350 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.458+0000 7f25477fe640 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9581350 0x5578a9583770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.459+0000 7f25477fe640 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9581350 0x5578a9583770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe8d5757e8b9618 server_cookie=5353fb3d8fd87459 in_seq=0 out_seq=0 2024-02-14T15:05:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.459+0000 7f2526ffd640 10 client.16398 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.468+0000 7f2526ffd640 1 -- 172.21.15.17:0/1495574945 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578a91b4550 con 0x5578a9581350 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.468+0000 7f2526ffd640 10 client.16398 handle_command_reply: tid=0 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.468+0000 7f2550655740 1 client.16398 shutdown 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 msgr2=0x5578a951eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a951eea0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f253c010f00 tx=0x7f253c06a110 comp rx=0 tx=0).stop 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 shutdown_connections 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5578a9581350 0x5578a9583770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5578a9511040 0x5578a951eea0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5578a9519c20 0x5578a951f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2526ffd640 0 client.16398 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 --2- 172.21.15.17:0/1495574945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578a9510720 0x5578a9520ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 >> 172.21.15.17:0/1495574945 conn(0x5578a950ae50 msgr2=0x5578a9509ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 shutdown_connections 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 1 -- 172.21.15.17:0/1495574945 wait complete. 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f2550655740 20 client.16398 trim_cache size 0 max 16384 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:05:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:05:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25300a2220 msgr2=0x7f25300a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25300a2220 0x7f25300a46e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2540005fd0 tx=0x7f2540009500 comp rx=0 tx=0).stop 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2548102dc0 msgr2=0x7f254819d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2548102dc0 0x7f254819d9b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f253c00e9f0 tx=0x7f253c00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 shutdown_connections 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f25300a2220 0x7f25300a46e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2548102dc0 0x7f254819d9b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254810b7e0 0x7f2548197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 --2- 172.21.15.17:0/2038134074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25481024a0 0x7f254819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.469+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 >> 172.21.15.17:0/2038134074 conn(0x7f2548077340 msgr2=0x7f25480fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.470+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 shutdown_connections 2024-02-14T15:05:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:44.470+0000 7f254da84640 1 -- 172.21.15.17:0/2038134074 wait complete. 2024-02-14T15:05:44.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 82822 10.06 MB/sec warmup 46 sec latency 15.768 ms 2024-02-14T15:05:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:44 smithi076 ceph-mon[37057]: pgmap v72: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-02-14T15:05:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:44 smithi017 ceph-mon[32549]: pgmap v72: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-02-14T15:05:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:44 smithi050 ceph-mon[36512]: pgmap v72: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-02-14T15:05:45.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 85712 10.21 MB/sec warmup 47 sec latency 8.278 ms 2024-02-14T15:05:46.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 88840 10.37 MB/sec warmup 48 sec latency 7.705 ms 2024-02-14T15:05:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:46 smithi076 ceph-mon[37057]: pgmap v73: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-02-14T15:05:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:46 smithi017 ceph-mon[32549]: pgmap v73: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-02-14T15:05:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:46 smithi050 ceph-mon[36512]: pgmap v73: 129 pgs: 129 active+clean; 54 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-02-14T15:05:47.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 91217 10.34 MB/sec warmup 49 sec latency 18.381 ms 2024-02-14T15:05:48.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 93248 10.38 MB/sec warmup 50 sec latency 29.749 ms 2024-02-14T15:05:48.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:48 smithi076 ceph-mon[37057]: pgmap v74: 129 pgs: 129 active+clean; 61 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-02-14T15:05:49.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:48 smithi017 ceph-mon[32549]: pgmap v74: 129 pgs: 129 active+clean; 61 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-02-14T15:05:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:48 smithi050 ceph-mon[36512]: pgmap v74: 129 pgs: 129 active+clean; 61 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-02-14T15:05:49.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 95932 10.51 MB/sec warmup 51 sec latency 188.893 ms 2024-02-14T15:05:50.861 INFO:tasks.workunit.client.0.smithi017.stdout: 1 98206 10.47 MB/sec warmup 52 sec latency 26.084 ms 2024-02-14T15:05:50.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:50 smithi076 ceph-mon[37057]: pgmap v75: 129 pgs: 129 active+clean; 61 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-02-14T15:05:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:50 smithi017 ceph-mon[32549]: pgmap v75: 129 pgs: 129 active+clean; 61 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-02-14T15:05:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:50 smithi050 ceph-mon[36512]: pgmap v75: 129 pgs: 129 active+clean; 61 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-02-14T15:05:51.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 100341 10.52 MB/sec warmup 53 sec latency 11.904 ms 2024-02-14T15:05:52.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 103331 10.66 MB/sec warmup 54 sec latency 22.116 ms 2024-02-14T15:05:52.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:52 smithi076 ceph-mon[37057]: pgmap v76: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-02-14T15:05:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:52 smithi017 ceph-mon[32549]: pgmap v76: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-02-14T15:05:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:52 smithi050 ceph-mon[36512]: pgmap v76: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-02-14T15:05:53.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 106446 10.78 MB/sec warmup 55 sec latency 6.628 ms 2024-02-14T15:05:54.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 108118 10.71 MB/sec warmup 56 sec latency 94.552 ms 2024-02-14T15:05:54.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:54 smithi050 ceph-mon[36512]: pgmap v77: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-02-14T15:05:54.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:54 smithi076 ceph-mon[37057]: pgmap v77: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-02-14T15:05:55.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:54 smithi017 ceph-mon[32549]: pgmap v77: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-02-14T15:05:55.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 110432 10.77 MB/sec warmup 57 sec latency 60.864 ms 2024-02-14T15:05:56.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-14T15:05:56.680 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-14T15:05:56.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.789+0000 7f6476823640 1 -- 172.21.15.17:0/3274032875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470101df0 msgr2=0x7f6470105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.789+0000 7f6476823640 1 --2- 172.21.15.17:0/3274032875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470101df0 0x7f6470105e30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6464009930 tx=0x7f646402bc30 comp rx=0 tx=0).stop 2024-02-14T15:05:56.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 -- 172.21.15.17:0/3274032875 shutdown_connections 2024-02-14T15:05:56.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 --2- 172.21.15.17:0/3274032875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470101df0 0x7f6470105e30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 --2- 172.21.15.17:0/3274032875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 0x7f647010e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 --2- 172.21.15.17:0/3274032875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6470101440 0x7f6470101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 -- 172.21.15.17:0/3274032875 >> 172.21.15.17:0/3274032875 conn(0x7f64700fccc0 msgr2=0x7f64700ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:56.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 -- 172.21.15.17:0/3274032875 shutdown_connections 2024-02-14T15:05:56.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 -- 172.21.15.17:0/3274032875 wait complete. 2024-02-14T15:05:56.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 Processor -- start 2024-02-14T15:05:56.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.790+0000 7f6476823640 1 -- start start 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f6476823640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6470101440 0x7f6470073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f6476823640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 0x7f6470073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f6476823640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470073fb0 0x7f64701a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f6476823640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a6030 con 0x7f6470073fb0 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f6476823640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a61a0 con 0x7f6470101440 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f6476823640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a6360 con 0x7f6470106510 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 0x7f6470073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 0x7f6470073a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33094/0 (socket says 172.21.15.17:33094) 2024-02-14T15:05:56.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.791+0000 7f646f7fe640 1 -- 172.21.15.17:0/3941604724 learned_addr learned my addr 172.21.15.17:0/3941604724 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f6474dc0640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470073fb0 0x7f64701a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f646f7fe640 1 -- 172.21.15.17:0/3941604724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6470101440 msgr2=0x7f6470073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f646f7fe640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6470101440 0x7f6470073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f646f7fe640 1 -- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470073fb0 msgr2=0x7f64701a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f646f7fe640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470073fb0 0x7f64701a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f646f7fe640 1 -- 172.21.15.17:0/3941604724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464009560 con 0x7f6470106510 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f6474dc0640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470073fb0 0x7f64701a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.792+0000 7f646f7fe640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 0x7f6470073a70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6464009230 tx=0x7f646402c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.793+0000 7f646d7fa640 1 -- 172.21.15.17:0/3941604724 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464004350 con 0x7f6470106510 2024-02-14T15:05:56.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.793+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64701a65e0 con 0x7f6470106510 2024-02-14T15:05:56.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.793+0000 7f646d7fa640 1 -- 172.21.15.17:0/3941604724 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64640044b0 con 0x7f6470106510 2024-02-14T15:05:56.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.793+0000 7f646d7fa640 1 -- 172.21.15.17:0/3941604724 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464003d30 con 0x7f6470106510 2024-02-14T15:05:56.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.793+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64701a6a30 con 0x7f6470106510 2024-02-14T15:05:56.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 Processor -- start 2024-02-14T15:05:56.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 -- start start 2024-02-14T15:05:56.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f646d7fa640 1 -- 172.21.15.17:0/3941604724 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f646403e450 con 0x7f6470106510 2024-02-14T15:05:56.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40edf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e409d5b0 con 0x5627e40ebb30 2024-02-14T15:05:56.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e40512b0 con 0x5627e40e2f50 2024-02-14T15:05:56.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.797+0000 7f64793f2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e404e7e0 con 0x5627e40e2630 2024-02-14T15:05:56.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40edf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39788/0 (socket says 172.21.15.17:39788) 2024-02-14T15:05:56.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 -- 172.21.15.17:0/158451610 learned_addr learned my addr 172.21.15.17:0/158451610 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:56.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f6474dc0640 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646ffff640 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646d7fa640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64580a2330 0x7f64580a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 -- 172.21.15.17:0/158451610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 msgr2=0x5627e40e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 -- 172.21.15.17:0/158451610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 msgr2=0x5627e40eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646f7fe640 1 -- 172.21.15.17:0/158451610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e400c020 con 0x5627e40ebb30 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f6474dc0640 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.798+0000 7f646d7fa640 1 -- 172.21.15.17:0/3941604724 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f64640f0210 con 0x7f6470106510 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.799+0000 7f646f7fe640 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40edf20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f64640ed5f0 tx=0x7f646404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1344a7d1b3bc205 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.799+0000 7f6474dc0640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64580a2330 0x7f64580a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.799+0000 7f64567fc640 1 -- 172.21.15.17:0/158451610 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464003e90 con 0x5627e40ebb30 2024-02-14T15:05:56.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.799+0000 7f64567fc640 1 -- 172.21.15.17:0/158451610 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64640bc870 con 0x5627e40ebb30 2024-02-14T15:05:56.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.800+0000 7f64567fc640 1 -- 172.21.15.17:0/158451610 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640f3cf0 con 0x5627e40ebb30 2024-02-14T15:05:56.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.800+0000 7f64793f2740 1 -- 172.21.15.17:0/158451610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 msgr2=0x5627e40edf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.800+0000 7f64793f2740 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40edf20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f64640ed5f0 tx=0x7f646404c000 comp rx=0 tx=0).stop 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f6474dc0640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64580a2330 0x7f64580a47f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f64680094d0 tx=0x7f6468014040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 -- 172.21.15.17:0/158451610 shutdown_connections 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40edf20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 --2- 172.21.15.17:0/158451610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 -- 172.21.15.17:0/158451610 >> 172.21.15.17:0/158451610 conn(0x5627e40dcd60 msgr2=0x5627e40dd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 -- 172.21.15.17:0/158451610 shutdown_connections 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 -- 172.21.15.17:0/158451610 wait complete. 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.801+0000 7f64793f2740 1 -- start start 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 1 Processor -- start 2024-02-14T15:05:56.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4049fc0 con 0x5627e40ebb30 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e3d1d430 con 0x5627e40e2f50 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e3cd9e00 con 0x5627e40e2630 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f64793f2740 10 client.0 _ll_register_callbacks cb 0x5627e3fcb870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57018/0 (socket says 172.21.15.17:57018) 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.802+0000 7f646f7fe640 1 -- 172.21.15.17:0/138446401 learned_addr learned my addr 172.21.15.17:0/138446401 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f646ffff640 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f6474dc0640 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f646f7fe640 1 -- 172.21.15.17:0/138446401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 msgr2=0x5627e40e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f646f7fe640 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f646f7fe640 1 -- 172.21.15.17:0/138446401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 msgr2=0x5627e40e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f646f7fe640 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f646f7fe640 1 -- 172.21.15.17:0/138446401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64701a6a30 con 0x5627e40e2f50 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f646f7fe640 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e8f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f64640f0500 tx=0x7f646404d6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717d91d68feea829 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.803+0000 7f644f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.804+0000 7f644f7fe640 1 -- 172.21.15.17:0/138446401 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464003e90 con 0x5627e40e2f50 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.804+0000 7f644f7fe640 1 -- 172.21.15.17:0/138446401 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f64640035f0 con 0x5627e40e2f50 2024-02-14T15:05:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.804+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627e3d497e0 con 0x5627e40e2f50 2024-02-14T15:05:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.806+0000 7f644f7fe640 1 -- 172.21.15.17:0/138446401 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64640f3cf0 con 0x5627e40e2f50 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.806+0000 7f64793f2740 10 client.25811 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.806+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5627e4048640 con 0x5627e40e2f50 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.807+0000 7f644f7fe640 1 -- 172.21.15.17:0/138446401 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64640bc5e0 con 0x5627e40e2f50 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.807+0000 7f644f7fe640 1 -- 172.21.15.17:0/138446401 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f646403c070 con 0x5627e40e2f50 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.807+0000 7f64793f2740 10 client.25811 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.808+0000 7f64793f2740 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.808+0000 7f64793f2740 20 client.25811 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.808+0000 7f64793f2740 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e4103fa0 0x5627e4106400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.808+0000 7f64793f2740 4 client.25811 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.808+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627e3d497e0 con 0x5627e4103fa0 2024-02-14T15:05:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.808+0000 7f646ffff640 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e4103fa0 0x5627e4106400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.809+0000 7f644f7fe640 10 client.25811 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.809+0000 7f646ffff640 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e4103fa0 0x5627e4106400 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18db152f4b2bc961 server_cookie=3859b1d92ff07279 in_seq=0 out_seq=0 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f644f7fe640 1 -- 172.21.15.17:0/138446401 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627e3d497e0 con 0x5627e4103fa0 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f644f7fe640 10 client.25811 handle_command_reply: tid=0 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 client.25811 shutdown 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 msgr2=0x5627e40e8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e8f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f64640f0500 tx=0x7f646404d6f0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 shutdown_connections 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e4103fa0 0x5627e4106400 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e8f80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 --2- 172.21.15.17:0/138446401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 >> 172.21.15.17:0/138446401 conn(0x5627e40dcd60 msgr2=0x5627e40deab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.810+0000 7f644f7fe640 0 client.25811 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.811+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 shutdown_connections 2024-02-14T15:05:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.811+0000 7f64793f2740 1 -- 172.21.15.17:0/138446401 wait complete. 2024-02-14T15:05:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.811+0000 7f64793f2740 20 client.25811 trim_cache size 0 max 16384 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 Processor -- start 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 -- start start 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e3fcefd0 con 0x5627e40ebb30 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e40d8490 con 0x5627e40e2f50 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f64793f2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e40e2290 con 0x5627e40e2630 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f6474dc0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f646ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57020/0 (socket says 172.21.15.17:57020) 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f6474dc0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39804/0 (socket says 172.21.15.17:39804) 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f646f7fe640 1 -- 172.21.15.17:0/3985457480 learned_addr learned my addr 172.21.15.17:0/3985457480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.827+0000 7f646ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40eb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33126/0 (socket says 172.21.15.17:33126) 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f646f7fe640 1 -- 172.21.15.17:0/3985457480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 msgr2=0x5627e40eb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f646f7fe640 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40eb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f646f7fe640 1 -- 172.21.15.17:0/3985457480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 msgr2=0x5627e40e5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f646f7fe640 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f646f7fe640 1 -- 172.21.15.17:0/3985457480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e3fe4b30 con 0x5627e40e2f50 2024-02-14T15:05:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f6474dc0640 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f646f7fe640 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e7a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f646402cde0 tx=0x7f64640d0870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83c4746d9e95dd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f6455ffb640 1 -- 172.21.15.17:0/3985457480 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464003e90 con 0x5627e40e2f50 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f6455ffb640 1 -- 172.21.15.17:0/3985457480 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64640b87f0 con 0x5627e40e2f50 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.828+0000 7f6455ffb640 1 -- 172.21.15.17:0/3985457480 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640c50f0 con 0x5627e40e2f50 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 -- 172.21.15.17:0/3985457480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 msgr2=0x5627e40e7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e7a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f646402cde0 tx=0x7f64640d0870 comp rx=0 tx=0).stop 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 -- 172.21.15.17:0/3985457480 shutdown_connections 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2f50 0x5627e40e7a20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 --2- 172.21.15.17:0/3985457480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2630 0x5627e40eb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 -- 172.21.15.17:0/3985457480 >> 172.21.15.17:0/3985457480 conn(0x5627e40dcd60 msgr2=0x5627e40e7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 -- 172.21.15.17:0/3985457480 shutdown_connections 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 -- 172.21.15.17:0/3985457480 wait complete. 2024-02-14T15:05:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.829+0000 7f64793f2740 1 -- start start 2024-02-14T15:05:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 1 Processor -- start 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2630 0x5627e40fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2f50 0x5627e40f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e40fa300 con 0x5627e40ebb30 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e40fa470 con 0x5627e40e2630 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e415a4f0 con 0x5627e40e2f50 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f64793f2740 10 client.0 _ll_register_callbacks cb 0x7f64640f3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f646ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2630 0x5627e40fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39816/0 (socket says 172.21.15.17:39816) 2024-02-14T15:05:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 -- 172.21.15.17:0/531682402 learned_addr learned my addr 172.21.15.17:0/531682402 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f646f7fe640 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2f50 0x5627e40f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 -- 172.21.15.17:0/531682402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2f50 msgr2=0x5627e40f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2f50 0x5627e40f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 -- 172.21.15.17:0/531682402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2630 msgr2=0x5627e40fb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2630 0x5627e40fb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 -- 172.21.15.17:0/531682402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6468012180 con 0x5627e40ebb30 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f6474dc0640 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40f9c00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6468012d60 tx=0x7f6468009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3aef336f70b3efe server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.830+0000 7f644f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f644f7fe640 1 -- 172.21.15.17:0/531682402 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468023070 con 0x5627e40ebb30 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f644f7fe640 1 -- 172.21.15.17:0/531682402 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6468010dc0 con 0x5627e40ebb30 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64640bc870 con 0x5627e40ebb30 2024-02-14T15:05:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f644f7fe640 1 -- 172.21.15.17:0/531682402 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6468029460 con 0x5627e40ebb30 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 10 client.16416 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5627e4048640 con 0x5627e40ebb30 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f644f7fe640 1 -- 172.21.15.17:0/531682402 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64640bc870 con 0x5627e40ebb30 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f644f7fe640 1 -- 172.21.15.17:0/531682402 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6468031070 con 0x5627e40ebb30 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 10 client.16416 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 20 client.16416 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e415bda0 0x5627e415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 4 client.16416 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5627e3d86720 con 0x5627e415bda0 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.831+0000 7f646ffff640 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e415bda0 0x5627e415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f646ffff640 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e415bda0 0x5627e415e1c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7acc6e3a1b831c9 server_cookie=6e878c0d4fdb1e06 in_seq=0 out_seq=0 2024-02-14T15:05:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f644f7fe640 10 client.16416 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f644f7fe640 1 -- 172.21.15.17:0/531682402 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5627e3d86720 con 0x5627e415bda0 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f644f7fe640 10 client.16416 handle_command_reply: tid=0 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 client.16416 shutdown 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 msgr2=0x5627e40f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40f9c00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6468012d60 tx=0x7f6468009500 comp rx=0 tx=0).stop 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 shutdown_connections 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627e415bda0 0x5627e415e1c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627e40ebb30 0x5627e40f9c00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627e40e2f50 0x5627e40f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f644f7fe640 0 client.16416 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 --2- 172.21.15.17:0/531682402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627e40e2630 0x5627e40fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 >> 172.21.15.17:0/531682402 conn(0x5627e40dcd60 msgr2=0x5627e40e7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 shutdown_connections 2024-02-14T15:05:56.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 1 -- 172.21.15.17:0/531682402 wait complete. 2024-02-14T15:05:56.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.832+0000 7f64793f2740 20 client.16416 trim_cache size 0 max 16384 2024-02-14T15:05:56.842 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:05:56.842 INFO:teuthology.orchestra.run.smithi017.stdout: "status": "no active scrubs running", 2024-02-14T15:05:56.842 INFO:teuthology.orchestra.run.smithi017.stdout: "scrubs": {} 2024-02-14T15:05:56.842 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:05:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64580a2330 msgr2=0x7f64580a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64580a2330 0x7f64580a47f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f64680094d0 tx=0x7f6468014040 comp rx=0 tx=0).stop 2024-02-14T15:05:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 msgr2=0x7f6470073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 0x7f6470073a70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6464009230 tx=0x7f646402c680 comp rx=0 tx=0).stop 2024-02-14T15:05:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 shutdown_connections 2024-02-14T15:05:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f64580a2330 0x7f64580a47f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6470073fb0 0x7f64701a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6470106510 0x7f6470073a70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 --2- 172.21.15.17:0/3941604724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6470101440 0x7f6470073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 >> 172.21.15.17:0/3941604724 conn(0x7f64700fccc0 msgr2=0x7f64700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 shutdown_connections 2024-02-14T15:05:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.833+0000 7f6476823640 1 -- 172.21.15.17:0/3941604724 wait complete. 2024-02-14T15:05:56.845 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-14T15:05:56.845 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:05:56.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 113533 10.88 MB/sec warmup 58 sec latency 11.847 ms 2024-02-14T15:05:56.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2520148311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 msgr2=0x7ffa4810e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2520148311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa4810e0e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009cd0 tx=0x7ffa4002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2520148311 shutdown_connections 2024-02-14T15:05:56.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2520148311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa4810e0e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2520148311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa48101df0 0x7ffa48105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2520148311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa48101440 0x7ffa48101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2520148311 >> 172.21.15.17:0/2520148311 conn(0x7ffa480fcce0 msgr2=0x7ffa480ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2520148311 shutdown_connections 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.949+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2520148311 wait complete. 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 Processor -- start 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 -- start start 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa48101440 0x7ffa48073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa48073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa48073fb0 0x7ffa481a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa481a6010 con 0x7ffa48106510 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa481a6180 con 0x7ffa48073fb0 2024-02-14T15:05:56.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4e45c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa481a6340 con 0x7ffa48101440 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa48073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa48073a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39834/0 (socket says 172.21.15.17:39834) 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa48101440 0x7ffa48073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa477fe640 1 -- 172.21.15.17:0/2005640829 learned_addr learned my addr 172.21.15.17:0/2005640829 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.950+0000 7ffa4c9f9640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa48073fb0 0x7ffa481a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa477fe640 1 -- 172.21.15.17:0/2005640829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa48101440 msgr2=0x7ffa48073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa477fe640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa48101440 0x7ffa48073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa477fe640 1 -- 172.21.15.17:0/2005640829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa48073fb0 msgr2=0x7ffa481a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa477fe640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa48073fb0 0x7ffa481a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa477fe640 1 -- 172.21.15.17:0/2005640829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40009900 con 0x7ffa48106510 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa47fff640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa48101440 0x7ffa48073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa477fe640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa48073a70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9f0 tx=0x7ffa3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa457fa640 1 -- 172.21.15.17:0/2005640829 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0096c0 con 0x7ffa48106510 2024-02-14T15:05:56.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa457fa640 1 -- 172.21.15.17:0/2005640829 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa3c005500 con 0x7ffa48106510 2024-02-14T15:05:56.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa481a6620 con 0x7ffa48106510 2024-02-14T15:05:56.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa457fa640 1 -- 172.21.15.17:0/2005640829 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c010430 con 0x7ffa48106510 2024-02-14T15:05:56.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.951+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa481a6ae0 con 0x7ffa48106510 2024-02-14T15:05:56.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.952+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa48068500 con 0x7ffa48106510 2024-02-14T15:05:56.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.953+0000 7ffa457fa640 1 -- 172.21.15.17:0/2005640829 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ffa3c005020 con 0x7ffa48106510 2024-02-14T15:05:56.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.954+0000 7ffa457fa640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ffa300a2260 0x7ffa300a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:56.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.954+0000 7ffa47fff640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ffa300a2260 0x7ffa300a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:56.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.954+0000 7ffa457fa640 1 -- 172.21.15.17:0/2005640829 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7ffa3c014070 con 0x7ffa48106510 2024-02-14T15:05:56.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.954+0000 7ffa47fff640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ffa300a2260 0x7ffa300a4720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009fc0 tx=0x7ffa38005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:56.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:56.959+0000 7ffa457fa640 1 -- 172.21.15.17:0/2005640829 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa3c096fd0 con 0x7ffa48106510 2024-02-14T15:05:56.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:56 smithi076 ceph-mon[37057]: pgmap v78: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-02-14T15:05:57.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:56 smithi017 ceph-mon[32549]: pgmap v78: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-02-14T15:05:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:56 smithi050 ceph-mon[36512]: pgmap v78: 129 pgs: 129 active+clean; 67 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2024-02-14T15:05:57.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.179+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa48074a60 con 0x7ffa48106510 2024-02-14T15:05:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.180+0000 7ffa457fa640 1 -- 172.21.15.17:0/2005640829 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11320 (secure 0 0 0) 0x7ffa3c09a690 con 0x7ffa48106510 2024-02-14T15:05:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:05:57.182 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/580745027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":580745027},{"type":"v1","addr":"172.21.15.76:6833","nonce":580745027}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:05:30.605802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6837/1706654420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":1706654420},{"type":"v1","addr":"172.21.15.50:6837","nonce":1706654420}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":77,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:05:57.183 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2024-02-14T15:05:57.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ffa300a2260 msgr2=0x7ffa300a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ffa300a2260 0x7ffa300a4720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009fc0 tx=0x7ffa38005f90 comp rx=0 tx=0).stop 2024-02-14T15:05:57.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 msgr2=0x7ffa48073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa48073a70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9f0 tx=0x7ffa3c00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 shutdown_connections 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ffa300a2260 0x7ffa300a4720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa48106510 0x7ffa48073a70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa48073fb0 0x7ffa481a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 --2- 172.21.15.17:0/2005640829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa48101440 0x7ffa48073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 >> 172.21.15.17:0/2005640829 conn(0x7ffa480fcce0 msgr2=0x7ffa480ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 shutdown_connections 2024-02-14T15:05:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.182+0000 7ffa4e45c640 1 -- 172.21.15.17:0/2005640829 wait complete. 2024-02-14T15:05:57.195 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:05:57.195 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-14T15:05:57.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 -- 172.21.15.17:0/2780765337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258100cf0 msgr2=0x7f2258104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 --2- 172.21.15.17:0/2780765337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258100cf0 0x7f2258104d30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2248009930 tx=0x7f224802bc30 comp rx=0 tx=0).stop 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 -- 172.21.15.17:0/2780765337 shutdown_connections 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 --2- 172.21.15.17:0/2780765337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258100cf0 0x7f2258104d30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 --2- 172.21.15.17:0/2780765337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2258105410 0x7f225810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 --2- 172.21.15.17:0/2780765337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2258100340 0x7f2258100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 -- 172.21.15.17:0/2780765337 >> 172.21.15.17:0/2780765337 conn(0x7f22580772e0 msgr2=0x7f22580776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 -- 172.21.15.17:0/2780765337 shutdown_connections 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 -- 172.21.15.17:0/2780765337 wait complete. 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 Processor -- start 2024-02-14T15:05:57.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.341+0000 7f225f459640 1 -- start start 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225f459640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2258100340 0x7f22581986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225f459640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258105410 0x7f2258198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225f459640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22581991c0 0x7f22581a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225f459640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22581a3e10 con 0x7f2258105410 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225f459640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22581a3f80 con 0x7f22581991c0 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225f459640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22581a4140 con 0x7f2258100340 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258105410 0x7f2258198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258105410 0x7f2258198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39858/0 (socket says 172.21.15.17:39858) 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 -- 172.21.15.17:0/4051674675 learned_addr learned my addr 172.21.15.17:0/4051674675 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225d1f5640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2258100340 0x7f22581986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225d9f6640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22581991c0 0x7f22581a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 -- 172.21.15.17:0/4051674675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2258100340 msgr2=0x7f22581986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2258100340 0x7f22581986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 -- 172.21.15.17:0/4051674675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22581991c0 msgr2=0x7f22581a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22581991c0 0x7f22581a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 -- 172.21.15.17:0/4051674675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2248009560 con 0x7f2258105410 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f225c9f4640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258105410 0x7f2258198c30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f2248004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.342+0000 7f224e7fc640 1 -- 172.21.15.17:0/4051674675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224802cbc0 con 0x7f2258105410 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.343+0000 7f224e7fc640 1 -- 172.21.15.17:0/4051674675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22480052d0 con 0x7f2258105410 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.343+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22581a43c0 con 0x7f2258105410 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.343+0000 7f224e7fc640 1 -- 172.21.15.17:0/4051674675 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248003e30 con 0x7f2258105410 2024-02-14T15:05:57.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.343+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22581a48d0 con 0x7f2258105410 2024-02-14T15:05:57.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f224e7fc640 1 -- 172.21.15.17:0/4051674675 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2248005440 con 0x7f2258105410 2024-02-14T15:05:57.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 Processor -- start 2024-02-14T15:05:57.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 -- start start 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f224e7fc640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f22440a2230 0x7f22440a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792824b80 0x564792824f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792832050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x564792834a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f224e7fc640 1 -- 172.21.15.17:0/4051674675 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f22480f0770 con 0x7f2258105410 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f225d9f6640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f22440a2230 0x7f22440a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564792783c80 con 0x564792832620 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564792793320 con 0x564792825530 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.345+0000 7f2262021740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564792790850 con 0x564792824b80 2024-02-14T15:05:57.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792824b80 0x564792824f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d9f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x564792834a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792824b80 0x564792824f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33184/0 (socket says 172.21.15.17:33184) 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 -- 172.21.15.17:0/3123627454 learned_addr learned my addr 172.21.15.17:0/3123627454 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225c9f4640 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792832050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 -- 172.21.15.17:0/3123627454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 msgr2=0x564792832050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792832050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 -- 172.21.15.17:0/3123627454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 msgr2=0x564792834a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x564792834a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 -- 172.21.15.17:0/3123627454 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564792752e30 con 0x564792824b80 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d9f6640 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x564792834a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d9f6640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f22440a2230 0x7f22440a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2250005fd0 tx=0x7f2250009e30 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f225d1f5640 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792824b80 0x564792824f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f225400ac20 tx=0x7f225400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f9b2c3daa7ef668 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f22437fe640 1 -- 172.21.15.17:0/3123627454 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400a720 con 0x564792824b80 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f22437fe640 1 -- 172.21.15.17:0/3123627454 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2254018b20 con 0x564792824b80 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.346+0000 7f22437fe640 1 -- 172.21.15.17:0/3123627454 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254010c00 con 0x564792824b80 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 -- 172.21.15.17:0/3123627454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792824b80 msgr2=0x564792824f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792824b80 0x564792824f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f225400ac20 tx=0x7f225400f790 comp rx=0 tx=0).stop 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 -- 172.21.15.17:0/3123627454 shutdown_connections 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x564792834a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792832050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 --2- 172.21.15.17:0/3123627454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792824b80 0x564792824f60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 -- 172.21.15.17:0/3123627454 >> 172.21.15.17:0/3123627454 conn(0x56479281f3b0 msgr2=0x56479281f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 -- 172.21.15.17:0/3123627454 shutdown_connections 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 -- 172.21.15.17:0/3123627454 wait complete. 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.347+0000 7f2262021740 1 -- start start 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 1 Processor -- start 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792824b80 0x56479282f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x56479282ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479278c030 con 0x564792832620 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479245f430 con 0x564792824b80 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479241be00 con 0x564792825530 2024-02-14T15:05:57.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f2262021740 10 client.0 _ll_register_callbacks cb 0x56479249cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x56479282ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x56479282ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39872/0 (socket says 172.21.15.17:39872) 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 -- 172.21.15.17:0/3285876255 learned_addr learned my addr 172.21.15.17:0/3285876255 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225c9f4640 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d1f5640 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792824b80 0x56479282f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 -- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 msgr2=0x56479282fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 -- 172.21.15.17:0/3285876255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792824b80 msgr2=0x56479282f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792824b80 0x56479282f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.348+0000 7f225d9f6640 1 -- 172.21.15.17:0/3285876255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225400a880 con 0x564792832620 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f225d9f6640 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x56479282ffd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2250009980 tx=0x7f2250004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56666ab8e238e3b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2240ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2240ff9640 1 -- 172.21.15.17:0/3285876255 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500047c0 con 0x564792832620 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56479248b7e0 con 0x564792832620 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2240ff9640 1 -- 172.21.15.17:0/3285876255 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2250020410 con 0x564792832620 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2240ff9640 1 -- 172.21.15.17:0/3285876255 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22500205d0 con 0x564792832620 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 10 client.16434 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56479278a6b0 con 0x564792832620 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2240ff9640 1 -- 172.21.15.17:0/3285876255 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56479248b7e0 con 0x564792832620 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2240ff9640 1 -- 172.21.15.17:0/3285876255 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2250012580 con 0x564792832620 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 10 client.16434 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 10 client.16434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 20 client.16434 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5647928427f0 0x564792844c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 4 client.16434 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647924c8720 con 0x5647928427f0 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.349+0000 7f225d1f5640 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5647928427f0 0x564792844c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.350+0000 7f225d1f5640 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5647928427f0 0x564792844c10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ecb4824acae8d68 server_cookie=87f04442be6419b4 in_seq=0 out_seq=0 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.350+0000 7f2240ff9640 10 client.16434 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.350+0000 7f2240ff9640 1 -- 172.21.15.17:0/3285876255 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647924c8720 con 0x5647928427f0 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.350+0000 7f2240ff9640 10 client.16434 handle_command_reply: tid=0 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 client.16434 shutdown 2024-02-14T15:05:57.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 msgr2=0x56479282ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x56479282ffd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2250009980 tx=0x7f2250004ab0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 shutdown_connections 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x5647928427f0 0x564792844c10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792832620 0x56479282ffd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2240ff9640 0 client.16434 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 --2- 172.21.15.17:0/3285876255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792824b80 0x56479282f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 >> 172.21.15.17:0/3285876255 conn(0x56479281f3b0 msgr2=0x564792821100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 shutdown_connections 2024-02-14T15:05:57.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 1 -- 172.21.15.17:0/3285876255 wait complete. 2024-02-14T15:05:57.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.351+0000 7f2262021740 20 client.16434 trim_cache size 0 max 16384 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.363+0000 7f2262021740 1 Processor -- start 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.363+0000 7f2262021740 1 -- start start 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f2262021740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x56479282a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f2262021740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f2262021740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792832620 0x564792839650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f2262021740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564792711130 con 0x564792824b80 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f2262021740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564792835430 con 0x564792832620 2024-02-14T15:05:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f2262021740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479248b7e0 con 0x564792825530 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x56479282a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x56479282a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39874/0 (socket says 172.21.15.17:39874) 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 -- 172.21.15.17:0/2282875850 learned_addr learned my addr 172.21.15.17:0/2282875850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225c9f4640 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d9f6640 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792832620 0x564792839650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 -- 172.21.15.17:0/2282875850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 msgr2=0x56479282fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 -- 172.21.15.17:0/2282875850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792832620 msgr2=0x564792839650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792832620 0x564792839650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 -- 172.21.15.17:0/2282875850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564792726c90 con 0x564792824b80 2024-02-14T15:05:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225c9f4640 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d9f6640 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792832620 0x564792839650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.364+0000 7f225d1f5640 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x56479282a830 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f22540058c0 tx=0x7f225400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd9dcc0a8f20804 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2242ffd640 1 -- 172.21.15.17:0/2282875850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400a720 con 0x564792824b80 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2242ffd640 1 -- 172.21.15.17:0/2282875850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f225400a3e0 con 0x564792824b80 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2242ffd640 1 -- 172.21.15.17:0/2282875850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254010d50 con 0x564792824b80 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2262021740 1 -- 172.21.15.17:0/2282875850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 msgr2=0x56479282a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2262021740 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x56479282a830 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f22540058c0 tx=0x7f225400a2d0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2262021740 1 -- 172.21.15.17:0/2282875850 shutdown_connections 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2262021740 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792832620 0x564792839650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.365+0000 7f2262021740 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792825530 0x56479282fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 --2- 172.21.15.17:0/2282875850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x56479282a830 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 -- 172.21.15.17:0/2282875850 >> 172.21.15.17:0/2282875850 conn(0x56479281f3b0 msgr2=0x56479281e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 -- 172.21.15.17:0/2282875850 shutdown_connections 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 -- 172.21.15.17:0/2282875850 wait complete. 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 -- start start 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 Processor -- start 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x5647928389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792838ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792832620 0x564792892d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647928932a0 con 0x564792824b80 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647928933e0 con 0x564792825530 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.366+0000 7f2262021740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647928935a0 con 0x564792832620 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2262021740 10 client.0 _ll_register_callbacks cb 0x7f225400a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x5647928389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x5647928389b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39890/0 (socket says 172.21.15.17:39890) 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 -- 172.21.15.17:0/1367685987 learned_addr learned my addr 172.21.15.17:0/1367685987 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225c9f4640 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792838ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d9f6640 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792832620 0x564792892d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 -- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792832620 msgr2=0x564792892d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792832620 0x564792892d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 -- 172.21.15.17:0/1367685987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 msgr2=0x564792838ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792838ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 -- 172.21.15.17:0/1367685987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22581a48d0 con 0x564792824b80 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f225d1f5640 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x5647928389b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f225402e040 tx=0x7f225400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2061f52ffcc8d2be server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2240ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2240ff9640 1 -- 172.21.15.17:0/1367685987 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254010de0 con 0x564792824b80 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2240ff9640 1 -- 172.21.15.17:0/1367685987 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2254018440 con 0x564792824b80 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2254018b20 con 0x564792824b80 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2240ff9640 1 -- 172.21.15.17:0/1367685987 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f225402ae10 con 0x564792824b80 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2262021740 10 client.16440 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.367+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56479278a6b0 con 0x564792824b80 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2240ff9640 1 -- 172.21.15.17:0/1367685987 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2254018b20 con 0x564792824b80 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2240ff9640 1 -- 172.21.15.17:0/1367685987 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f225403f070 con 0x564792824b80 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2262021740 10 client.16440 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2262021740 10 client.16440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2262021740 20 client.16440 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2262021740 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56479289a9f0 0x56479289ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2262021740 4 client.16440 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 --> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564792517890 con 0x56479289a9f0 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f225c9f4640 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56479289a9f0 0x56479289ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f2240ff9640 10 client.16440 ms_handle_connect on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.368+0000 7f225c9f4640 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56479289a9f0 0x56479289ce10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38474ebcc1e93d5 server_cookie=dd4a2b976bc293c5 in_seq=0 out_seq=0 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2240ff9640 1 -- 172.21.15.17:0/1367685987 <== mds.2 v2:172.21.15.76:6836/3638854935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+15452 (crc 0 0 0) 0x564792517890 con 0x56479289a9f0 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2240ff9640 10 client.16440 handle_command_reply: tid=0 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 client.16440 shutdown 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 msgr2=0x5647928389b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x5647928389b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f225402e040 tx=0x7f225400ffd0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 shutdown_connections 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:6836/3638854935,v1:172.21.15.76:6837/3638854935] conn(0x56479289a9f0 0x56479289ce10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564792832620 0x564792892d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564792825530 0x564792838ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 --2- 172.21.15.17:0/1367685987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564792824b80 0x5647928389b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 >> 172.21.15.17:0/1367685987 conn(0x56479281f3b0 msgr2=0x5647928203b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2240ff9640 0 client.16440 ms_handle_reset on v2:172.21.15.76:6836/3638854935 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 shutdown_connections 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.369+0000 7f2262021740 1 -- 172.21.15.17:0/1367685987 wait complete. 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f2262021740 20 client.16440 trim_cache size 0 max 16384 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-02-14T15:05:57.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 62058807, 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628327, 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC" 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 70282672, 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628342, 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB" 2024-02-14T15:05:57.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 131118889, 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628310, 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 132892387, 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511627783, 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients" 2024-02-14T15:05:57.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 151978286, 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628311, 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 198235716, 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628335, 2024-02-14T15:05:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX" 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 254722844, 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628340, 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL" 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 345384551, 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628326, 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS" 2024-02-14T15:05:57.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 421242307, 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628302, 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 634550466, 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628348, 2024-02-14T15:05:57.384 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL" 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 700496905, 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628294, 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.003" 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 754216507, 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628338, 2024-02-14T15:05:57.385 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1" 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 755351011, 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628324, 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP" 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 822254122, 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628299, 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC" 2024-02-14T15:05:57.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 862507500, 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628345, 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03" 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1035963386, 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628312, 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1042504449, 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628318, 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1200305259, 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628315, 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1228079811, 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628329, 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX" 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1255107092, 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628308, 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT" 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1337282388, 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628321, 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO" 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1355102890, 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628298, 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC" 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1407324553, 2024-02-14T15:05:57.390 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628346, 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL" 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1447176037, 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628343, 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX" 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1629560916, 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628296, 2024-02-14T15:05:57.391 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD" 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1662983673, 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628301, 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1671995221, 2024-02-14T15:05:57.392 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511627784, 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0" 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1688619646, 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628303, 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT" 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1920694774, 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628339, 2024-02-14T15:05:57.393 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1" 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2015262532, 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628349, 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL" 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2079098749, 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628304, 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT" 2024-02-14T15:05:57.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2141058533, 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628314, 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2256613811, 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628341, 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB" 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2308388004, 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628320, 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2434721239, 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628325, 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM" 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.396 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2460322358, 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628292, 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.001" 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2756721182, 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628316, 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2765577132, 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511627785, 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp" 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2772182371, 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628337, 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0" 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.398 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2853713412, 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628351, 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB" 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2896672087, 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628309, 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL" 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2906779718, 2024-02-14T15:05:57.399 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628293, 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.002" 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2967929424, 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628313, 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2985393586, 2024-02-14T15:05:57.400 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628323, 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS" 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3015502775, 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628354, 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL" 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3062645563, 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628355, 2024-02-14T15:05:57.401 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL" 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3142125753, 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628333, 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04" 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3170141149, 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628328, 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC" 2024-02-14T15:05:57.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3192401249, 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628322, 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP" 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3356741973, 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628300, 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL" 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3426846455, 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628305, 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP" 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3471131228, 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511627786, 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.000" 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.404 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3480715737, 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628330, 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB" 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3579274792, 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628352, 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED" 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.405 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3688877219, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628317, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3708698930, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628353, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL" 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3747691089, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628347, 2024-02-14T15:05:57.406 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV" 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3827904488, 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628307, 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT" 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3946849072, 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628306, 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.407 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 3990271225, 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628332, 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04" 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4005225290, 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628350, 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS" 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.408 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4038077303, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628344, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03" 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4065279409, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628336, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0" 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4075018131, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628331, 2024-02-14T15:05:57.409 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX" 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4095022917, 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628334, 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB" 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4133363028, 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628319, 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4269652947, 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628295, 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.004" 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "damage_type": "backtrace", 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4270378812, 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 1099511628297, 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: "path": "/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC" 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:05:57.411 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f22440a2230 msgr2=0x7f22440a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f22440a2230 0x7f22440a46f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2250005fd0 tx=0x7f2250009e30 comp rx=0 tx=0).stop 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258105410 msgr2=0x7f2258198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258105410 0x7f2258198c30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f2248004270 comp rx=0 tx=0).stop 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 shutdown_connections 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f22440a2230 0x7f22440a46f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2258105410 0x7f2258198c30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22581991c0 0x7f22581a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 --2- 172.21.15.17:0/4051674675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2258100340 0x7f22581986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.370+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 >> 172.21.15.17:0/4051674675 conn(0x7f22580772e0 msgr2=0x7f22580fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.371+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 shutdown_connections 2024-02-14T15:05:57.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.371+0000 7f225f459640 1 -- 172.21.15.17:0/4051674675 wait complete. 2024-02-14T15:05:57.414 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:05:57.414 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.525+0000 7f91db34f640 1 -- 172.21.15.17:0/2663723852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d4102d90 msgr2=0x7f91d410b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.525+0000 7f91db34f640 1 --2- 172.21.15.17:0/2663723852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d4102d90 0x7f91d410b270 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009900 tx=0x7f91c402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.525+0000 7f91db34f640 1 -- 172.21.15.17:0/2663723852 shutdown_connections 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.525+0000 7f91db34f640 1 --2- 172.21.15.17:0/2663723852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d4102d90 0x7f91d410b270 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.525+0000 7f91db34f640 1 --2- 172.21.15.17:0/2663723852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d410b7b0 0x7f91d410dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.525+0000 7f91db34f640 1 --2- 172.21.15.17:0/2663723852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d4102470 0x7f91d4102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.525+0000 7f91db34f640 1 -- 172.21.15.17:0/2663723852 >> 172.21.15.17:0/2663723852 conn(0x7f91d40772e0 msgr2=0x7f91d40776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.526+0000 7f91db34f640 1 -- 172.21.15.17:0/2663723852 shutdown_connections 2024-02-14T15:05:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.526+0000 7f91db34f640 1 -- 172.21.15.17:0/2663723852 wait complete. 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.526+0000 7f91db34f640 1 Processor -- start 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.526+0000 7f91db34f640 1 -- start start 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91db34f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d4102470 0x7f91d419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91db34f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d4102d90 0x7f91d419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91db34f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d410b7b0 0x7f91d41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91db34f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4197e80 con 0x7f91d410b7b0 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91db34f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4197ff0 con 0x7f91d4102d90 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91db34f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41981b0 con 0x7f91d4102470 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d410b7b0 0x7f91d41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d4102470 0x7f91d419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d88ea640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d4102d90 0x7f91d419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d410b7b0 0x7f91d41976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39908/0 (socket says 172.21.15.17:39908) 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d4102470 0x7f91d419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33220/0 (socket says 172.21.15.17:33220) 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 -- 172.21.15.17:0/1874757054 learned_addr learned my addr 172.21.15.17:0/1874757054 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d88ea640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d4102d90 0x7f91d419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57108/0 (socket says 172.21.15.17:57108) 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 -- 172.21.15.17:0/1874757054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d4102470 msgr2=0x7f91d419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d4102470 0x7f91d419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 -- 172.21.15.17:0/1874757054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d4102d90 msgr2=0x7f91d419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d4102d90 0x7f91d419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 -- 172.21.15.17:0/1874757054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc009840 con 0x7f91d410b7b0 2024-02-14T15:05:57.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d88ea640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d4102d90 0x7f91d419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91d98ec640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d410b7b0 0x7f91d41976f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00ec70 tx=0x7f91cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91ca7fc640 1 -- 172.21.15.17:0/1874757054 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc003b70 con 0x7f91d410b7b0 2024-02-14T15:05:57.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c4009560 con 0x7f91d410b7b0 2024-02-14T15:05:57.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91ca7fc640 1 -- 172.21.15.17:0/1874757054 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91cc00ee80 con 0x7f91d410b7b0 2024-02-14T15:05:57.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.527+0000 7f91ca7fc640 1 -- 172.21.15.17:0/1874757054 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc010810 con 0x7f91d410b7b0 2024-02-14T15:05:57.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.528+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d41a45b0 con 0x7f91d410b7b0 2024-02-14T15:05:57.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.530+0000 7f91ddffa740 1 Processor -- start 2024-02-14T15:05:57.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ca7fc640 1 -- 172.21.15.17:0/1874757054 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f91cc0109d0 con 0x7f91d410b7b0 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ddffa740 1 -- start start 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065db40 0x55dc3065df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc30667030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc30667ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ddffa740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc305bcc80 con 0x55dc30667650 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ddffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc305cc320 con 0x55dc3065db40 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ddffa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc305c9850 con 0x55dc3065e4f0 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc30667030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc30667030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33222/0 (socket says 172.21.15.17:33222) 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91d90eb640 1 -- 172.21.15.17:0/325791620 learned_addr learned my addr 172.21.15.17:0/325791620 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91d88ea640 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc30667ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91ca7fc640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f91c00a2230 0x7f91c00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.531+0000 7f91d88ea640 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065db40 0x55dc3065df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d90eb640 1 -- 172.21.15.17:0/325791620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065db40 msgr2=0x55dc3065df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d90eb640 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065db40 0x55dc3065df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91ca7fc640 1 -- 172.21.15.17:0/1874757054 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f91cc014070 con 0x7f91d410b7b0 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d90eb640 1 -- 172.21.15.17:0/325791620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 msgr2=0x55dc30667ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d90eb640 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc30667ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d90eb640 1 -- 172.21.15.17:0/325791620 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc3058be30 con 0x55dc3065e4f0 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d88ea640 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065db40 0x55dc3065df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:05:57.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d98ec640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f91c00a2230 0x7f91c00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91d90eb640 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc30667030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009d20 tx=0x7f91d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a482b1da6f46be server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91bf7fe640 1 -- 172.21.15.17:0/325791620 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0019630 con 0x55dc3065e4f0 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91bf7fe640 1 -- 172.21.15.17:0/325791620 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91d0019c70 con 0x55dc3065e4f0 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.532+0000 7f91bf7fe640 1 -- 172.21.15.17:0/325791620 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000ec70 con 0x55dc3065e4f0 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 -- 172.21.15.17:0/325791620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 msgr2=0x55dc30667030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91d98ec640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f91c00a2230 0x7f91c00a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f91cc09af50 tx=0x7f91cc009b50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc30667030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009d20 tx=0x7f91d0011810 comp rx=0 tx=0).stop 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 -- 172.21.15.17:0/325791620 shutdown_connections 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc30667ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc30667030 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 --2- 172.21.15.17:0/325791620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065db40 0x55dc3065df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 -- 172.21.15.17:0/325791620 >> 172.21.15.17:0/325791620 conn(0x55dc306583b0 msgr2=0x55dc30658790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 -- 172.21.15.17:0/325791620 shutdown_connections 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.533+0000 7f91ddffa740 1 -- 172.21.15.17:0/325791620 wait complete. 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 -- start start 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 Processor -- start 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065db40 0x55dc3066b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc3066b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc3066bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc305c5030 con 0x55dc3065db40 2024-02-14T15:05:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc30298430 con 0x55dc30667650 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc30254e00 con 0x55dc3065e4f0 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91ddffa740 10 client.0 _ll_register_callbacks cb 0x55dc302d5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc3066b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.534+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc3066b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33238/0 (socket says 172.21.15.17:33238) 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d90eb640 1 -- 172.21.15.17:0/521702189 learned_addr learned my addr 172.21.15.17:0/521702189 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d88ea640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc3066bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d88ea640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065db40 0x55dc3066b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d90eb640 1 -- 172.21.15.17:0/521702189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 msgr2=0x55dc3066bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d90eb640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc3066bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d90eb640 1 -- 172.21.15.17:0/521702189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065db40 msgr2=0x55dc3066b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d90eb640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065db40 0x55dc3066b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d90eb640 1 -- 172.21.15.17:0/521702189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d0009950 con 0x55dc3065e4f0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d88ea640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc3066bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91d90eb640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc3066b600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91d000ee40 tx=0x7f91d000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aba20bc7890563e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91bcff9640 1 -- 172.21.15.17:0/521702189 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0004230 con 0x55dc3065e4f0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91bcff9640 1 -- 172.21.15.17:0/521702189 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f91d0004390 con 0x55dc3065e4f0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc302c47e0 con 0x55dc3065e4f0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.535+0000 7f91bcff9640 1 -- 172.21.15.17:0/521702189 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d000e610 con 0x55dc3065e4f0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 10 client.25978 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc305c36b0 con 0x55dc3065e4f0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91bcff9640 1 -- 172.21.15.17:0/521702189 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dc302c47e0 con 0x55dc3065e4f0 2024-02-14T15:05:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91bcff9640 1 -- 172.21.15.17:0/521702189 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f91d00198f0 con 0x55dc3065e4f0 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 10 client.25978 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 10 client.25978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 20 client.25978 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc3067b6f0 0x55dc3067db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 4 client.25978 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc30301720 con 0x55dc3067b6f0 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.536+0000 7f91d88ea640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc3067b6f0 0x55dc3067db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.537+0000 7f91d88ea640 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc3067b6f0 0x55dc3067db10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e842a49cbd836b1d server_cookie=29d0383122f1ff5a in_seq=0 out_seq=0 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.537+0000 7f91bcff9640 10 client.25978 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.537+0000 7f91bcff9640 1 -- 172.21.15.17:0/521702189 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc30301720 con 0x55dc3067b6f0 2024-02-14T15:05:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.537+0000 7f91bcff9640 10 client.25978 handle_command_reply: tid=0 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.537+0000 7f91ddffa740 1 client.25978 shutdown 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.537+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 msgr2=0x55dc3066b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.537+0000 7f91ddffa740 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc3066b600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91d000ee40 tx=0x7f91d000ee70 comp rx=0 tx=0).stop 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 shutdown_connections 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc3067b6f0 0x55dc3067db10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc3066bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065e4f0 0x55dc3066b600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 --2- 172.21.15.17:0/521702189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065db40 0x55dc3066b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 >> 172.21.15.17:0/521702189 conn(0x55dc306583b0 msgr2=0x55dc3065a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91bcff9640 0 client.25978 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:57.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 shutdown_connections 2024-02-14T15:05:57.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 1 -- 172.21.15.17:0/521702189 wait complete. 2024-02-14T15:05:57.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.538+0000 7f91ddffa740 20 client.25978 trim_cache size 0 max 16384 2024-02-14T15:05:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 Processor -- start 2024-02-14T15:05:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 -- start start 2024-02-14T15:05:57.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc3066d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065e4f0 0x55dc30668a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc3066b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3054a130 con 0x55dc30667650 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc30657c50 con 0x55dc3065e4f0 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91ddffa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3065cd70 con 0x55dc3065db40 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91d88ea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc3066b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91d88ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc3066d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91d88ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc3066d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33242/0 (socket says 172.21.15.17:33242) 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91d88ea640 1 -- 172.21.15.17:0/3846412286 learned_addr learned my addr 172.21.15.17:0/3846412286 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.552+0000 7f91d90eb640 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065e4f0 0x55dc30668a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91d88ea640 1 -- 172.21.15.17:0/3846412286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065e4f0 msgr2=0x55dc30668a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065e4f0 0x55dc30668a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91d88ea640 1 -- 172.21.15.17:0/3846412286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 msgr2=0x55dc3066b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc3066b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91d88ea640 1 -- 172.21.15.17:0/3846412286 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc3055fc90 con 0x55dc3065db40 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc3066d7f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f91c4004c50 tx=0x7f91c4046040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5536df47b61fc0d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91beffd640 1 -- 172.21.15.17:0/3846412286 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4049050 con 0x55dc3065db40 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91beffd640 1 -- 172.21.15.17:0/3846412286 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91c4055030 con 0x55dc3065db40 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.553+0000 7f91beffd640 1 -- 172.21.15.17:0/3846412286 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c404da90 con 0x55dc3065db40 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 -- 172.21.15.17:0/3846412286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 msgr2=0x55dc3066d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc3066d7f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f91c4004c50 tx=0x7f91c4046040 comp rx=0 tx=0).stop 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 -- 172.21.15.17:0/3846412286 shutdown_connections 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc30667650 0x55dc3066b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc3065e4f0 0x55dc30668a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3846412286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc3066d7f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 -- 172.21.15.17:0/3846412286 >> 172.21.15.17:0/3846412286 conn(0x55dc306583b0 msgr2=0x55dc30665000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 -- 172.21.15.17:0/3846412286 shutdown_connections 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 -- 172.21.15.17:0/3846412286 wait complete. 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 -- start start 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.554+0000 7f91ddffa740 1 Processor -- start 2024-02-14T15:05:57.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc30672ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065e4f0 0x55dc30670e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91ddffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc30671400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91ddffa740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc30671b00 con 0x55dc3065e4f0 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91ddffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc30671c70 con 0x55dc30667650 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91ddffa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc306d1c40 con 0x55dc3065db40 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91ddffa740 10 client.0 _ll_register_callbacks cb 0x7f91d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065e4f0 0x55dc30670e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065e4f0 0x55dc30670e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39948/0 (socket says 172.21.15.17:39948) 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 -- 172.21.15.17:0/3512301858 learned_addr learned my addr 172.21.15.17:0/3512301858 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc30672ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc30671400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 -- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 msgr2=0x55dc30672ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc30672ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 -- 172.21.15.17:0/3512301858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 msgr2=0x55dc30671400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc30671400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 -- 172.21.15.17:0/3512301858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d410e8d0 con 0x55dc3065e4f0 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc30671400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91d90eb640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065e4f0 0x55dc30670e80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55dc3066a920 tx=0x7f91d0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6a0dc067b4ebac server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.555+0000 7f91bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91bcff9640 1 -- 172.21.15.17:0/3512301858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0004030 con 0x55dc3065e4f0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91bcff9640 1 -- 172.21.15.17:0/3512301858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f91d0004190 con 0x55dc3065e4f0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc30653a40 con 0x55dc3065e4f0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91bcff9640 1 -- 172.21.15.17:0/3512301858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d0004350 con 0x55dc3065e4f0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 10 client.16458 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc30585aa0 con 0x55dc3065e4f0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91bcff9640 1 -- 172.21.15.17:0/3512301858 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dc30653a40 con 0x55dc3065e4f0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91bcff9640 1 -- 172.21.15.17:0/3512301858 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f91d003d070 con 0x55dc3065e4f0 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 10 client.16458 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 10 client.16458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 20 client.16458 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc306d3040 0x55dc306d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 4 client.16458 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 --> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f91d0019c70 con 0x55dc306d3040 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.556+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc306d3040 0x55dc306d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91bcff9640 10 client.16458 ms_handle_connect on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91d88ea640 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc306d3040 0x55dc306d5460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd567e3e1d9d9b09 server_cookie=e29764a77848897b in_seq=0 out_seq=0 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91bcff9640 1 -- 172.21.15.17:0/3512301858 <== mds.1 v2:172.21.15.76:6838/666966898 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f91d0019c70 con 0x55dc306d3040 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91bcff9640 10 client.16458 handle_command_reply: tid=0 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 client.16458 shutdown 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065e4f0 msgr2=0x55dc30670e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065e4f0 0x55dc30670e80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55dc3066a920 tx=0x7f91d0009310 comp rx=0 tx=0).stop 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 shutdown_connections 2024-02-14T15:05:57.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:6838/666966898,v1:172.21.15.76:6839/666966898] conn(0x55dc306d3040 0x55dc306d5460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc3065e4f0 0x55dc30670e80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dc30667650 0x55dc30671400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91bcff9640 0 client.16458 ms_handle_reset on v2:172.21.15.76:6838/666966898 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 --2- 172.21.15.17:0/3512301858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc3065db40 0x55dc30672ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 >> 172.21.15.17:0/3512301858 conn(0x55dc306583b0 msgr2=0x55dc30669970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 shutdown_connections 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 1 -- 172.21.15.17:0/3512301858 wait complete. 2024-02-14T15:05:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.557+0000 7f91ddffa740 20 client.16458 trim_cache size 0 max 16384 2024-02-14T15:05:57.569 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:05:57.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f91c00a2230 msgr2=0x7f91c00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f91c00a2230 0x7f91c00a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f91cc09af50 tx=0x7f91cc009b50 comp rx=0 tx=0).stop 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d410b7b0 msgr2=0x7f91d41976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d410b7b0 0x7f91d41976f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00ec70 tx=0x7f91cc00c6a0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 shutdown_connections 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f91c00a2230 0x7f91c00a46f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91d410b7b0 0x7f91d41976f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91d4102d90 0x7f91d419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 --2- 172.21.15.17:0/1874757054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d4102470 0x7f91d419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 >> 172.21.15.17:0/1874757054 conn(0x7f91d40772e0 msgr2=0x7f91d40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 shutdown_connections 2024-02-14T15:05:57.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.558+0000 7f91db34f640 1 -- 172.21.15.17:0/1874757054 wait complete. 2024-02-14T15:05:57.572 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-14T15:05:57.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/2513979042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08100340 msgr2=0x7f6e08100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/2513979042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08100340 0x7f6e08100720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009860 tx=0x7f6df802bb70 comp rx=0 tx=0).stop 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/2513979042 shutdown_connections 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/2513979042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08105410 0x7f6e0810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/2513979042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e08100cf0 0x7f6e08104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/2513979042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08100340 0x7f6e08100720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/2513979042 >> 172.21.15.17:0/2513979042 conn(0x7f6e08077220 msgr2=0x7f6e08077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/2513979042 shutdown_connections 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.569+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/2513979042 wait complete. 2024-02-14T15:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 Processor -- start 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 -- start start 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08100340 0x7f6e08198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e08105410 0x7f6e08198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08199160 0x7f6e081a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081a3db0 con 0x7f6e08100340 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081a3f20 con 0x7f6e08105410 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e0d4f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081a40e0 con 0x7f6e08199160 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08100340 0x7f6e08198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08100340 0x7f6e08198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39974/0 (socket says 172.21.15.17:39974) 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e06ffd640 1 -- 172.21.15.17:0/1477234973 learned_addr learned my addr 172.21.15.17:0/1477234973 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e077fe640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08199160 0x7f6e081a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6dfffff640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e08105410 0x7f6e08198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e06ffd640 1 -- 172.21.15.17:0/1477234973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08199160 msgr2=0x7f6e081a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.570+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08199160 0x7f6e081a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e06ffd640 1 -- 172.21.15.17:0/1477234973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e08105410 msgr2=0x7f6e08198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e08105410 0x7f6e08198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e06ffd640 1 -- 172.21.15.17:0/1477234973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e00009900 con 0x7f6e08100340 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08100340 0x7f6e08198690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6df8037040 tx=0x7f6df80045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e04ff9640 1 -- 172.21.15.17:0/1477234973 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80349d0 con 0x7f6e08100340 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df8009490 con 0x7f6e08100340 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e04ff9640 1 -- 172.21.15.17:0/1477234973 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6df802c9f0 con 0x7f6e08100340 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e04ff9640 1 -- 172.21.15.17:0/1477234973 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8032600 con 0x7f6e08100340 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.571+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e081a4590 con 0x7f6e08100340 2024-02-14T15:05:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.573+0000 7f6e04ff9640 1 -- 172.21.15.17:0/1477234973 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6df803b040 con 0x7f6e08100340 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 Processor -- start 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 -- start start 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x564401365b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x564401371010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644012d39e0 con 0x56440136ec20 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644012d0f10 con 0x564401366040 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e101a2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644012cc6f0 con 0x564401365720 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x564401365b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x564401365b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33274/0 (socket says 172.21.15.17:33274) 2024-02-14T15:05:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e077fe640 1 -- 172.21.15.17:0/565592115 learned_addr learned my addr 172.21.15.17:0/565592115 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6dfffff640 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x564401371010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e04ff9640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6df40a2220 0x7f6df40a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.574+0000 7f6e077fe640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6df40a2220 0x7f6df40a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfffff640 1 -- 172.21.15.17:0/565592115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 msgr2=0x564401365b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfffff640 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x564401365b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfffff640 1 -- 172.21.15.17:0/565592115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 msgr2=0x56440136e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6e04ff9640 1 -- 172.21.15.17:0/1477234973 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6df80efe80 con 0x7f6e08100340 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfffff640 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfffff640 1 -- 172.21.15.17:0/565592115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564400fdd920 con 0x56440136ec20 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6e077fe640 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x564401365b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfffff640 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x564401371010 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6e0806a980 tx=0x7f6df0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d832bddc75d6257e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6e077fe640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6df40a2220 0x7f6df40a46e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009c50 tx=0x7f6e00004ab0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfd7fa640 1 -- 172.21.15.17:0/565592115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0004050 con 0x56440136ec20 2024-02-14T15:05:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfd7fa640 1 -- 172.21.15.17:0/565592115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6df0005500 con 0x56440136ec20 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.575+0000 7f6dfd7fa640 1 -- 172.21.15.17:0/565592115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0004c30 con 0x56440136ec20 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 -- 172.21.15.17:0/565592115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 msgr2=0x564401371010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x564401371010 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6e0806a980 tx=0x7f6df0011810 comp rx=0 tx=0).stop 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 -- 172.21.15.17:0/565592115 shutdown_connections 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x564401371010 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 --2- 172.21.15.17:0/565592115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x564401365b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 -- 172.21.15.17:0/565592115 >> 172.21.15.17:0/565592115 conn(0x56440135fe50 msgr2=0x564401360230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 -- 172.21.15.17:0/565592115 shutdown_connections 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 -- 172.21.15.17:0/565592115 wait complete. 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 -- start start 2024-02-14T15:05:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.576+0000 7f6e101a2740 1 Processor -- start 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x56440136e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x56440136b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x56440136be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e101a2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564401268fa0 con 0x564401365720 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e101a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564400fa0250 con 0x56440136ec20 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e101a2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564401252200 con 0x564401366040 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e101a2740 10 client.0 _ll_register_callbacks cb 0x5644012c1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x56440136e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x56440136b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x56440136e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40004/0 (socket says 172.21.15.17:40004) 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 -- 172.21.15.17:0/3617826695 learned_addr learned my addr 172.21.15.17:0/3617826695 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e077fe640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x56440136be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 -- 172.21.15.17:0/3617826695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 msgr2=0x56440136b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x56440136b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 -- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 msgr2=0x56440136be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x56440136be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 -- 172.21.15.17:0/3617826695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df0009850 con 0x564401365720 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6dfffff640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x56440136b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e077fe640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x56440136be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x56440136e580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ed380 tx=0x7f6df80b8e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c7216973ab1782 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6dee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6dee7fc640 1 -- 172.21.15.17:0/3617826695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80bcad0 con 0x564401365720 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.577+0000 7f6dee7fc640 1 -- 172.21.15.17:0/3617826695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6df80bc670 con 0x564401365720 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564400fcc600 con 0x564401365720 2024-02-14T15:05:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6dee7fc640 1 -- 172.21.15.17:0/3617826695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6df80b8910 con 0x564401365720 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 10 client.16470 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5644012cad70 con 0x564401365720 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6dee7fc640 1 -- 172.21.15.17:0/3617826695 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564400fcc600 con 0x564401365720 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6dee7fc640 1 -- 172.21.15.17:0/3617826695 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6df80c48d0 con 0x564401365720 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 10 client.16470 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 20 client.16470 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56440137e670 0x564401380a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 4 client.16470 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56440101d1b0 con 0x56440137e670 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.578+0000 7f6dfffff640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56440137e670 0x564401380a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6dee7fc640 10 client.16470 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6dfffff640 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56440137e670 0x564401380a60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7981c6768813c6a2 server_cookie=f5f4bda0d5cc01f7 in_seq=0 out_seq=0 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6dee7fc640 1 -- 172.21.15.17:0/3617826695 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56440101d1b0 con 0x56440137e670 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6dee7fc640 10 client.16470 handle_command_reply: tid=0 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6e101a2740 1 client.16470 shutdown 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 msgr2=0x56440136e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6e101a2740 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x56440136e580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ed380 tx=0x7f6df80b8e80 comp rx=0 tx=0).stop 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.579+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 shutdown_connections 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56440137e670 0x564401380a60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x56440136be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x56440136b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6dee7fc640 0 client.16470 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 1 --2- 172.21.15.17:0/3617826695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x56440136e580 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 >> 172.21.15.17:0/3617826695 conn(0x56440135fe50 msgr2=0x564401361ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 shutdown_connections 2024-02-14T15:05:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 1 -- 172.21.15.17:0/3617826695 wait complete. 2024-02-14T15:05:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.580+0000 7f6e101a2740 20 client.16470 trim_cache size 0 max 16384 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.597+0000 7f6e101a2740 1 Processor -- start 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.597+0000 7f6e101a2740 1 -- start start 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x56440136b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x56440136e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e101a2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564401364900 con 0x56440136ec20 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e101a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564401365380 con 0x564401366040 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e101a2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56440137e470 con 0x564401365720 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x56440136e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x56440136b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x56440136e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40008/0 (socket says 172.21.15.17:40008) 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e077fe640 1 -- 172.21.15.17:0/2549465479 learned_addr learned my addr 172.21.15.17:0/2549465479 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6dfffff640 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6dfffff640 1 -- 172.21.15.17:0/2549465479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 msgr2=0x56440136b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6dfffff640 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x56440136b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6dfffff640 1 -- 172.21.15.17:0/2549465479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 msgr2=0x56440136e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6dfffff640 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x56440136e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6dfffff640 1 -- 172.21.15.17:0/2549465479 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564401267820 con 0x564401366040 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x56440136b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.598+0000 7f6dfffff640 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009350 tx=0x7f6df00096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add6fdb1c3f9e950 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6dfcff9640 1 -- 172.21.15.17:0/2549465479 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df002a040 con 0x564401366040 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6dfcff9640 1 -- 172.21.15.17:0/2549465479 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6df0011d10 con 0x564401366040 2024-02-14T15:05:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6dfcff9640 1 -- 172.21.15.17:0/2549465479 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0003850 con 0x564401366040 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6e101a2740 1 -- 172.21.15.17:0/2549465479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 msgr2=0x56440136e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6e101a2740 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009350 tx=0x7f6df00096c0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6e101a2740 1 -- 172.21.15.17:0/2549465479 shutdown_connections 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6e101a2740 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56440136ec20 0x56440136e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6e101a2740 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564401366040 0x56440136e040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6e101a2740 1 --2- 172.21.15.17:0/2549465479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401365720 0x56440136b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.599+0000 7f6e101a2740 1 -- 172.21.15.17:0/2549465479 >> 172.21.15.17:0/2549465479 conn(0x56440135fe50 msgr2=0x56440136b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 -- 172.21.15.17:0/2549465479 shutdown_connections 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 -- 172.21.15.17:0/2549465479 wait complete. 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 -- start start 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 Processor -- start 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x564401374550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x564401374a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x564401374fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564401376380 con 0x564401365720 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644013d4b90 con 0x56440136ec20 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644013d4cd0 con 0x564401366040 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e101a2740 10 client.0 _ll_register_callbacks cb 0x7f6df00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x564401374550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x564401374a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e06ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x564401374550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40010/0 (socket says 172.21.15.17:40010) 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e06ffd640 1 -- 172.21.15.17:0/276717231 learned_addr learned my addr 172.21.15.17:0/276717231 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.600+0000 7f6e077fe640 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x564401374fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dfffff640 1 -- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 msgr2=0x564401374fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dfffff640 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x564401374fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dfffff640 1 -- 172.21.15.17:0/276717231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 msgr2=0x564401374550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dfffff640 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x564401374550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dfffff640 1 -- 172.21.15.17:0/276717231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644012cad70 con 0x564401366040 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x564401374550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dfffff640 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x564401374a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x564401381060 tx=0x7f6df0005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56541b7efe634edc server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dee7fc640 1 -- 172.21.15.17:0/276717231 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df002a050 con 0x564401366040 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6df0005500 con 0x564401366040 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dee7fc640 1 -- 172.21.15.17:0/276717231 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f6df002d380 con 0x564401366040 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dee7fc640 1 -- 172.21.15.17:0/276717231 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6df002d560 con 0x564401366040 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6e101a2740 10 client.26002 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564400fdd920 con 0x564401366040 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.601+0000 7f6dee7fc640 1 -- 172.21.15.17:0/276717231 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6df0005500 con 0x564401366040 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6dee7fc640 1 -- 172.21.15.17:0/276717231 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6df0021040 con 0x564401366040 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e101a2740 10 client.26002 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e101a2740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e101a2740 20 client.26002 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e101a2740 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5644013d6250 0x5644013d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e101a2740 4 client.26002 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564401009550 con 0x5644013d6250 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5644013d6250 0x5644013d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6dee7fc640 10 client.26002 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.602+0000 7f6e06ffd640 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5644013d6250 0x5644013d8670 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531c358cf58ffd9b server_cookie=67be9e668829efca in_seq=0 out_seq=0 2024-02-14T15:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.609+0000 7f6dee7fc640 1 -- 172.21.15.17:0/276717231 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564401009550 con 0x5644013d6250 2024-02-14T15:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.609+0000 7f6dee7fc640 10 client.26002 handle_command_reply: tid=0 2024-02-14T15:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.609+0000 7f6e101a2740 1 client.26002 shutdown 2024-02-14T15:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 msgr2=0x564401374a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x564401374a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x564401381060 tx=0x7f6df0005f10 comp rx=0 tx=0).stop 2024-02-14T15:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 shutdown_connections 2024-02-14T15:05:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5644013d6250 0x5644013d8670 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56440136ec20 0x564401374fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564401366040 0x564401374a90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 --2- 172.21.15.17:0/276717231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564401365720 0x564401374550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6dee7fc640 0 client.26002 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 >> 172.21.15.17:0/276717231 conn(0x56440135fe50 msgr2=0x5644013611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 shutdown_connections 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 1 -- 172.21.15.17:0/276717231 wait complete. 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e101a2740 20 client.26002 trim_cache size 0 max 16384 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:05:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:05:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6df40a2220 msgr2=0x7f6df40a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6df40a2220 0x7f6df40a46e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009c50 tx=0x7f6e00004ab0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08100340 msgr2=0x7f6e08198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08100340 0x7f6e08198690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6df8037040 tx=0x7f6df80045b0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 shutdown_connections 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6df40a2220 0x7f6df40a46e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e08199160 0x7f6e081a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e08105410 0x7f6e08198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 --2- 172.21.15.17:0/1477234973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e08100340 0x7f6e08198690 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.610+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 >> 172.21.15.17:0/1477234973 conn(0x7f6e08077220 msgr2=0x7f6e080fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.611+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 shutdown_connections 2024-02-14T15:05:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:05:57.611+0000 7f6e0d4f4640 1 -- 172.21.15.17:0/1477234973 wait complete. 2024-02-14T15:05:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 -- 172.21.15.17:0/3759513554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 msgr2=0x7f83541054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 --2- 172.21.15.17:0/3759513554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f83541054a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8348009900 tx=0x7f834802bc80 comp rx=0 tx=0).stop 2024-02-14T15:05:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 -- 172.21.15.17:0/3759513554 shutdown_connections 2024-02-14T15:05:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 --2- 172.21.15.17:0/3759513554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 0x7f835410cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 --2- 172.21.15.17:0/3759513554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8354100860 0x7f8354100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 --2- 172.21.15.17:0/3759513554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f83541054a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 -- 172.21.15.17:0/3759513554 >> 172.21.15.17:0/3759513554 conn(0x7f83540776a0 msgr2=0x7f83540fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 -- 172.21.15.17:0/3759513554 shutdown_connections 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.688+0000 7f835ab2c640 1 -- 172.21.15.17:0/3759513554 wait complete. 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 Processor -- start 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 -- start start 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8354100860 0x7f83541965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 0x7f8354196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f8354197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541a5d00 con 0x7f83541050c0 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541a5e40 con 0x7f8354101200 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f835ab2c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541a5fd0 con 0x7f8354100860 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.689+0000 7f83590c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f8354197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 0x7f8354196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83588c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8354100860 0x7f83541965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 0x7f8354196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57204/0 (socket says 172.21.15.17:57204) 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f8353fff640 1 -- 172.21.15.17:0/4294856673 learned_addr learned my addr 172.21.15.17:0/4294856673 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83588c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8354100860 0x7f83541965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33308/0 (socket says 172.21.15.17:33308) 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83590c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f8354197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40030/0 (socket says 172.21.15.17:40030) 2024-02-14T15:05:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83590c9640 1 -- 172.21.15.17:0/4294856673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8354100860 msgr2=0x7f83541965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83590c9640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8354100860 0x7f83541965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83590c9640 1 -- 172.21.15.17:0/4294856673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 msgr2=0x7f8354196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83590c9640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 0x7f8354196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83590c9640 1 -- 172.21.15.17:0/4294856673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348009560 con 0x7f83541050c0 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f8353fff640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 0x7f8354196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f83590c9640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f8354197070 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f834c00da40 tx=0x7f834c00df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f8351ffb640 1 -- 172.21.15.17:0/4294856673 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00bdd0 con 0x7f83541050c0 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83541a62b0 con 0x7f83541050c0 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f8351ffb640 1 -- 172.21.15.17:0/4294856673 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f834c003e50 con 0x7f83541050c0 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f8351ffb640 1 -- 172.21.15.17:0/4294856673 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0106c0 con 0x7f83541050c0 2024-02-14T15:05:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.690+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83541a6770 con 0x7f83541050c0 2024-02-14T15:05:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 Processor -- start 2024-02-14T15:05:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 -- start start 2024-02-14T15:05:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb1690 0x55fa40bb9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bba210 0x55fa40bbc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40b0fc80 con 0x55fa40bba210 2024-02-14T15:05:57.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40b1f320 con 0x55fa40bb0ce0 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f835d7d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40b1c850 con 0x55fa40bb1690 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f83588c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb1690 0x55fa40bb9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57214/0 (socket says 172.21.15.17:57214) 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f8353fff640 1 -- 172.21.15.17:0/3487330010 learned_addr learned my addr 172.21.15.17:0/3487330010 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.693+0000 7f8353fff640 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bba210 0x55fa40bbc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.694+0000 7f83588c8640 1 -- 172.21.15.17:0/3487330010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb0ce0 msgr2=0x55fa40bb10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.694+0000 7f83588c8640 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.694+0000 7f83588c8640 1 -- 172.21.15.17:0/3487330010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bba210 msgr2=0x55fa40bbc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.694+0000 7f83588c8640 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bba210 0x55fa40bbc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.694+0000 7f83588c8640 1 -- 172.21.15.17:0/3487330010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa40adee30 con 0x55fa40bb1690 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.694+0000 7f83588c8640 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb1690 0x55fa40bb9c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f834802c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f5741bdd5ddeec8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.695+0000 7f8351ffb640 1 -- 172.21.15.17:0/4294856673 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f834c004000 con 0x7f83541050c0 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.696+0000 7f8351ffb640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f83440a2230 0x7f83440a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.696+0000 7f8353fff640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f83440a2230 0x7f83440a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.696+0000 7f8351ffb640 1 -- 172.21.15.17:0/4294856673 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f834c0ce6d0 con 0x7f83541050c0 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.696+0000 7f8353fff640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f83440a2230 0x7f83440a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8340009fd0 tx=0x7f8340005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f833effd640 1 -- 172.21.15.17:0/3487330010 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834802c980 con 0x55fa40bb1690 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f833effd640 1 -- 172.21.15.17:0/3487330010 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f834802cae0 con 0x55fa40bb1690 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f833effd640 1 -- 172.21.15.17:0/3487330010 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480051b0 con 0x55fa40bb1690 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 -- 172.21.15.17:0/3487330010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb1690 msgr2=0x55fa40bb9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb1690 0x55fa40bb9c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f834802c740 comp rx=0 tx=0).stop 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 -- 172.21.15.17:0/3487330010 shutdown_connections 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bba210 0x55fa40bbc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb1690 0x55fa40bb9c40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3487330010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 -- 172.21.15.17:0/3487330010 >> 172.21.15.17:0/3487330010 conn(0x55fa40bab3b0 msgr2=0x55fa40bab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 -- 172.21.15.17:0/3487330010 shutdown_connections 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.697+0000 7f835d7d8740 1 -- 172.21.15.17:0/3487330010 wait complete. 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 -- start start 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 Processor -- start 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40b18030 con 0x55fa40bb0ce0 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa407eb430 con 0x55fa40bb1690 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa407a7e00 con 0x55fa40bba210 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f835d7d8740 10 client.0 _ll_register_callbacks cb 0x55fa40828be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f83588c8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f83590c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57226/0 (socket says 172.21.15.17:57226) 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f83588c8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40050/0 (socket says 172.21.15.17:40050) 2024-02-14T15:05:57.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f8353fff640 1 -- 172.21.15.17:0/3210058348 learned_addr learned my addr 172.21.15.17:0/3210058348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.698+0000 7f83590c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33314/0 (socket says 172.21.15.17:33314) 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f83590c9640 1 -- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 msgr2=0x55fa40bb76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f83590c9640 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f83590c9640 1 -- 172.21.15.17:0/3210058348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 msgr2=0x55fa40bb7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f83590c9640 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f83590c9640 1 -- 172.21.15.17:0/3210058348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835410d8e0 con 0x55fa40bba210 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f8353fff640 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f83588c8640 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f83590c9640 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb7be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f834c0cbac0 tx=0x7f834c09a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d6cd8c7d47fdbe server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f8333fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:57.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f8333fff640 1 -- 172.21.15.17:0/3210058348 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c09eeb0 con 0x55fa40bba210 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f8333fff640 1 -- 172.21.15.17:0/3210058348 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f834c0a3500 con 0x55fa40bba210 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa408177e0 con 0x55fa40bba210 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f8333fff640 1 -- 172.21.15.17:0/3210058348 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f834c0b9120 con 0x55fa40bba210 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f835d7d8740 10 client.26014 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa40b166b0 con 0x55fa40bba210 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.699+0000 7f8333fff640 1 -- 172.21.15.17:0/3210058348 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f834c07d020 con 0x55fa40bba210 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f8333fff640 1 -- 172.21.15.17:0/3210058348 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f834c0ce050 con 0x55fa40bba210 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f835d7d8740 10 client.26014 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f835d7d8740 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f835d7d8740 20 client.26014 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40bcc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f835d7d8740 4 client.26014 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa40854720 con 0x55fa40bca380 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f83588c8640 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40bcc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f8333fff640 10 client.26014 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.700+0000 7f83588c8640 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40bcc7a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ffa2992e1a968c server_cookie=ae75a869a3336b23 in_seq=0 out_seq=0 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f8333fff640 1 -- 172.21.15.17:0/3210058348 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa40854720 con 0x55fa40bca380 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f8333fff640 10 client.26014 handle_command_reply: tid=0 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 client.26014 shutdown 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 msgr2=0x55fa40bb7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb7be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f834c0cbac0 tx=0x7f834c09a780 comp rx=0 tx=0).stop 2024-02-14T15:05:57.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 shutdown_connections 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40bcc7a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb7be0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f8333fff640 0 client.26014 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 --2- 172.21.15.17:0/3210058348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 >> 172.21.15.17:0/3210058348 conn(0x55fa40bab3b0 msgr2=0x55fa40bad100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 shutdown_connections 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 1 -- 172.21.15.17:0/3210058348 wait complete. 2024-02-14T15:05:57.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.701+0000 7f835d7d8740 20 client.26014 trim_cache size 0 max 16384 2024-02-14T15:05:57.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 Processor -- start 2024-02-14T15:05:57.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 -- start start 2024-02-14T15:05:57.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40a9d130 con 0x55fa40bb0ce0 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40854720 con 0x55fa40bb1690 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f835d7d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40bb0920 con 0x55fa40bba210 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f83588c8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f8353fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57240/0 (socket says 172.21.15.17:57240) 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f8353fff640 1 -- 172.21.15.17:0/82850205 learned_addr learned my addr 172.21.15.17:0/82850205 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.715+0000 7f83590c9640 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f8353fff640 1 -- 172.21.15.17:0/82850205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 msgr2=0x55fa40bb2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f8353fff640 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f8353fff640 1 -- 172.21.15.17:0/82850205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 msgr2=0x55fa40bb9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f8353fff640 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f8353fff640 1 -- 172.21.15.17:0/82850205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa40ab2c90 con 0x55fa40bb1690 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f83588c8640 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f8353fff640 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb7a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8340011f30 tx=0x7f8340016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0998cf83fac614 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f833e7fc640 1 -- 172.21.15.17:0/82850205 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340027050 con 0x55fa40bb1690 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f833e7fc640 1 -- 172.21.15.17:0/82850205 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8340028040 con 0x55fa40bb1690 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.716+0000 7f833e7fc640 1 -- 172.21.15.17:0/82850205 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340016070 con 0x55fa40bb1690 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 -- 172.21.15.17:0/82850205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 msgr2=0x55fa40bb7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb7a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8340011f30 tx=0x7f8340016000 comp rx=0 tx=0).stop 2024-02-14T15:05:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 -- 172.21.15.17:0/82850205 shutdown_connections 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bba210 0x55fa40bb2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bb1690 0x55fa40bb7a00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 --2- 172.21.15.17:0/82850205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb0ce0 0x55fa40bb9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 -- 172.21.15.17:0/82850205 >> 172.21.15.17:0/82850205 conn(0x55fa40bab3b0 msgr2=0x55fa40bab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 -- 172.21.15.17:0/82850205 shutdown_connections 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 -- 172.21.15.17:0/82850205 wait complete. 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 -- start start 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.717+0000 7f835d7d8740 1 Processor -- start 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb0ce0 0x55fa40bc1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb1690 0x55fa40bbf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bba210 0x55fa40bbfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40bc0370 con 0x55fa40bb1690 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40bc04e0 con 0x55fa40bba210 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa40bc06a0 con 0x55fa40bb0ce0 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 10 client.0 _ll_register_callbacks cb 0x7f8348009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb0ce0 0x55fa40bc1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f8353fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb1690 0x55fa40bbf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb0ce0 0x55fa40bc1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33328/0 (socket says 172.21.15.17:33328) 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 -- 172.21.15.17:0/2763588834 learned_addr learned my addr 172.21.15.17:0/2763588834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83590c9640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bba210 0x55fa40bbfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 -- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bba210 msgr2=0x55fa40bbfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bba210 0x55fa40bbfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 -- 172.21.15.17:0/2763588834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb1690 msgr2=0x55fa40bbf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:05:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb1690 0x55fa40bbf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 -- 172.21.15.17:0/2763588834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa40adee30 con 0x55fa40bb0ce0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83590c9640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bba210 0x55fa40bbfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f8353fff640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb1690 0x55fa40bbf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f83588c8640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb0ce0 0x55fa40bc1400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8348004550 tx=0x7f83480044e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0660ed6771b73c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f8333fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f8333fff640 1 -- 172.21.15.17:0/2763588834 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348051de0 con 0x55fa40bb0ce0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f8333fff640 1 -- 172.21.15.17:0/2763588834 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8348055030 con 0x55fa40bb0ce0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.718+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f834802cae0 con 0x55fa40bb0ce0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f8333fff640 1 -- 172.21.15.17:0/2763588834 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83480513a0 con 0x55fa40bb0ce0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 10 client.26020 fetch_fsmap learned FSMap version 27 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f8340019040 con 0x55fa40bb0ce0 2024-02-14T15:05:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f8333fff640 1 -- 172.21.15.17:0/2763588834 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f834802cae0 con 0x55fa40bb0ce0 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f8333fff640 1 -- 172.21.15.17:0/2763588834 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f834803c050 con 0x55fa40bb0ce0 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 10 client.26020 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 10 client.26020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 20 client.26020 populate_metadata read hostname 'smithi017' 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40c23b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 4 client.26020 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa408a3890 con 0x55fa40bca380 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.719+0000 7f8353fff640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40c23b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f8333fff640 10 client.26020 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f8353fff640 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40c23b50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e891b45b496c73ff server_cookie=a8fe6cec0b4afac1 in_seq=0 out_seq=0 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f8333fff640 1 -- 172.21.15.17:0/2763588834 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa408a3890 con 0x55fa40bca380 2024-02-14T15:05:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f8333fff640 10 client.26020 handle_command_reply: tid=0 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 client.26020 shutdown 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb0ce0 msgr2=0x55fa40bc1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb0ce0 0x55fa40bc1400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8348004550 tx=0x7f83480044e0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 shutdown_connections 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fa40bca380 0x55fa40c23b50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fa40bb1690 0x55fa40bbf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa40bba210 0x55fa40bbfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 --2- 172.21.15.17:0/2763588834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa40bb0ce0 0x55fa40bc1400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f8333fff640 0 client.26020 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 >> 172.21.15.17:0/2763588834 conn(0x55fa40bab3b0 msgr2=0x55fa40bb2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 shutdown_connections 2024-02-14T15:05:57.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 1 -- 172.21.15.17:0/2763588834 wait complete. 2024-02-14T15:05:57.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.720+0000 7f835d7d8740 20 client.26020 trim_cache size 0 max 16384 2024-02-14T15:05:57.731 INFO:teuthology.orchestra.run.smithi017.stdout:[] 2024-02-14T15:05:57.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f83440a2230 msgr2=0x7f83440a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f83440a2230 0x7f83440a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8340009fd0 tx=0x7f8340005f90 comp rx=0 tx=0).stop 2024-02-14T15:05:57.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 msgr2=0x7f8354197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:05:57.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f8354197070 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f834c00da40 tx=0x7f834c00df00 comp rx=0 tx=0).stop 2024-02-14T15:05:57.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 shutdown_connections 2024-02-14T15:05:57.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f83440a2230 0x7f83440a46f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83541050c0 0x7f8354197070 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8354101200 0x7f8354196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 --2- 172.21.15.17:0/4294856673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8354100860 0x7f83541965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:05:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 >> 172.21.15.17:0/4294856673 conn(0x7f83540776a0 msgr2=0x7f8354103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:05:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 shutdown_connections 2024-02-14T15:05:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:05:57.721+0000 7f835ab2c640 1 -- 172.21.15.17:0/4294856673 wait complete. 2024-02-14T15:05:57.734 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 62058807, 'ino': 1099511628327, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 70282672, 'ino': 1099511628342, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 131118889, 'ino': 1099511628310, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 132892387, 'ino': 1099511627783, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients'}, {'damage_type': 'backtrace', 'id': 151978286, 'ino': 1099511628311, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 198235716, 'ino': 1099511628335, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 254722844, 'ino': 1099511628340, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 345384551, 'ino': 1099511628326, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS'}, {'damage_type': 'backtrace', 'id': 421242307, 'ino': 1099511628302, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 634550466, 'ino': 1099511628348, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 700496905, 'ino': 1099511628294, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.003'}, {'damage_type': 'backtrace', 'id': 754216507, 'ino': 1099511628338, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 755351011, 'ino': 1099511628324, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 822254122, 'ino': 1099511628299, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 862507500, 'ino': 1099511628345, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 1035963386, 'ino': 1099511628312, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 1042504449, 'ino': 1099511628318, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 1200305259, 'ino': 1099511628315, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 1228079811, 'ino': 1099511628329, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX'}, {'damage_type': 'backtrace', 'id': 1255107092, 'ino': 1099511628308, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 1337282388, 'ino': 1099511628321, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO'}, {'damage_type': 'backtrace', 'id': 1355102890, 'ino': 1099511628298, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 1407324553, 'ino': 1099511628346, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 1447176037, 'ino': 1099511628343, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 1629560916, 'ino': 1099511628296, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD'}, {'damage_type': 'backtrace', 'id': 1662983673, 'ino': 1099511628301, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 1671995221, 'ino': 1099511627784, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0'}, {'damage_type': 'backtrace', 'id': 1688619646, 'ino': 1099511628303, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT'}, {'damage_type': 'backtrace', 'id': 1920694774, 'ino': 1099511628339, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 2015262532, 'ino': 1099511628349, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL'}, {'damage_type': 'backtrace', 'id': 2079098749, 'ino': 1099511628304, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 2141058533, 'ino': 1099511628314, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 2256613811, 'ino': 1099511628341, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 2308388004, 'ino': 1099511628320, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 2434721239, 'ino': 1099511628325, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM'}, {'damage_type': 'backtrace', 'id': 2460322358, 'ino': 1099511628292, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.001'}, {'damage_type': 'backtrace', 'id': 2756721182, 'ino': 1099511628316, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 2765577132, 'ino': 1099511627785, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp'}, {'damage_type': 'backtrace', 'id': 2772182371, 'ino': 1099511628337, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 2853713412, 'ino': 1099511628351, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 2896672087, 'ino': 1099511628309, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL'}, {'damage_type': 'backtrace', 'id': 2906779718, 'ino': 1099511628293, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.002'}, {'damage_type': 'backtrace', 'id': 2967929424, 'ino': 1099511628313, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 2985393586, 'ino': 1099511628323, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 3015502775, 'ino': 1099511628354, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 3062645563, 'ino': 1099511628355, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 3142125753, 'ino': 1099511628333, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 3170141149, 'ino': 1099511628328, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 3192401249, 'ino': 1099511628322, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 3356741973, 'ino': 1099511628300, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/EXCEL'}, {'damage_type': 'backtrace', 'id': 3426846455, 'ino': 1099511628305, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 3471131228, 'ino': 1099511627786, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.000'}, {'damage_type': 'backtrace', 'id': 3480715737, 'ino': 1099511628330, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 3579274792, 'ino': 1099511628352, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED'}, {'damage_type': 'backtrace', 'id': 3688877219, 'ino': 1099511628317, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 3708698930, 'ino': 1099511628353, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL'}, {'damage_type': 'backtrace', 'id': 3747691089, 'ino': 1099511628347, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 3827904488, 'ino': 1099511628307, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT'}, {'damage_type': 'backtrace', 'id': 3946849072, 'ino': 1099511628306, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 3990271225, 'ino': 1099511628332, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 4005225290, 'ino': 1099511628350, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/ACCESS'}, {'damage_type': 'backtrace', 'id': 4038077303, 'ino': 1099511628344, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 4065279409, 'ino': 1099511628336, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 4075018131, 'ino': 1099511628331, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX'}, {'damage_type': 'backtrace', 'id': 4095022917, 'ino': 1099511628334, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 4133363028, 'ino': 1099511628319, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 4269652947, 'ino': 1099511628295, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/filler.004'}, {'damage_type': 'backtrace', 'id': 4270378812, 'ino': 1099511628297, 'path': '/volumes/qa/sv_1/5a1981ee-3c8b-4fe5-965e-923670054a4f/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC'}] 2024-02-14T15:05:57.734 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-02-14T15:05:57.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 115933 10.85 MB/sec warmup 59 sec latency 16.217 ms 2024-02-14T15:05:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:57 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2005640829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:05:58.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:57 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2005640829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:05:58.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:57 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2005640829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:05:58.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 118148 10.90 MB/sec warmup 60 sec latency 25.737 ms 2024-02-14T15:05:58.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:05:58 smithi076 ceph-mon[37057]: pgmap v79: 129 pgs: 129 active+clean; 72 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.5 MiB/s wr, 314 op/s 2024-02-14T15:05:59.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:05:58 smithi017 ceph-mon[32549]: pgmap v79: 129 pgs: 129 active+clean; 72 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.5 MiB/s wr, 314 op/s 2024-02-14T15:05:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:05:58 smithi050 ceph-mon[36512]: pgmap v79: 129 pgs: 129 active+clean; 72 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.5 MiB/s wr, 314 op/s 2024-02-14T15:05:59.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 120976 11.00 MB/sec warmup 61 sec latency 88.729 ms 2024-02-14T15:06:00.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 122897 10.92 MB/sec warmup 62 sec latency 105.048 ms 2024-02-14T15:06:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:00 smithi076 ceph-mon[37057]: pgmap v80: 129 pgs: 129 active+clean; 72 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-02-14T15:06:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:00 smithi017 ceph-mon[32549]: pgmap v80: 129 pgs: 129 active+clean; 72 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-02-14T15:06:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:00 smithi050 ceph-mon[36512]: pgmap v80: 129 pgs: 129 active+clean; 72 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-02-14T15:06:01.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 124140 10.93 MB/sec warmup 63 sec latency 86.380 ms 2024-02-14T15:06:02.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 126364 10.88 MB/sec warmup 64 sec latency 16.997 ms 2024-02-14T15:06:02.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:02 smithi076 ceph-mon[37057]: pgmap v81: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-02-14T15:06:03.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:02 smithi017 ceph-mon[32549]: pgmap v81: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-02-14T15:06:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:02 smithi050 ceph-mon[36512]: pgmap v81: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.6 MiB/s wr, 367 op/s 2024-02-14T15:06:03.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 128626 10.93 MB/sec warmup 65 sec latency 14.104 ms 2024-02-14T15:06:04.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 131605 11.03 MB/sec warmup 66 sec latency 13.631 ms 2024-02-14T15:06:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:04 smithi076 ceph-mon[37057]: pgmap v82: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-02-14T15:06:05.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:04 smithi017 ceph-mon[32549]: pgmap v82: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-02-14T15:06:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:04 smithi050 ceph-mon[36512]: pgmap v82: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-02-14T15:06:05.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 133859 10.99 MB/sec warmup 67 sec latency 115.366 ms 2024-02-14T15:06:06.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 135747 11.01 MB/sec warmup 68 sec latency 14.474 ms 2024-02-14T15:06:06.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:06 smithi076 ceph-mon[37057]: pgmap v83: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-02-14T15:06:07.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:06 smithi017 ceph-mon[32549]: pgmap v83: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-02-14T15:06:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:06 smithi050 ceph-mon[36512]: pgmap v83: 129 pgs: 129 active+clean; 78 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-02-14T15:06:07.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 138799 11.11 MB/sec warmup 69 sec latency 14.621 ms 2024-02-14T15:06:08.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 141921 11.20 MB/sec warmup 70 sec latency 6.540 ms 2024-02-14T15:06:08.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:08 smithi076 ceph-mon[37057]: pgmap v84: 129 pgs: 129 active+clean; 84 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-02-14T15:06:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:08 smithi017 ceph-mon[32549]: pgmap v84: 129 pgs: 129 active+clean; 84 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-02-14T15:06:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:08 smithi050 ceph-mon[36512]: pgmap v84: 129 pgs: 129 active+clean; 84 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-02-14T15:06:09.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 144257 11.17 MB/sec warmup 71 sec latency 13.174 ms 2024-02-14T15:06:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 -- 172.21.15.17:0/1270533715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c740696f0 msgr2=0x7f3c74105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 --2- 172.21.15.17:0/1270533715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c740696f0 0x7f3c74105e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:06:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 -- 172.21.15.17:0/1270533715 shutdown_connections 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 --2- 172.21.15.17:0/1270533715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c74106510 0x7f3c7410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 --2- 172.21.15.17:0/1270533715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c740696f0 0x7f3c74105e30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 --2- 172.21.15.17:0/1270533715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 0x7f3c74069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 -- 172.21.15.17:0/1270533715 >> 172.21.15.17:0/1270533715 conn(0x7f3c74077220 msgr2=0x7f3c74077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 -- 172.21.15.17:0/1270533715 shutdown_connections 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.701+0000 7f3c7906f640 1 -- 172.21.15.17:0/1270533715 wait complete. 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 Processor -- start 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 -- start start 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 0x7f3c7419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c74106510 0x7f3c7419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c74197670 0x7f3c74197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74198140 con 0x7f3c74068d40 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c741982b0 con 0x7f3c74197670 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7906f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74198470 con 0x7f3c74106510 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 0x7f3c7419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 0x7f3c7419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43308/0 (socket says 172.21.15.17:43308) 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c72d9d640 1 -- 172.21.15.17:0/3898566640 learned_addr learned my addr 172.21.15.17:0/3898566640 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c74106510 0x7f3c7419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.702+0000 7f3c7359e640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c74197670 0x7f3c74197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c6bfff640 1 -- 172.21.15.17:0/3898566640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c74197670 msgr2=0x7f3c74197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c74197670 0x7f3c74197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c6bfff640 1 -- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 msgr2=0x7f3c7419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 0x7f3c7419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c6bfff640 1 -- 172.21.15.17:0/3898566640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c009560 con 0x7f3c74106510 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 0x7f3c7419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c7359e640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c74197670 0x7f3c74197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c74106510 0x7f3c7419daf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c036040 tx=0x7f3c5c02c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c70d99640 1 -- 172.21.15.17:0/3898566640 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c004350 con 0x7f3c74106510 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c70d99640 1 -- 172.21.15.17:0/3898566640 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c5c0044b0 con 0x7f3c74106510 2024-02-14T15:06:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c741a43b0 con 0x7f3c74106510 2024-02-14T15:06:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c70d99640 1 -- 172.21.15.17:0/3898566640 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c003cc0 con 0x7f3c74106510 2024-02-14T15:06:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.703+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c741a48e0 con 0x7f3c74106510 2024-02-14T15:06:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.705+0000 7f3c70d99640 1 -- 172.21.15.17:0/3898566640 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3c5c005560 con 0x7f3c74106510 2024-02-14T15:06:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 Processor -- start 2024-02-14T15:06:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 -- start start 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c70d99640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3c600a22a0 0x7f3c600a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a55b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a61010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d39c39e0 con 0x5628d3a55720 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d39c0f10 con 0x5628d3a5ec20 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d39bc6f0 con 0x5628d3a56040 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a55b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a55b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43314/0 (socket says 172.21.15.17:43314) 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c72d9d640 1 -- 172.21.15.17:0/1752515726 learned_addr learned my addr 172.21.15.17:0/1752515726 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c70d99640 1 -- 172.21.15.17:0/3898566640 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f3c5c0f0350 con 0x7f3c74106510 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.706+0000 7f3c7359e640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3c600a22a0 0x7f3c600a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c72d9d640 1 -- 172.21.15.17:0/1752515726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 msgr2=0x5628d3a5e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c7359e640 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a61010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c72d9d640 1 -- 172.21.15.17:0/1752515726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 msgr2=0x5628d3a61010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a61010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c72d9d640 1 -- 172.21.15.17:0/1752515726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628d36cd920 con 0x5628d3a55720 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c7359e640 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a61010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a55b00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c64011650 tx=0x7f3c64009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d830ad7d6a9680 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c7359e640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3c600a22a0 0x7f3c600a4760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5628d3a50fe0 tx=0x7f3c6c0091e0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c697fa640 1 -- 172.21.15.17:0/1752515726 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400f6c0 con 0x5628d3a55720 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c697fa640 1 -- 172.21.15.17:0/1752515726 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c6400fd00 con 0x5628d3a55720 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.707+0000 7f3c697fa640 1 -- 172.21.15.17:0/1752515726 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64019e20 con 0x5628d3a55720 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1752515726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 msgr2=0x5628d3a55b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a55b00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c64011650 tx=0x7f3c64009f90 comp rx=0 tx=0).stop 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1752515726 shutdown_connections 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a61010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1752515726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a55b00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1752515726 >> 172.21.15.17:0/1752515726 conn(0x5628d3a4fe50 msgr2=0x5628d3a50230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1752515726 shutdown_connections 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1752515726 wait complete. 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 -- start start 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.708+0000 7f3c7bc42740 1 Processor -- start 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a5be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3958fa0 con 0x5628d3a55720 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3690250 con 0x5628d3a5ec20 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3942200 con 0x5628d3a56040 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7bc42740 10 client.0 _ll_register_callbacks cb 0x5628d39b1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43318/0 (socket says 172.21.15.17:43318) 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 -- 172.21.15.17:0/611779285 learned_addr learned my addr 172.21.15.17:0/611779285 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c7359e640 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a5be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 -- 172.21.15.17:0/611779285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 msgr2=0x5628d3a5b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 -- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 msgr2=0x5628d3a5be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a5be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 -- 172.21.15.17:0/611779285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c64009e00 con 0x5628d3a55720 2024-02-14T15:06:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5e580 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3c6401c040 tx=0x7f3c6401ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7028fde365f3abb server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.709+0000 7f3c5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.710+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/611779285 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64019780 con 0x5628d3a55720 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.710+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/611779285 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f3c64004250 con 0x5628d3a55720 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.710+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628d36bc600 con 0x5628d3a55720 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/611779285 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c64004410 con 0x5628d3a55720 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 10 client.16506 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5628d39bad70 con 0x5628d3a55720 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/611779285 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5628d36bc600 con 0x5628d3a55720 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/611779285 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3c64003c60 con 0x5628d3a55720 2024-02-14T15:06:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 10 client.16506 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 20 client.16506 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3a6e810 0x5628d3a70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 4 client.16506 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628d370d1b0 con 0x5628d3a6e810 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.711+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3a6e810 0x5628d3a70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3a6e810 0x5628d3a70c30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab79787e39c80b80 server_cookie=901ce91473a33ccb in_seq=0 out_seq=0 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c5a7fc640 10 client.16506 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/611779285 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628d370d1b0 con 0x5628d3a6e810 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c5a7fc640 10 client.16506 handle_command_reply: tid=0 2024-02-14T15:06:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 client.16506 shutdown 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 msgr2=0x5628d3a5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5e580 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3c6401c040 tx=0x7f3c6401ae20 comp rx=0 tx=0).stop 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 shutdown_connections 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3a6e810 0x5628d3a70c30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a5be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a56040 0x5628d3a5b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/611779285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5e580 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c5a7fc640 0 client.16506 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.712+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 >> 172.21.15.17:0/611779285 conn(0x5628d3a4fe50 msgr2=0x5628d3a51ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.713+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 shutdown_connections 2024-02-14T15:06:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.713+0000 7f3c7bc42740 1 -- 172.21.15.17:0/611779285 wait complete. 2024-02-14T15:06:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.713+0000 7f3c7bc42740 20 client.16506 trim_cache size 0 max 16384 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.726+0000 7f3c7bc42740 1 Processor -- start 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.726+0000 7f3c7bc42740 1 -- start start 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a56040 0x5628d3a5e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a5ec20 0x5628d3a5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3a4e2d0 con 0x5628d3a55720 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3a4fcb0 con 0x5628d3a56040 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3a4b090 con 0x5628d3a5ec20 2024-02-14T15:06:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43326/0 (socket says 172.21.15.17:43326) 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 -- 172.21.15.17:0/2049180790 learned_addr learned my addr 172.21.15.17:0/2049180790 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c7359e640 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a5ec20 0x5628d3a5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a56040 0x5628d3a5e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 -- 172.21.15.17:0/2049180790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a5ec20 msgr2=0x5628d3a5e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a5ec20 0x5628d3a5e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 -- 172.21.15.17:0/2049180790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a56040 msgr2=0x5628d3a5e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a56040 0x5628d3a5e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 -- 172.21.15.17:0/2049180790 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628d3957820 con 0x5628d3a55720 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5b8b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3c64001eb0 tx=0x7f3c640114d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e8a655a2609cff server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:10.734 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c68ff9640 1 -- 172.21.15.17:0/2049180790 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400f3c0 con 0x5628d3a55720 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c68ff9640 1 -- 172.21.15.17:0/2049180790 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c640052f0 con 0x5628d3a55720 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.727+0000 7f3c68ff9640 1 -- 172.21.15.17:0/2049180790 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64022c80 con 0x5628d3a55720 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 -- 172.21.15.17:0/2049180790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 msgr2=0x5628d3a5b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5b8b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3c64001eb0 tx=0x7f3c640114d0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 -- 172.21.15.17:0/2049180790 shutdown_connections 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a5ec20 0x5628d3a5e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a56040 0x5628d3a5e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/2049180790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a55720 0x5628d3a5b8b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 -- 172.21.15.17:0/2049180790 >> 172.21.15.17:0/2049180790 conn(0x5628d3a4fe50 msgr2=0x5628d3a5b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 -- 172.21.15.17:0/2049180790 shutdown_connections 2024-02-14T15:06:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.728+0000 7f3c7bc42740 1 -- 172.21.15.17:0/2049180790 wait complete. 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 -- start start 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 Processor -- start 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a55720 0x5628d3a65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a56040 0x5628d3a64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3a64e60 con 0x5628d3a56040 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3a64fd0 con 0x5628d3a5ec20 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d3ac4d60 con 0x5628d3a55720 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7bc42740 10 client.0 _ll_register_callbacks cb 0x7f3c64009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a56040 0x5628d3a64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a56040 0x5628d3a64220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43340/0 (socket says 172.21.15.17:43340) 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 -- 172.21.15.17:0/1042277349 learned_addr learned my addr 172.21.15.17:0/1042277349 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a55720 0x5628d3a65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c7359e640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 -- 172.21.15.17:0/1042277349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a55720 msgr2=0x5628d3a65e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a55720 0x5628d3a65e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 -- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 msgr2=0x5628d3a64760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a64760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c6bfff640 1 -- 172.21.15.17:0/1042277349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628d39bad70 con 0x5628d3a56040 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.729+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a55720 0x5628d3a65e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c6bfff640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a56040 0x5628d3a64220 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c02cdd0 tx=0x7f3c5c0bc990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cbcce62ec4a974 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/1042277349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0bca90 con 0x5628d3a56040 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/1042277349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f3c5c0ed570 con 0x5628d3a56040 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c6400fd00 con 0x5628d3a56040 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/1042277349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c5c0c0be0 con 0x5628d3a56040 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 10 client.16512 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5628d397d720 con 0x5628d3a56040 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/1042277349 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c6400fd00 con 0x5628d3a56040 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/1042277349 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3c5c03c070 con 0x5628d3a56040 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 10 client.16512 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 20 client.16512 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3ac62e0 0x5628d3ac8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 4 client.16512 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:06:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5628d36f9550 con 0x5628d3ac62e0 2024-02-14T15:06:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.730+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3ac62e0 0x5628d3ac8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.731+0000 7f3c72d9d640 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3ac62e0 0x5628d3ac8700 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f70d1c471f4e443 server_cookie=a22a301676cfdbe3 in_seq=0 out_seq=0 2024-02-14T15:06:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.731+0000 7f3c5a7fc640 10 client.16512 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c5a7fc640 1 -- 172.21.15.17:0/1042277349 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5628d36f9550 con 0x5628d3ac62e0 2024-02-14T15:06:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c5a7fc640 10 client.16512 handle_command_reply: tid=0 2024-02-14T15:06:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 client.16512 shutdown 2024-02-14T15:06:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a56040 msgr2=0x5628d3a64220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a56040 0x5628d3a64220 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c02cdd0 tx=0x7f3c5c0bc990 comp rx=0 tx=0).stop 2024-02-14T15:06:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 shutdown_connections 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5628d3ac62e0 0x5628d3ac8700 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5628d3a56040 0x5628d3a64220 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5628d3a5ec20 0x5628d3a64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c5a7fc640 0 client.16512 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 --2- 172.21.15.17:0/1042277349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5628d3a55720 0x5628d3a65e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 >> 172.21.15.17:0/1042277349 conn(0x5628d3a4fe50 msgr2=0x5628d3a4e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 shutdown_connections 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 1 -- 172.21.15.17:0/1042277349 wait complete. 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.738+0000 7f3c7bc42740 20 client.16512 trim_cache size 0 max 16384 2024-02-14T15:06:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:06:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:06:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:06:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:06:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3c600a22a0 msgr2=0x7f3c600a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3c600a22a0 0x7f3c600a4760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5628d3a50fe0 tx=0x7f3c6c0091e0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c74106510 msgr2=0x7f3c7419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c74106510 0x7f3c7419daf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c036040 tx=0x7f3c5c02c680 comp rx=0 tx=0).stop 2024-02-14T15:06:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 shutdown_connections 2024-02-14T15:06:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3c600a22a0 0x7f3c600a4760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c74197670 0x7f3c74197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c74106510 0x7f3c7419daf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 --2- 172.21.15.17:0/3898566640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c74068d40 0x7f3c7419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 >> 172.21.15.17:0/3898566640 conn(0x7f3c74077220 msgr2=0x7f3c740fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 shutdown_connections 2024-02-14T15:06:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:10.739+0000 7f3c7906f640 1 -- 172.21.15.17:0/3898566640 wait complete. 2024-02-14T15:06:10.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 146126 11.18 MB/sec warmup 72 sec latency 100.852 ms 2024-02-14T15:06:10.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:10 smithi076 ceph-mon[37057]: pgmap v85: 129 pgs: 129 active+clean; 84 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-02-14T15:06:11.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:10 smithi017 ceph-mon[32549]: pgmap v85: 129 pgs: 129 active+clean; 84 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-02-14T15:06:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:10 smithi050 ceph-mon[36512]: pgmap v85: 129 pgs: 129 active+clean; 84 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-02-14T15:06:11.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 147845 11.12 MB/sec warmup 73 sec latency 168.055 ms 2024-02-14T15:06:12.862 INFO:tasks.workunit.client.0.smithi017.stdout: 1 149974 11.15 MB/sec warmup 74 sec latency 9.214 ms 2024-02-14T15:06:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:12 smithi076 ceph-mon[37057]: pgmap v86: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 380 op/s 2024-02-14T15:06:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:12 smithi017 ceph-mon[32549]: pgmap v86: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 380 op/s 2024-02-14T15:06:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:12 smithi050 ceph-mon[36512]: pgmap v86: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 380 op/s 2024-02-14T15:06:13.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 152930 11.24 MB/sec warmup 75 sec latency 16.258 ms 2024-02-14T15:06:14.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 155919 11.29 MB/sec warmup 76 sec latency 15.200 ms 2024-02-14T15:06:14.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:14 smithi076 ceph-mon[37057]: pgmap v87: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-02-14T15:06:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:14 smithi017 ceph-mon[32549]: pgmap v87: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-02-14T15:06:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:14 smithi050 ceph-mon[36512]: pgmap v87: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-02-14T15:06:15.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 157742 11.27 MB/sec warmup 77 sec latency 76.007 ms 2024-02-14T15:06:16.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 160057 11.30 MB/sec warmup 78 sec latency 58.906 ms 2024-02-14T15:06:16.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:16 smithi076 ceph-mon[37057]: pgmap v88: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-02-14T15:06:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:16 smithi017 ceph-mon[32549]: pgmap v88: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-02-14T15:06:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:16 smithi050 ceph-mon[36512]: pgmap v88: 129 pgs: 129 active+clean; 98 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-02-14T15:06:17.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 163130 11.38 MB/sec warmup 79 sec latency 13.516 ms 2024-02-14T15:06:18.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 165210 11.33 MB/sec warmup 80 sec latency 21.778 ms 2024-02-14T15:06:18.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:18 smithi076 ceph-mon[37057]: pgmap v89: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 357 op/s 2024-02-14T15:06:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:18 smithi017 ceph-mon[32549]: pgmap v89: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 357 op/s 2024-02-14T15:06:19.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:18 smithi050 ceph-mon[36512]: pgmap v89: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 357 op/s 2024-02-14T15:06:19.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 167470 11.36 MB/sec warmup 81 sec latency 16.283 ms 2024-02-14T15:06:20.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 170372 11.43 MB/sec warmup 82 sec latency 77.406 ms 2024-02-14T15:06:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:20 smithi076 ceph-mon[37057]: pgmap v90: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 229 op/s 2024-02-14T15:06:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:20 smithi017 ceph-mon[32549]: pgmap v90: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 229 op/s 2024-02-14T15:06:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:20 smithi050 ceph-mon[36512]: pgmap v90: 129 pgs: 129 active+clean; 104 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 229 op/s 2024-02-14T15:06:21.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 172609 11.39 MB/sec warmup 83 sec latency 123.523 ms 2024-02-14T15:06:22.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 174630 11.41 MB/sec warmup 84 sec latency 17.294 ms 2024-02-14T15:06:22.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:22 smithi076 ceph-mon[37057]: pgmap v91: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-02-14T15:06:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:22 smithi017 ceph-mon[32549]: pgmap v91: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-02-14T15:06:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:22 smithi050 ceph-mon[36512]: pgmap v91: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-02-14T15:06:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/2337874499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8106530 msgr2=0x7fe3c810e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/2337874499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8106530 0x7fe3c810e100 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009cd0 tx=0x7fe3c002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/2337874499 shutdown_connections 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/2337874499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8106530 0x7fe3c810e100 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/2337874499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3c8101e10 0x7fe3c8105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/2337874499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c8101460 0x7fe3c8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/2337874499 >> 172.21.15.17:0/2337874499 conn(0x7fe3c80fccc0 msgr2=0x7fe3c80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/2337874499 shutdown_connections 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.834+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/2337874499 wait complete. 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 Processor -- start 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 -- start start 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8101460 0x7fe3c819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c8106530 0x7fe3c819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3c8197780 0x7fe3c8197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8198250 con 0x7fe3c8101460 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c81983c0 con 0x7fe3c8106530 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cf8a3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8198580 con 0x7fe3c8197780 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8101460 0x7fe3c819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8101460 0x7fe3c819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49362/0 (socket says 172.21.15.17:49362) 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 -- 172.21.15.17:0/3247840673 learned_addr learned my addr 172.21.15.17:0/3247840673 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cde40640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3c8197780 0x7fe3c8197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c8106530 0x7fe3c819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 -- 172.21.15.17:0/3247840673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3c8197780 msgr2=0x7fe3c8197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3c8197780 0x7fe3c8197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 -- 172.21.15.17:0/3247840673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c8106530 msgr2=0x7fe3c819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c8106530 0x7fe3c819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 -- 172.21.15.17:0/3247840673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009900 con 0x7fe3c8101460 2024-02-14T15:06:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.835+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8101460 0x7fe3c819d670 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400e9f0 tx=0x7fe3c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.836+0000 7fe3be7fc640 1 -- 172.21.15.17:0/3247840673 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4009700 con 0x7fe3c8101460 2024-02-14T15:06:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.836+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c81a44d0 con 0x7fe3c8101460 2024-02-14T15:06:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.836+0000 7fe3be7fc640 1 -- 172.21.15.17:0/3247840673 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c400cd80 con 0x7fe3c8101460 2024-02-14T15:06:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.836+0000 7fe3be7fc640 1 -- 172.21.15.17:0/3247840673 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4018920 con 0x7fe3c8101460 2024-02-14T15:06:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.836+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c81a4a00 con 0x7fe3c8101460 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.838+0000 7fe3be7fc640 1 -- 172.21.15.17:0/3247840673 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe3c4018a80 con 0x7fe3c8101460 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 Processor -- start 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 -- start start 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3be7fc640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe3b40a2220 0x7fe3b40a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3cde40640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe3b40a2220 0x7fe3b40a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x558895539b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x5588955426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895545010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3be7fc640 1 -- 172.21.15.17:0/3247840673 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe3c4014070 con 0x7fe3c8101460 2024-02-14T15:06:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588954a79e0 con 0x558895539720 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588954a4f10 con 0x558895542c20 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3d2471740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588954a06f0 con 0x55889553a040 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x558895539b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3cce3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x5588955426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x558895539b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49372/0 (socket says 172.21.15.17:49372) 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3cd63f640 1 -- 172.21.15.17:0/1743626606 learned_addr learned my addr 172.21.15.17:0/1743626606 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.839+0000 7fe3cde40640 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895545010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cd63f640 1 -- 172.21.15.17:0/1743626606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 msgr2=0x5588955426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x5588955426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cd63f640 1 -- 172.21.15.17:0/1743626606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 msgr2=0x558895545010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895545010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cd63f640 1 -- 172.21.15.17:0/1743626606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588951b1920 con 0x558895539720 2024-02-14T15:06:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x5588955426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cde40640 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895545010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cde40640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe3b40a2220 0x7fe3b40a46e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005cc0 tx=0x7fe3c0032ef0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x558895539b00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40a3120 tx=0x7fe3c409a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8008e92a91e674a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3b37fe640 1 -- 172.21.15.17:0/1743626606 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40a3650 con 0x558895539720 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3b37fe640 1 -- 172.21.15.17:0/1743626606 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c40991b0 con 0x558895539720 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.840+0000 7fe3b37fe640 1 -- 172.21.15.17:0/1743626606 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c409a300 con 0x558895539720 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 -- 172.21.15.17:0/1743626606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 msgr2=0x558895539b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x558895539b00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40a3120 tx=0x7fe3c409a8d0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 -- 172.21.15.17:0/1743626606 shutdown_connections 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895545010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x5588955426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 --2- 172.21.15.17:0/1743626606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x558895539b00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 -- 172.21.15.17:0/1743626606 >> 172.21.15.17:0/1743626606 conn(0x558895533e50 msgr2=0x558895534230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 -- 172.21.15.17:0/1743626606 shutdown_connections 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 -- 172.21.15.17:0/1743626606 wait complete. 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 -- start start 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.841+0000 7fe3d2471740 1 Processor -- start 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895539720 0x558895542620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x55889553f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895542c20 0x55889553ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3d2471740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55889543cfa0 con 0x55889553a040 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3d2471740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558895174250 con 0x558895539720 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3d2471740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558895426200 con 0x558895542c20 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3d2471740 10 client.0 _ll_register_callbacks cb 0x558895495b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x55889553f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895539720 0x558895542620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x55889553f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49390/0 (socket says 172.21.15.17:49390) 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895539720 0x558895542620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:45080/0 (socket says 172.21.15.17:45080) 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 -- 172.21.15.17:0/317068002 learned_addr learned my addr 172.21.15.17:0/317068002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cde40640 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895542c20 0x55889553ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 -- 172.21.15.17:0/317068002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895542c20 msgr2=0x55889553ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895542c20 0x55889553ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 -- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895539720 msgr2=0x558895542620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895539720 0x558895542620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 -- 172.21.15.17:0/317068002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c81a4a00 con 0x55889553a040 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895539720 0x558895542620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x55889553f9b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8012700 tx=0x7fe3b8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd703fe5d3288db4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.842+0000 7fe3b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/317068002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8017070 con 0x55889553a040 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588951a0600 con 0x55889553a040 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/317068002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe3b8004260 con 0x55889553a040 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/317068002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3b8004420 con 0x55889553a040 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 10 client.16524 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55889549ed70 con 0x55889553a040 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/317068002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5588951a0600 con 0x55889553a040 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/317068002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe3b8003cf0 con 0x55889553a040 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 10 client.16524 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 20 client.16524 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558895552ae0 0x558895554f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 4 client.16524 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588951f11b0 con 0x558895552ae0 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.843+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558895552ae0 0x558895554f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.844+0000 7fe3b0ff9640 10 client.16524 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.844+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558895552ae0 0x558895554f00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38a42632deffe124 server_cookie=ab3295655f14dca3 in_seq=0 out_seq=0 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.844+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/317068002 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588951f11b0 con 0x558895552ae0 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.844+0000 7fe3b0ff9640 10 client.16524 handle_command_reply: tid=0 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.844+0000 7fe3d2471740 1 client.16524 shutdown 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.844+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 msgr2=0x55889553f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.844+0000 7fe3d2471740 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x55889553f9b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8012700 tx=0x7fe3b8012bc0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 shutdown_connections 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558895552ae0 0x558895554f00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x55889553f9b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895542c20 0x55889553ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 --2- 172.21.15.17:0/317068002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895539720 0x558895542620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3b0ff9640 0 client.16524 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 >> 172.21.15.17:0/317068002 conn(0x558895533e50 msgr2=0x558895535ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 shutdown_connections 2024-02-14T15:06:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 1 -- 172.21.15.17:0/317068002 wait complete. 2024-02-14T15:06:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.845+0000 7fe3d2471740 20 client.16524 trim_cache size 0 max 16384 2024-02-14T15:06:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 Processor -- start 2024-02-14T15:06:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 -- start start 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x5588955420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x558895542620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x55889553f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55889552f0e0 con 0x558895539720 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558895532300 con 0x558895542c20 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3d2471740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55889553df30 con 0x55889553a040 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x5588955420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3cce3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x558895542620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3cd63f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x5588955420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49394/0 (socket says 172.21.15.17:49394) 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3cd63f640 1 -- 172.21.15.17:0/2690565560 learned_addr learned my addr 172.21.15.17:0/2690565560 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.859+0000 7fe3cde40640 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x55889553f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3cd63f640 1 -- 172.21.15.17:0/2690565560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 msgr2=0x558895542620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x558895542620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3cd63f640 1 -- 172.21.15.17:0/2690565560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 msgr2=0x55889553f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x55889553f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3cd63f640 1 -- 172.21.15.17:0/2690565560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55889543b820 con 0x558895539720 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x558895542620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x5588955420e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4097310 tx=0x7fe3c409aff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432d5f5922e9663b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3b2ffd640 1 -- 172.21.15.17:0/2690565560 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4010b80 con 0x558895539720 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3b2ffd640 1 -- 172.21.15.17:0/2690565560 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c40cbe90 con 0x558895539720 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.860+0000 7fe3b2ffd640 1 -- 172.21.15.17:0/2690565560 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c409a300 con 0x558895539720 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 -- 172.21.15.17:0/2690565560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 msgr2=0x5588955420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x5588955420e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4097310 tx=0x7fe3c409aff0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 -- 172.21.15.17:0/2690565560 shutdown_connections 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x55889553f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55889553a040 0x558895542620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 --2- 172.21.15.17:0/2690565560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558895539720 0x5588955420e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 -- 172.21.15.17:0/2690565560 >> 172.21.15.17:0/2690565560 conn(0x558895533e50 msgr2=0x55889553e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 -- 172.21.15.17:0/2690565560 shutdown_connections 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 -- 172.21.15.17:0/2690565560 wait complete. 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 -- start start 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 Processor -- start 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895539720 0x558895549780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x558895549cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.861+0000 7fe3d2471740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895547ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3d2471740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558895548460 con 0x55889553a040 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3d2471740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588955485d0 con 0x558895542c20 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3d2471740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558895548790 con 0x558895539720 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3d2471740 10 client.0 _ll_register_callbacks cb 0x7fe3c4096a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x558895549cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cde40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895547ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x558895549cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49396/0 (socket says 172.21.15.17:49396) 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 -- 172.21.15.17:0/3304980479 learned_addr learned my addr 172.21.15.17:0/3304980479 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895539720 0x558895549780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 -- 172.21.15.17:0/3304980479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895539720 msgr2=0x558895549780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895539720 0x558895549780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 -- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 msgr2=0x558895547ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895547ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 -- 172.21.15.17:0/3304980479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b8021040 con 0x55889553a040 2024-02-14T15:06:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cde40640 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895547ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3cce3e640 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x558895549cc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x558895540f80 tx=0x7fe3b8005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bb9961cfba9a19 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:06:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/3304980479 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8017070 con 0x55889553a040 2024-02-14T15:06:23.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 177647 11.49 MB/sec warmup 85 sec latency 8.043 ms 2024-02-14T15:06:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/3304980479 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fe3b8009c70 con 0x55889553a040 2024-02-14T15:06:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3c40991b0 con 0x55889553a040 2024-02-14T15:06:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/3304980479 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3b8011db0 con 0x55889553a040 2024-02-14T15:06:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.862+0000 7fe3d2471740 10 client.16530 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588951b1920 con 0x55889553a040 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/3304980479 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3c40991b0 con 0x55889553a040 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/3304980479 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe3b80038f0 con 0x55889553a040 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3d2471740 10 client.16530 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3d2471740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3d2471740 20 client.16530 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3d2471740 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5588955aa180 0x5588955ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3d2471740 4 client.16530 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588951dd550 con 0x5588955aa180 2024-02-14T15:06:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5588955aa180 0x5588955ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3b0ff9640 10 client.16530 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.863+0000 7fe3cd63f640 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5588955aa180 0x5588955ac5a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b607fc22b478e96 server_cookie=52e5283e39d6f932 in_seq=0 out_seq=0 2024-02-14T15:06:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3b0ff9640 1 -- 172.21.15.17:0/3304980479 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588951dd550 con 0x5588955aa180 2024-02-14T15:06:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3b0ff9640 10 client.16530 handle_command_reply: tid=0 2024-02-14T15:06:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 client.16530 shutdown 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 msgr2=0x558895549cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x558895549cc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x558895540f80 tx=0x7fe3b8005580 comp rx=0 tx=0).stop 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 shutdown_connections 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5588955aa180 0x5588955ac5a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55889553a040 0x558895549cc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558895542c20 0x558895547ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 --2- 172.21.15.17:0/3304980479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558895539720 0x558895549780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3b0ff9640 0 client.16530 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 >> 172.21.15.17:0/3304980479 conn(0x558895533e50 msgr2=0x55889553e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 shutdown_connections 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 1 -- 172.21.15.17:0/3304980479 wait complete. 2024-02-14T15:06:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.871+0000 7fe3d2471740 20 client.16530 trim_cache size 0 max 16384 2024-02-14T15:06:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:06:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:06:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:06:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:06:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe3b40a2220 msgr2=0x7fe3b40a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe3b40a2220 0x7fe3b40a46e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005cc0 tx=0x7fe3c0032ef0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8101460 msgr2=0x7fe3c819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8101460 0x7fe3c819d670 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400e9f0 tx=0x7fe3c400eeb0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 shutdown_connections 2024-02-14T15:06:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe3b40a2220 0x7fe3b40a46e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3c8197780 0x7fe3c8197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c8106530 0x7fe3c819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 --2- 172.21.15.17:0/3247840673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c8101460 0x7fe3c819d670 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 >> 172.21.15.17:0/3247840673 conn(0x7fe3c80fccc0 msgr2=0x7fe3c80fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 shutdown_connections 2024-02-14T15:06:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:23.872+0000 7fe3cf8a3640 1 -- 172.21.15.17:0/3247840673 wait complete. 2024-02-14T15:06:24.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 180374 11.51 MB/sec warmup 86 sec latency 14.559 ms 2024-02-14T15:06:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:24 smithi076 ceph-mon[37057]: pgmap v92: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-02-14T15:06:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:24 smithi017 ceph-mon[32549]: pgmap v92: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-02-14T15:06:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:24 smithi050 ceph-mon[36512]: pgmap v92: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-02-14T15:06:25.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 182783 11.51 MB/sec warmup 87 sec latency 12.903 ms 2024-02-14T15:06:26.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 184941 11.54 MB/sec warmup 88 sec latency 94.968 ms 2024-02-14T15:06:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:26 smithi076 ceph-mon[37057]: pgmap v93: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-02-14T15:06:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:26 smithi017 ceph-mon[32549]: pgmap v93: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-02-14T15:06:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:26 smithi050 ceph-mon[36512]: pgmap v93: 129 pgs: 129 active+clean; 110 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-02-14T15:06:27.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 187805 11.57 MB/sec warmup 89 sec latency 54.692 ms 2024-02-14T15:06:28.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 190097 11.56 MB/sec warmup 90 sec latency 8.360 ms 2024-02-14T15:06:28.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:28 smithi076 ceph-mon[37057]: pgmap v94: 129 pgs: 129 active+clean; 120 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-02-14T15:06:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:28 smithi017 ceph-mon[32549]: pgmap v94: 129 pgs: 129 active+clean; 120 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-02-14T15:06:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:28 smithi050 ceph-mon[36512]: pgmap v94: 129 pgs: 129 active+clean; 120 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.0 MiB/s wr, 361 op/s 2024-02-14T15:06:29.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 192354 11.58 MB/sec warmup 91 sec latency 21.246 ms 2024-02-14T15:06:30.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 195348 11.66 MB/sec warmup 92 sec latency 15.551 ms 2024-02-14T15:06:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:30 smithi076 ceph-mon[37057]: pgmap v95: 129 pgs: 129 active+clean; 120 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 240 op/s 2024-02-14T15:06:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:30 smithi017 ceph-mon[32549]: pgmap v95: 129 pgs: 129 active+clean; 120 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 240 op/s 2024-02-14T15:06:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:30 smithi050 ceph-mon[36512]: pgmap v95: 129 pgs: 129 active+clean; 120 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 240 op/s 2024-02-14T15:06:31.863 INFO:tasks.workunit.client.0.smithi017.stdout: 1 197641 11.62 MB/sec warmup 93 sec latency 51.633 ms 2024-02-14T15:06:32.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 199552 11.62 MB/sec warmup 94 sec latency 27.341 ms 2024-02-14T15:06:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:32 smithi076 ceph-mon[37057]: pgmap v96: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.0 MiB/s wr, 341 op/s 2024-02-14T15:06:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:32 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:06:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:32 smithi017 ceph-mon[32549]: pgmap v96: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.0 MiB/s wr, 341 op/s 2024-02-14T15:06:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:32 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:06:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:32 smithi050 ceph-mon[36512]: pgmap v96: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.0 MiB/s wr, 341 op/s 2024-02-14T15:06:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:32 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:06:33.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 202129 11.68 MB/sec warmup 95 sec latency 304.618 ms 2024-02-14T15:06:33.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:33 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:06:33.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:33 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:06:33.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:33 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:06:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:33 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:06:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:33 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:06:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:33 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:06:34.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:33 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:06:34.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:33 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:06:34.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:33 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:06:34.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 204012 11.63 MB/sec warmup 96 sec latency 353.569 ms 2024-02-14T15:06:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:34 smithi050 ceph-mon[36512]: pgmap v97: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-02-14T15:06:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:34 smithi076 ceph-mon[37057]: pgmap v97: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-02-14T15:06:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:34 smithi017 ceph-mon[32549]: pgmap v97: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-02-14T15:06:35.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 206469 11.65 MB/sec warmup 97 sec latency 11.990 ms 2024-02-14T15:06:36.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 209371 11.71 MB/sec warmup 98 sec latency 57.325 ms 2024-02-14T15:06:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.972+0000 7f432b0e6640 1 -- 172.21.15.17:0/3686608563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 msgr2=0x7f431c006f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.972+0000 7f432b0e6640 1 --2- 172.21.15.17:0/3686608563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 0x7f431c006f40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4314009900 tx=0x7f431402bc30 comp rx=0 tx=0).stop 2024-02-14T15:06:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.972+0000 7f432b0e6640 1 -- 172.21.15.17:0/3686608563 shutdown_connections 2024-02-14T15:06:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.972+0000 7f432b0e6640 1 --2- 172.21.15.17:0/3686608563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 0x7f431c006f40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.972+0000 7f432b0e6640 1 --2- 172.21.15.17:0/3686608563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c098790 0x7f431c098c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.972+0000 7f432b0e6640 1 --2- 172.21.15.17:0/3686608563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c09ac90 0x7f431c0065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.972+0000 7f432b0e6640 1 -- 172.21.15.17:0/3686608563 >> 172.21.15.17:0/3686608563 conn(0x7f431c00a7e0 msgr2=0x7f431c00abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.974+0000 7f432b0e6640 1 -- 172.21.15.17:0/3686608563 shutdown_connections 2024-02-14T15:06:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.974+0000 7f432b0e6640 1 -- 172.21.15.17:0/3686608563 wait complete. 2024-02-14T15:06:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 Processor -- start 2024-02-14T15:06:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 -- start start 2024-02-14T15:06:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 0x7f431c1385c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c098790 0x7f431c138b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c09ac90 0x7f431c1326d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c132e60 con 0x7f431c006ae0 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c132fd0 con 0x7f431c09ac90 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432b0e6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c133190 con 0x7f431c098790 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432a8e5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c09ac90 0x7f431c1326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432a8e5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c09ac90 0x7f431c1326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:35672/0 (socket says 172.21.15.17:35672) 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432a8e5640 1 -- 172.21.15.17:0/1639273487 learned_addr learned my addr 172.21.15.17:0/1639273487 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f432a0e4640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 0x7f431c1385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.975+0000 7f43298e3640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c098790 0x7f431c138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.977+0000 7f43298e3640 1 -- 172.21.15.17:0/1639273487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c09ac90 msgr2=0x7f431c1326d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.977+0000 7f43298e3640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c09ac90 0x7f431c1326d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.977+0000 7f43298e3640 1 -- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 msgr2=0x7f431c1385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.977+0000 7f43298e3640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 0x7f431c1385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.977+0000 7f43298e3640 1 -- 172.21.15.17:0/1639273487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240677f0 con 0x7f431c098790 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.977+0000 7f432a0e4640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 0x7f431c1385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.977+0000 7f43298e3640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c098790 0x7f431c138b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4314009850 tx=0x7f431402ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.978+0000 7f431b7fe640 1 -- 172.21.15.17:0/1639273487 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431402cc70 con 0x7f431c098790 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.978+0000 7f431b7fe640 1 -- 172.21.15.17:0/1639273487 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4314004250 con 0x7f431c098790 2024-02-14T15:06:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.978+0000 7f431b7fe640 1 -- 172.21.15.17:0/1639273487 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140039e0 con 0x7f431c098790 2024-02-14T15:06:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.979+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4314009560 con 0x7f431c098790 2024-02-14T15:06:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.979+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c09b470 con 0x7f431c098790 2024-02-14T15:06:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.983+0000 7f431b7fe640 1 -- 172.21.15.17:0/1639273487 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f43140043c0 con 0x7f431c098790 2024-02-14T15:06:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.983+0000 7f432eff2740 1 Processor -- start 2024-02-14T15:06:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f432eff2740 1 -- start start 2024-02-14T15:06:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f431b7fe640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43100a22f0 0x7f43100a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f432eff2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec73ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f432eff2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec7c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f432eff2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec7f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f432eff2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ebe19e0 con 0x562c5ec74220 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f432eff2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ebdef10 con 0x562c5ec73900 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.984+0000 7f432eff2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ebda6f0 con 0x562c5ec7ce00 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec7c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f432a8e5640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43100a22f0 0x7f43100a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec7c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47766/0 (socket says 172.21.15.17:47766) 2024-02-14T15:06:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 -- 172.21.15.17:0/321308067 learned_addr learned my addr 172.21.15.17:0/321308067 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f432a8e5640 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec7f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f432a0e4640 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec73ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f431b7fe640 1 -- 172.21.15.17:0/1639273487 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f4314101350 con 0x7f431c098790 2024-02-14T15:06:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 -- 172.21.15.17:0/321308067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 msgr2=0x562c5ec7f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec7f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 -- 172.21.15.17:0/321308067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 msgr2=0x562c5ec73ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec73ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.985+0000 7f43298e3640 1 -- 172.21.15.17:0/321308067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c5e8eb920 con 0x562c5ec74220 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.986+0000 7f432a8e5640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43100a22f0 0x7f43100a47b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4324064a50 tx=0x7f432404f8a0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.986+0000 7f43298e3640 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec7c8c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f43140fe850 tx=0x7f4314059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e509f6c8ce2dac87 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.986+0000 7f4318bfe640 1 -- 172.21.15.17:0/321308067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140ac2f0 con 0x562c5ec74220 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.986+0000 7f4318bfe640 1 -- 172.21.15.17:0/321308067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43140cd3f0 con 0x562c5ec74220 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.986+0000 7f4318bfe640 1 -- 172.21.15.17:0/321308067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140ce190 con 0x562c5ec74220 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 -- 172.21.15.17:0/321308067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 msgr2=0x562c5ec7c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec7c8c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f43140fe850 tx=0x7f4314059000 comp rx=0 tx=0).stop 2024-02-14T15:06:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 -- 172.21.15.17:0/321308067 shutdown_connections 2024-02-14T15:06:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec7c8c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec7f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 --2- 172.21.15.17:0/321308067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec73ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 -- 172.21.15.17:0/321308067 >> 172.21.15.17:0/321308067 conn(0x562c5ec6e030 msgr2=0x562c5ec6e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.987+0000 7f432eff2740 1 -- 172.21.15.17:0/321308067 shutdown_connections 2024-02-14T15:06:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.988+0000 7f432eff2740 1 -- 172.21.15.17:0/321308067 wait complete. 2024-02-14T15:06:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.988+0000 7f432eff2740 1 -- start start 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.988+0000 7f432eff2740 1 Processor -- start 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432eff2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec73900 0x562c5ec7c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432eff2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432eff2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec7ce00 0x562c5ec79f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432eff2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5eb76fa0 con 0x562c5ec73900 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432eff2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e8ae250 con 0x562c5ec7ce00 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432eff2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5eb60200 con 0x562c5ec74220 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432eff2740 10 client.0 _ll_register_callbacks cb 0x562c5ebcfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432a0e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec73900 0x562c5ec7c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432a0e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec73900 0x562c5ec7c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47782/0 (socket says 172.21.15.17:47782) 2024-02-14T15:06:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432a0e4640 1 -- 172.21.15.17:0/1479509775 learned_addr learned my addr 172.21.15.17:0/1479509775 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f43298e3640 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.989+0000 7f432a8e5640 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec7ce00 0x562c5ec79f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f43298e3640 1 -- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec7ce00 msgr2=0x562c5ec79f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f43298e3640 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec7ce00 0x562c5ec79f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f43298e3640 1 -- 172.21.15.17:0/1479509775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec73900 msgr2=0x562c5ec7c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f43298e3640 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec73900 0x562c5ec7c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f43298e3640 1 -- 172.21.15.17:0/1479509775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c09b470 con 0x562c5ec74220 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f43298e3640 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec799e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x562c5ec733d0 tx=0x7f43140ac2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeeeaedab47b1b49 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f430dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f430dffb640 1 -- 172.21.15.17:0/1479509775 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140acac0 con 0x562c5ec74220 2024-02-14T15:06:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c5e8da600 con 0x562c5ec74220 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f430dffb640 1 -- 172.21.15.17:0/1479509775 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f4314101120 con 0x562c5ec74220 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f430dffb640 1 -- 172.21.15.17:0/1479509775 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43140ac650 con 0x562c5ec74220 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f432eff2740 10 client.26068 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.990+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562c5ebd8d70 con 0x562c5ec74220 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f430dffb640 1 -- 172.21.15.17:0/1479509775 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43140b2020 con 0x562c5ec74220 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f430dffb640 1 -- 172.21.15.17:0/1479509775 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f43140aac60 con 0x562c5ec74220 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f432eff2740 10 client.26068 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f432eff2740 10 client.26068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f432eff2740 20 client.26068 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f432eff2740 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ec8c120 0x562c5ec8e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f432eff2740 4 client.26068 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:06:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.991+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c5e92b1b0 con 0x562c5ec8c120 2024-02-14T15:06:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.992+0000 7f432a0e4640 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ec8c120 0x562c5ec8e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.993+0000 7f430dffb640 10 client.26068 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.993+0000 7f432a0e4640 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ec8c120 0x562c5ec8e510 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2f4b1fed208ba5 server_cookie=f008c665e01353d6 in_seq=0 out_seq=0 2024-02-14T15:06:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f430dffb640 1 -- 172.21.15.17:0/1479509775 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c5e92b1b0 con 0x562c5ec8c120 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f430dffb640 10 client.26068 handle_command_reply: tid=0 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 client.26068 shutdown 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 msgr2=0x562c5ec799e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec799e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x562c5ec733d0 tx=0x7f43140ac2f0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 shutdown_connections 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ec8c120 0x562c5ec8e510 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec7ce00 0x562c5ec79f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f430dffb640 0 client.26068 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec799e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 --2- 172.21.15.17:0/1479509775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec73900 0x562c5ec7c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.994+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 >> 172.21.15.17:0/1479509775 conn(0x562c5ec6e030 msgr2=0x562c5ec6fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.995+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 shutdown_connections 2024-02-14T15:06:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.995+0000 7f432eff2740 1 -- 172.21.15.17:0/1479509775 wait complete. 2024-02-14T15:06:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:36.995+0000 7f432eff2740 20 client.26068 trim_cache size 0 max 16384 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.020+0000 7f432eff2740 1 Processor -- start 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432eff2740 1 -- start start 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432eff2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec74e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432eff2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec7c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432eff2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec7ce00 0x562c5ec7c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432eff2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ec6f060 con 0x562c5ec7ce00 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432eff2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ec77f30 con 0x562c5ec73900 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432a0e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec74e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432a0e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec74e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:35684/0 (socket says 172.21.15.17:35684) 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.021+0000 7f432a0e4640 1 -- 172.21.15.17:0/4166845984 learned_addr learned my addr 172.21.15.17:0/4166845984 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.022+0000 7f43298e3640 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec7c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.022+0000 7f432eff2740 1 -- 172.21.15.17:0/4166845984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ec70270 con 0x562c5ec74220 2024-02-14T15:06:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.022+0000 7f432a8e5640 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec7ce00 0x562c5ec7c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f432a8e5640 1 -- 172.21.15.17:0/4166845984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 msgr2=0x562c5ec7c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f432a8e5640 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec7c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f432a8e5640 1 -- 172.21.15.17:0/4166845984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 msgr2=0x562c5ec74e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f432a8e5640 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec74e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f432a8e5640 1 -- 172.21.15.17:0/4166845984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c5eb75820 con 0x562c5ec7ce00 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f432a8e5640 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec7ce00 0x562c5ec7c650 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4324050890 tx=0x7f43240672e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14a57534a2fbbaf server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f430ffff640 1 -- 172.21.15.17:0/4166845984 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324073ca0 con 0x562c5ec7ce00 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.023+0000 7f430ffff640 1 -- 172.21.15.17:0/4166845984 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4324062500 con 0x562c5ec7ce00 2024-02-14T15:06:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.024+0000 7f430ffff640 1 -- 172.21.15.17:0/4166845984 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324061a50 con 0x562c5ec7ce00 2024-02-14T15:06:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.024+0000 7f432eff2740 1 -- 172.21.15.17:0/4166845984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec7ce00 msgr2=0x562c5ec7c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.024+0000 7f432eff2740 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec7ce00 0x562c5ec7c650 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4324050890 tx=0x7f43240672e0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 -- 172.21.15.17:0/4166845984 shutdown_connections 2024-02-14T15:06:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec7ce00 0x562c5ec7c650 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec74220 0x562c5ec7c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 --2- 172.21.15.17:0/4166845984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec74e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 -- 172.21.15.17:0/4166845984 >> 172.21.15.17:0/4166845984 conn(0x562c5ec6e030 msgr2=0x562c5ec74740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 -- 172.21.15.17:0/4166845984 shutdown_connections 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 -- 172.21.15.17:0/4166845984 wait complete. 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 -- start start 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.025+0000 7f432eff2740 1 Processor -- start 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432eff2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec82850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432eff2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432eff2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f43298e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f43298e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec82d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47802/0 (socket says 172.21.15.17:47802) 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f43298e3640 1 -- 172.21.15.17:0/1626143898 learned_addr learned my addr 172.21.15.17:0/1626143898 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ece2680 con 0x562c5ec74220 2024-02-14T15:06:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ece27f0 con 0x562c5ec73900 2024-02-14T15:06:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5ece29b0 con 0x562c5ec7ce00 2024-02-14T15:06:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432a8e5640 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.026+0000 7f432eff2740 10 client.0 _ll_register_callbacks cb 0x7f4320009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f43298e3640 1 -- 172.21.15.17:0/1626143898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 msgr2=0x562c5ec832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f43298e3640 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f43298e3640 1 -- 172.21.15.17:0/1626143898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 msgr2=0x562c5ec82850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f43298e3640 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec82850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f43298e3640 1 -- 172.21.15.17:0/1626143898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4324088040 con 0x562c5ec74220 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f43298e3640 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec82d90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f43140cbd40 tx=0x7f431402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9ad4eea9b8eea4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f430dffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f430dffb640 1 -- 172.21.15.17:0/1626143898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140acac0 con 0x562c5ec74220 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f430dffb640 1 -- 172.21.15.17:0/1626143898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f43140b3080 con 0x562c5ec74220 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.027+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43140cd3f0 con 0x562c5ec74220 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f430dffb640 1 -- 172.21.15.17:0/1626143898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43140cd860 con 0x562c5ec74220 2024-02-14T15:06:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 10 client.16548 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562c5eb9b720 con 0x562c5ec74220 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f430dffb640 1 -- 172.21.15.17:0/1626143898 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43140cd3f0 con 0x562c5ec74220 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f430dffb640 1 -- 172.21.15.17:0/1626143898 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4314003b40 con 0x562c5ec74220 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 10 client.16548 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 20 client.16548 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ece3ca0 0x562c5ece60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 4 client.16548 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.028+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c5e917550 con 0x562c5ece3ca0 2024-02-14T15:06:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.029+0000 7f432a0e4640 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ece3ca0 0x562c5ece60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.031+0000 7f432a0e4640 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ece3ca0 0x562c5ece60c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a747550bdb3bfcc server_cookie=f8f7f09857cd6719 in_seq=0 out_seq=0 2024-02-14T15:06:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.031+0000 7f430dffb640 10 client.16548 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.047+0000 7f430dffb640 1 -- 172.21.15.17:0/1626143898 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c5e917550 con 0x562c5ece3ca0 2024-02-14T15:06:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.047+0000 7f430dffb640 10 client.16548 handle_command_reply: tid=0 2024-02-14T15:06:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 client.16548 shutdown 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 msgr2=0x562c5ec82d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec82d90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f43140cbd40 tx=0x7f431402bc30 comp rx=0 tx=0).stop 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 shutdown_connections 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c5ece3ca0 0x562c5ece60c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c5ec74220 0x562c5ec82d90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c5ec7ce00 0x562c5ec832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 --2- 172.21.15.17:0/1626143898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c5ec73900 0x562c5ec82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.048+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 >> 172.21.15.17:0/1626143898 conn(0x562c5ec6e030 msgr2=0x562c5ec6cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.049+0000 7f430dffb640 0 client.16548 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.049+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 shutdown_connections 2024-02-14T15:06:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.049+0000 7f432eff2740 1 -- 172.21.15.17:0/1626143898 wait complete. 2024-02-14T15:06:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.049+0000 7f432eff2740 20 client.16548 trim_cache size 0 max 16384 2024-02-14T15:06:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:06:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:06:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:06:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43100a22f0 msgr2=0x7f43100a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43100a22f0 0x7f43100a47b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4324064a50 tx=0x7f432404f8a0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c098790 msgr2=0x7f431c138b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c098790 0x7f431c138b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4314009850 tx=0x7f431402ca80 comp rx=0 tx=0).stop 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 shutdown_connections 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43100a22f0 0x7f43100a47b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f431c09ac90 0x7f431c1326d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c098790 0x7f431c138b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 --2- 172.21.15.17:0/1639273487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431c006ae0 0x7f431c1385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.050+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 >> 172.21.15.17:0/1639273487 conn(0x7f431c00a7e0 msgr2=0x7f431c092290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.052+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 shutdown_connections 2024-02-14T15:06:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:37.052+0000 7f432b0e6640 1 -- 172.21.15.17:0/1639273487 wait complete. 2024-02-14T15:06:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:36 smithi050 ceph-mon[36512]: pgmap v98: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-02-14T15:06:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:36 smithi076 ceph-mon[37057]: pgmap v98: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-02-14T15:06:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:36 smithi017 ceph-mon[32549]: pgmap v98: 129 pgs: 129 active+clean; 116 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-02-14T15:06:37.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 211681 11.68 MB/sec warmup 99 sec latency 22.495 ms 2024-02-14T15:06:38.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 213575 11.69 MB/sec warmup 100 sec latency 71.961 ms 2024-02-14T15:06:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:38 smithi050 ceph-mon[36512]: pgmap v99: 129 pgs: 129 active+clean; 123 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 333 op/s 2024-02-14T15:06:39.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:38 smithi076 ceph-mon[37057]: pgmap v99: 129 pgs: 129 active+clean; 123 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 333 op/s 2024-02-14T15:06:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:38 smithi017 ceph-mon[32549]: pgmap v99: 129 pgs: 129 active+clean; 123 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 333 op/s 2024-02-14T15:06:39.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 216703 11.75 MB/sec warmup 101 sec latency 22.448 ms 2024-02-14T15:06:40.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 219591 11.77 MB/sec warmup 102 sec latency 17.188 ms 2024-02-14T15:06:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:40 smithi050 ceph-mon[36512]: pgmap v100: 129 pgs: 129 active+clean; 123 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 212 op/s 2024-02-14T15:06:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:40 smithi076 ceph-mon[37057]: pgmap v100: 129 pgs: 129 active+clean; 123 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 212 op/s 2024-02-14T15:06:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:40 smithi017 ceph-mon[32549]: pgmap v100: 129 pgs: 129 active+clean; 123 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 212 op/s 2024-02-14T15:06:41.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 221384 11.76 MB/sec warmup 103 sec latency 55.596 ms 2024-02-14T15:06:42.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 223913 11.79 MB/sec warmup 104 sec latency 31.670 ms 2024-02-14T15:06:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:42 smithi050 ceph-mon[36512]: pgmap v101: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-02-14T15:06:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:42 smithi076 ceph-mon[37057]: pgmap v101: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-02-14T15:06:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:42 smithi017 ceph-mon[32549]: pgmap v101: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-02-14T15:06:43.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 226495 11.79 MB/sec warmup 105 sec latency 65.581 ms 2024-02-14T15:06:44.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 228509 11.79 MB/sec warmup 106 sec latency 111.426 ms 2024-02-14T15:06:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:44 smithi050 ceph-mon[36512]: pgmap v102: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:06:45.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:44 smithi076 ceph-mon[37057]: pgmap v102: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:06:45.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:44 smithi017 ceph-mon[32549]: pgmap v102: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:06:45.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 231072 11.81 MB/sec warmup 107 sec latency 11.838 ms 2024-02-14T15:06:46.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 233772 11.83 MB/sec warmup 108 sec latency 79.123 ms 2024-02-14T15:06:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:46 smithi050 ceph-mon[36512]: pgmap v103: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:06:47.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:46 smithi076 ceph-mon[37057]: pgmap v103: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:06:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:46 smithi017 ceph-mon[32549]: pgmap v103: 129 pgs: 129 active+clean; 129 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:06:47.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 235619 11.82 MB/sec warmup 109 sec latency 18.597 ms 2024-02-14T15:06:48.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 238373 11.84 MB/sec warmup 110 sec latency 15.592 ms 2024-02-14T15:06:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:48 smithi050 ceph-mon[36512]: pgmap v104: 129 pgs: 129 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 340 op/s 2024-02-14T15:06:49.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:48 smithi076 ceph-mon[37057]: pgmap v104: 129 pgs: 129 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 340 op/s 2024-02-14T15:06:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:48 smithi017 ceph-mon[32549]: pgmap v104: 129 pgs: 129 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 340 op/s 2024-02-14T15:06:49.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 241517 11.90 MB/sec warmup 111 sec latency 14.584 ms 2024-02-14T15:06:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 -- 172.21.15.17:0/1500802080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940101450 msgr2=0x7f0940101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 --2- 172.21.15.17:0/1500802080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940101450 0x7f0940101830 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0930009930 tx=0x7f093002bc30 comp rx=0 tx=0).stop 2024-02-14T15:06:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 -- 172.21.15.17:0/1500802080 shutdown_connections 2024-02-14T15:06:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 --2- 172.21.15.17:0/1500802080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0940106520 0x7f094010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 --2- 172.21.15.17:0/1500802080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0940101e00 0x7f0940105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 --2- 172.21.15.17:0/1500802080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940101450 0x7f0940101830 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 -- 172.21.15.17:0/1500802080 >> 172.21.15.17:0/1500802080 conn(0x7f09400fccb0 msgr2=0x7f09400ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 -- 172.21.15.17:0/1500802080 shutdown_connections 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.147+0000 7f0944c10640 1 -- 172.21.15.17:0/1500802080 wait complete. 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 Processor -- start 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 -- start start 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0940101450 0x7f094019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940106520 0x7f094019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09401976e0 0x7f0940197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401981b0 con 0x7f0940106520 2024-02-14T15:06:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0940198320 con 0x7f09401976e0 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0944c10640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401984e0 con 0x7f0940101450 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940106520 0x7f094019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f093ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09401976e0 0x7f0940197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f093e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0940101450 0x7f094019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940106520 0x7f094019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41322/0 (socket says 172.21.15.17:41322) 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.148+0000 7f0935d9b640 1 -- 172.21.15.17:0/1879684287 learned_addr learned my addr 172.21.15.17:0/1879684287 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f093ed9d640 1 -- 172.21.15.17:0/1879684287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0940101450 msgr2=0x7f094019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f093ed9d640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0940101450 0x7f094019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f093ed9d640 1 -- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940106520 msgr2=0x7f094019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f093ed9d640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940106520 0x7f094019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f093ed9d640 1 -- 172.21.15.17:0/1879684287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009560 con 0x7f09401976e0 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f093e59c640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0940101450 0x7f094019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f0935d9b640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940106520 0x7f094019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f093ed9d640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09401976e0 0x7f0940197b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0938009fd0 tx=0x7f093800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f0937fff640 1 -- 172.21.15.17:0/1879684287 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093800ce20 con 0x7f09401976e0 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09401a4480 con 0x7f09401976e0 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f0937fff640 1 -- 172.21.15.17:0/1879684287 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0938005020 con 0x7f09401976e0 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f0937fff640 1 -- 172.21.15.17:0/1879684287 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938010690 con 0x7f09401976e0 2024-02-14T15:06:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.149+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09401a4910 con 0x7f09401976e0 2024-02-14T15:06:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.151+0000 7f0937fff640 1 -- 172.21.15.17:0/1879684287 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0938005500 con 0x7f09401976e0 2024-02-14T15:06:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.151+0000 7f09477d9740 1 Processor -- start 2024-02-14T15:06:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f09477d9740 1 -- start start 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0937fff640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f092c0a2220 0x7f092c0a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f09477d9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cc720 0x56085f5ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f09477d9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cd040 0x56085f5d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f09477d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f09477d9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f53a9e0 con 0x56085f5cd040 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f09477d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f537f10 con 0x56085f5d5c20 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f09477d9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f5336f0 con 0x56085f5cc720 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0935d9b640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f092c0a2220 0x7f092c0a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0937fff640 1 -- 172.21.15.17:0/1879684287 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0938014070 con 0x7f09401976e0 2024-02-14T15:06:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f093e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cc720 0x56085f5ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cd040 0x56085f5d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f093e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cc720 0x56085f5ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:41702/0 (socket says 172.21.15.17:41702) 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f093e59c640 1 -- 172.21.15.17:0/3931923490 learned_addr learned my addr 172.21.15.17:0/3931923490 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f093ed9d640 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0935d9b640 1 -- 172.21.15.17:0/3931923490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cc720 msgr2=0x56085f5ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0935d9b640 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cc720 0x56085f5ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0935d9b640 1 -- 172.21.15.17:0/3931923490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 msgr2=0x56085f5d8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0935d9b640 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.152+0000 7f0935d9b640 1 -- 172.21.15.17:0/3931923490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085f244920 con 0x56085f5cd040 2024-02-14T15:06:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.153+0000 7f0935d9b640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f092c0a2220 0x7f092c0a46e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f092800b6c0 tx=0x7f0928012980 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.153+0000 7f0935d9b640 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cd040 0x56085f5d56e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0928005bb0 tx=0x7f092801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdd9d3bdd88ffef server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.153+0000 7f0934d99640 1 -- 172.21.15.17:0/3931923490 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800dbe0 con 0x56085f5cd040 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.153+0000 7f0934d99640 1 -- 172.21.15.17:0/3931923490 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f092800dd40 con 0x56085f5cd040 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.153+0000 7f0934d99640 1 -- 172.21.15.17:0/3931923490 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092801e650 con 0x56085f5cd040 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.153+0000 7f09477d9740 1 -- 172.21.15.17:0/3931923490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cd040 msgr2=0x56085f5d56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cd040 0x56085f5d56e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0928005bb0 tx=0x7f092801c000 comp rx=0 tx=0).stop 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- 172.21.15.17:0/3931923490 shutdown_connections 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cd040 0x56085f5d56e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 --2- 172.21.15.17:0/3931923490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cc720 0x56085f5ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- 172.21.15.17:0/3931923490 >> 172.21.15.17:0/3931923490 conn(0x56085f5c6e50 msgr2=0x56085f5c7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- 172.21.15.17:0/3931923490 shutdown_connections 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- 172.21.15.17:0/3931923490 wait complete. 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- start start 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 Processor -- start 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f4cffa0 con 0x56085f5cc720 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f207250 con 0x56085f5d5c20 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f4b9200 con 0x56085f5cd040 2024-02-14T15:06:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f09477d9740 10 client.0 _ll_register_callbacks cb 0x56085f528b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f093e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f093e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41346/0 (socket says 172.21.15.17:41346) 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f093e59c640 1 -- 172.21.15.17:0/372780996 learned_addr learned my addr 172.21.15.17:0/372780996 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.154+0000 7f0935d9b640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093ed9d640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093e59c640 1 -- 172.21.15.17:0/372780996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 msgr2=0x56085f5d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093e59c640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093e59c640 1 -- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 msgr2=0x56085f5d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093e59c640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093e59c640 1 -- 172.21.15.17:0/372780996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092801a040 con 0x56085f5cc720 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093ed9d640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f093e59c640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5d55c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0930009230 tx=0x7f0930004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcc7ce1f5057a40 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f0925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f0925ffb640 1 -- 172.21.15.17:0/372780996 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930004480 con 0x56085f5cc720 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f0925ffb640 1 -- 172.21.15.17:0/372780996 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f0930004df0 con 0x56085f5cc720 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56085f233600 con 0x56085f5cc720 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f0925ffb640 1 -- 172.21.15.17:0/372780996 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0930004fb0 con 0x56085f5cc720 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f09477d9740 10 client.16560 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56085f531d70 con 0x56085f5cc720 2024-02-14T15:06:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.155+0000 7f0925ffb640 1 -- 172.21.15.17:0/372780996 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56085f233600 con 0x56085f5cc720 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f0925ffb640 1 -- 172.21.15.17:0/372780996 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f093003c040 con 0x56085f5cc720 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f09477d9740 10 client.16560 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f09477d9740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f09477d9740 20 client.16560 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f09477d9740 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f5e59e0 0x56085f5e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f09477d9740 4 client.16560 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56085f2841b0 con 0x56085f5e59e0 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f0935d9b640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f5e59e0 0x56085f5e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f0935d9b640 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f5e59e0 0x56085f5e7e00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9bd04634a0c687 server_cookie=dce6550c381d6e31 in_seq=0 out_seq=0 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.156+0000 7f0925ffb640 10 client.16560 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f0925ffb640 1 -- 172.21.15.17:0/372780996 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56085f2841b0 con 0x56085f5e59e0 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f0925ffb640 10 client.16560 handle_command_reply: tid=0 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 client.16560 shutdown 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 msgr2=0x56085f5d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5d55c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0930009230 tx=0x7f0930004040 comp rx=0 tx=0).stop 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 shutdown_connections 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f5e59e0 0x56085f5e7e00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 --2- 172.21.15.17:0/372780996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5d55c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 >> 172.21.15.17:0/372780996 conn(0x56085f5c6e50 msgr2=0x56085f5c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f0925ffb640 0 client.16560 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 shutdown_connections 2024-02-14T15:06:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 1 -- 172.21.15.17:0/372780996 wait complete. 2024-02-14T15:06:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.157+0000 7f09477d9740 20 client.16560 trim_cache size 0 max 16384 2024-02-14T15:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 Processor -- start 2024-02-14T15:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 -- start start 2024-02-14T15:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f5cbaf0 con 0x56085f5cc720 2024-02-14T15:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f5c52d0 con 0x56085f5d5c20 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f09477d9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f5c6680 con 0x56085f5cd040 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.170+0000 7f093e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5dcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41360/0 (socket says 172.21.15.17:41360) 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 -- 172.21.15.17:0/1697193902 learned_addr learned my addr 172.21.15.17:0/1697193902 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f0935d9b640 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093ed9d640 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 -- 172.21.15.17:0/1697193902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 msgr2=0x56085f5d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 -- 172.21.15.17:0/1697193902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 msgr2=0x56085f5d5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 -- 172.21.15.17:0/1697193902 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085f4ce820 con 0x56085f5cc720 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f093e59c640 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5dcea0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0930004170 tx=0x7f093002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df1df812d4aa2c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f0927fff640 1 -- 172.21.15.17:0/1697193902 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930003ec0 con 0x56085f5cc720 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f0927fff640 1 -- 172.21.15.17:0/1697193902 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0930034490 con 0x56085f5cc720 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.171+0000 7f0927fff640 1 -- 172.21.15.17:0/1697193902 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930034ae0 con 0x56085f5cc720 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- 172.21.15.17:0/1697193902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 msgr2=0x56085f5dcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5dcea0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0930004170 tx=0x7f093002bc30 comp rx=0 tx=0).stop 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- 172.21.15.17:0/1697193902 shutdown_connections 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f5d5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f5d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 --2- 172.21.15.17:0/1697193902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f5cc720 0x56085f5dcea0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- 172.21.15.17:0/1697193902 >> 172.21.15.17:0/1697193902 conn(0x56085f5c6e50 msgr2=0x56085f5d1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- 172.21.15.17:0/1697193902 shutdown_connections 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- 172.21.15.17:0/1697193902 wait complete. 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- start start 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 Processor -- start 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f63a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f63a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f634520 0x56085f6349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f634ee0 con 0x56085f634520 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f635050 con 0x56085f5d5c20 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085f635210 con 0x56085f5cd040 2024-02-14T15:06:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f09477d9740 10 client.0 _ll_register_callbacks cb 0x7f0928012850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f093ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f634520 0x56085f6349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f093ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f634520 0x56085f6349a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41362/0 (socket says 172.21.15.17:41362) 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f093ed9d640 1 -- 172.21.15.17:0/3419013928 learned_addr learned my addr 172.21.15.17:0/3419013928 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f093e59c640 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f63a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.172+0000 7f0935d9b640 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f63a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f093ed9d640 1 -- 172.21.15.17:0/3419013928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 msgr2=0x56085f63a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f093ed9d640 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f63a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f093ed9d640 1 -- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 msgr2=0x56085f63a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f093ed9d640 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f63a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f093ed9d640 1 -- 172.21.15.17:0/3419013928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56085f531d70 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f093ed9d640 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f634520 0x56085f6349a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f09380cbef0 tx=0x7f093809b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f14937e638e473 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f0925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f0925ffb640 1 -- 172.21.15.17:0/3419013928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380cc320 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f0925ffb640 1 -- 172.21.15.17:0/3419013928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f09380a38d0 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56085f5c77d0 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f0925ffb640 1 -- 172.21.15.17:0/3419013928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09380a3640 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 10 client.16566 fetch_fsmap learned FSMap version 27 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56085f4f4720 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f0925ffb640 1 -- 172.21.15.17:0/3419013928 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56085f5c77d0 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f0925ffb640 1 -- 172.21.15.17:0/3419013928 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f093801f070 con 0x56085f634520 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 10 client.16566 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:06:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:06:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 20 client.16566 populate_metadata read hostname 'smithi017' 2024-02-14T15:06:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f642560 0x56085f644980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:06:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 4 client.16566 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:06:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.173+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f092800dd40 con 0x56085f642560 2024-02-14T15:06:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.174+0000 7f093e59c640 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f642560 0x56085f644980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:06:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.174+0000 7f093e59c640 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f642560 0x56085f644980 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edeaf908835f1c23 server_cookie=37d05f71c313a981 in_seq=0 out_seq=0 2024-02-14T15:06:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.174+0000 7f0925ffb640 10 client.16566 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f0925ffb640 1 -- 172.21.15.17:0/3419013928 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f092800dd40 con 0x56085f642560 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f0925ffb640 10 client.16566 handle_command_reply: tid=0 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 client.16566 shutdown 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f634520 msgr2=0x56085f6349a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f634520 0x56085f6349a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f09380cbef0 tx=0x7f093809b1f0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 shutdown_connections 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56085f642560 0x56085f644980 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56085f634520 0x56085f6349a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56085f5d5c20 0x56085f63a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f0925ffb640 0 client.16566 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 --2- 172.21.15.17:0/3419013928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56085f5cd040 0x56085f63a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 >> 172.21.15.17:0/3419013928 conn(0x56085f5c6e50 msgr2=0x56085f5d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 shutdown_connections 2024-02-14T15:06:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 1 -- 172.21.15.17:0/3419013928 wait complete. 2024-02-14T15:06:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.182+0000 7f09477d9740 20 client.16566 trim_cache size 0 max 16384 2024-02-14T15:06:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:06:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:06:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:06:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:06:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f092c0a2220 msgr2=0x7f092c0a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f092c0a2220 0x7f092c0a46e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f092800b6c0 tx=0x7f0928012980 comp rx=0 tx=0).stop 2024-02-14T15:06:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09401976e0 msgr2=0x7f0940197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:06:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09401976e0 0x7f0940197b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0938009fd0 tx=0x7f093800ef10 comp rx=0 tx=0).stop 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 shutdown_connections 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f092c0a2220 0x7f092c0a46e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0940106520 0x7f094019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09401976e0 0x7f0940197b60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 --2- 172.21.15.17:0/1879684287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0940101450 0x7f094019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 >> 172.21.15.17:0/1879684287 conn(0x7f09400fccb0 msgr2=0x7f09400fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 shutdown_connections 2024-02-14T15:06:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:06:50.183+0000 7f0944c10640 1 -- 172.21.15.17:0/1879684287 wait complete. 2024-02-14T15:06:50.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 243841 11.87 MB/sec warmup 112 sec latency 119.975 ms 2024-02-14T15:06:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:50 smithi050 ceph-mon[36512]: pgmap v105: 129 pgs: 129 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-02-14T15:06:51.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:50 smithi076 ceph-mon[37057]: pgmap v105: 129 pgs: 129 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-02-14T15:06:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:50 smithi017 ceph-mon[32549]: pgmap v105: 129 pgs: 129 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-02-14T15:06:51.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 245699 11.87 MB/sec warmup 113 sec latency 53.288 ms 2024-02-14T15:06:52.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 248593 11.92 MB/sec warmup 114 sec latency 26.453 ms 2024-02-14T15:06:53.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:52 smithi050 ceph-mon[36512]: pgmap v106: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-02-14T15:06:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:52 smithi076 ceph-mon[37057]: pgmap v106: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-02-14T15:06:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:52 smithi017 ceph-mon[32549]: pgmap v106: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-02-14T15:06:53.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 251301 11.93 MB/sec warmup 115 sec latency 11.376 ms 2024-02-14T15:06:54.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 253507 11.93 MB/sec warmup 116 sec latency 16.997 ms 2024-02-14T15:06:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:54 smithi050 ceph-mon[36512]: pgmap v107: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-02-14T15:06:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:54 smithi076 ceph-mon[37057]: pgmap v107: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-02-14T15:06:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:54 smithi017 ceph-mon[32549]: pgmap v107: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-02-14T15:06:55.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 255892 11.95 MB/sec warmup 117 sec latency 78.760 ms 2024-02-14T15:06:56.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 258434 11.95 MB/sec warmup 118 sec latency 42.479 ms 2024-02-14T15:06:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:56 smithi076 ceph-mon[37057]: pgmap v108: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-02-14T15:06:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:56 smithi017 ceph-mon[32549]: pgmap v108: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-02-14T15:06:57.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:56 smithi050 ceph-mon[36512]: pgmap v108: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-02-14T15:06:57.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 260475 11.95 MB/sec warmup 119 sec latency 22.122 ms 2024-02-14T15:06:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:06:58 smithi076 ceph-mon[37057]: pgmap v109: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 341 op/s 2024-02-14T15:06:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:06:58 smithi017 ceph-mon[32549]: pgmap v109: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 341 op/s 2024-02-14T15:06:59.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:06:58 smithi050 ceph-mon[36512]: pgmap v109: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 341 op/s 2024-02-14T15:06:59.864 INFO:tasks.workunit.client.0.smithi017.stdout: 1 266271 18.79 MB/sec execute 1 sec latency 15.374 ms 2024-02-14T15:07:00.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 268324 12.47 MB/sec execute 2 sec latency 88.324 ms 2024-02-14T15:07:01.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:00 smithi076 ceph-mon[37057]: pgmap v110: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-02-14T15:07:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:00 smithi017 ceph-mon[32549]: pgmap v110: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-02-14T15:07:01.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:00 smithi050 ceph-mon[36512]: pgmap v110: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-02-14T15:07:01.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 269033 10.55 MB/sec execute 3 sec latency 80.459 ms 2024-02-14T15:07:02.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 270757 10.07 MB/sec execute 4 sec latency 34.112 ms 2024-02-14T15:07:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:02 smithi076 ceph-mon[37057]: pgmap v111: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 368 op/s 2024-02-14T15:07:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:02 smithi017 ceph-mon[32549]: pgmap v111: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 368 op/s 2024-02-14T15:07:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 -- 172.21.15.17:0/4037442653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0100d30 msgr2=0x7fa2c0104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 --2- 172.21.15.17:0/4037442653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0100d30 0x7fa2c0104d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009900 tx=0x7fa2b002bc30 comp rx=0 tx=0).stop 2024-02-14T15:07:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 -- 172.21.15.17:0/4037442653 shutdown_connections 2024-02-14T15:07:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 --2- 172.21.15.17:0/4037442653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c0105450 0x7fa2c010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 --2- 172.21.15.17:0/4037442653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0100d30 0x7fa2c0104d70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 --2- 172.21.15.17:0/4037442653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0100380 0x7fa2c0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 -- 172.21.15.17:0/4037442653 >> 172.21.15.17:0/4037442653 conn(0x7fa2c0077250 msgr2=0x7fa2c0077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 -- 172.21.15.17:0/4037442653 shutdown_connections 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.272+0000 7fa2c773e640 1 -- 172.21.15.17:0/4037442653 wait complete. 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 Processor -- start 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 -- start start 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c0100380 0x7fa2c019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0105450 0x7fa2c019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0197740 0x7fa2c0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0198210 con 0x7fa2c0197740 2024-02-14T15:07:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0198380 con 0x7fa2c0100380 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c773e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0198540 con 0x7fa2c0105450 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c5cdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0197740 0x7fa2c0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c54da640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c0100380 0x7fa2c019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c5cdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0197740 0x7fa2c0197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35248/0 (socket says 172.21.15.17:35248) 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c5cdb640 1 -- 172.21.15.17:0/3380889996 learned_addr learned my addr 172.21.15.17:0/3380889996 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.273+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0105450 0x7fa2c019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c5cdb640 1 -- 172.21.15.17:0/3380889996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0105450 msgr2=0x7fa2c019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0105450 0x7fa2c019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c5cdb640 1 -- 172.21.15.17:0/3380889996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c0100380 msgr2=0x7fa2c019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c0100380 0x7fa2c019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c5cdb640 1 -- 172.21.15.17:0/3380889996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc009490 con 0x7fa2c0197740 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0105450 0x7fa2c019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c54da640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c0100380 0x7fa2c019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0197740 0x7fa2c0197bc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800eb10 tx=0x7fa2b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2b67fc640 1 -- 172.21.15.17:0/3380889996 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004b20 con 0x7fa2c0197740 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2b67fc640 1 -- 172.21.15.17:0/3380889996 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2b8004c80 con 0x7fa2c0197740 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2b67fc640 1 -- 172.21.15.17:0/3380889996 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8010650 con 0x7fa2c0197740 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b0009560 con 0x7fa2c0197740 2024-02-14T15:07:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.274+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c01a47e0 con 0x7fa2c0197740 2024-02-14T15:07:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2b67fc640 1 -- 172.21.15.17:0/3380889996 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa2b8005020 con 0x7fa2c0197740 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 Processor -- start 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 -- start start 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a30ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2b67fc640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa2ac0a22a0 0x7fa2ac0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a316010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a2789e0 con 0x55868a30b040 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a275f10 con 0x55868a30a720 2024-02-14T15:07:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.277+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a2716f0 con 0x55868a313c20 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c4cd9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a30ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c5cdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a316010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35254/0 (socket says 172.21.15.17:35254) 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 -- 172.21.15.17:0/166711597 learned_addr learned my addr 172.21.15.17:0/166711597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c4cd9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a30ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:43570/0 (socket says 172.21.15.17:43570) 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa2ac0a22a0 0x7fa2ac0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2b67fc640 1 -- 172.21.15.17:0/3380889996 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa2b8014070 con 0x7fa2c0197740 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 -- 172.21.15.17:0/166711597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 msgr2=0x55868a316010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a316010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 -- 172.21.15.17:0/166711597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 msgr2=0x55868a30ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a30ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 -- 172.21.15.17:0/166711597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558689f82920 con 0x55868a30b040 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a316010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a30ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c54da640 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3136e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0117e0 tx=0x7fa2bc011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97fe3bf9086a5ea server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa2ac0a22a0 0x7fa2ac0a4760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009230 tx=0x7fa2b0005fd0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2ab7fe640 1 -- 172.21.15.17:0/166711597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00fc60 con 0x55868a30b040 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2ab7fe640 1 -- 172.21.15.17:0/166711597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2bc00fdc0 con 0x55868a30b040 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.278+0000 7fa2ab7fe640 1 -- 172.21.15.17:0/166711597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00e750 con 0x55868a30b040 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/166711597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 msgr2=0x55868a3136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3136e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0117e0 tx=0x7fa2bc011be0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/166711597 shutdown_connections 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3136e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a316010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/166711597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a30ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/166711597 >> 172.21.15.17:0/166711597 conn(0x55868a304e50 msgr2=0x55868a305230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/166711597 shutdown_connections 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/166711597 wait complete. 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.279+0000 7fa2ca3e6740 1 -- start start 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 1 Processor -- start 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a313510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3108a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a310de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a20dfa0 con 0x55868a30b040 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558689f45250 con 0x55868a30a720 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a1f7200 con 0x55868a313c20 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2ca3e6740 10 client.0 _ll_register_callbacks cb 0x55868a266b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3108a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35264/0 (socket says 172.21.15.17:35264) 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 -- 172.21.15.17:0/3544611836 learned_addr learned my addr 172.21.15.17:0/3544611836 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a310de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c54da640 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a313510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 -- 172.21.15.17:0/3544611836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 msgr2=0x55868a310de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a310de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 -- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 msgr2=0x55868a313510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a313510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.280+0000 7fa2c4cd9640 1 -- 172.21.15.17:0/3544611836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc011440 con 0x55868a30b040 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3108a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005c40 tx=0x7fa2b002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156ff115faba4a00 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/3544611836 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0004a60 con 0x55868a30b040 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/3544611836 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa2b0005550 con 0x55868a30b040 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558689f71600 con 0x55868a30b040 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/3544611836 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b00035f0 con 0x55868a30b040 2024-02-14T15:07:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 10 client.16578 fetch_fsmap learned FSMap version 27 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55868a26fd70 con 0x55868a30b040 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/3544611836 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558689f71600 con 0x55868a30b040 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/3544611836 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa2b004f850 con 0x55868a30b040 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 10 client.16578 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 20 client.16578 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a3239c0 0x55868a325de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 4 client.16578 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558689fc21b0 con 0x55868a3239c0 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.281+0000 7fa2c54da640 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a3239c0 0x55868a325de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.282+0000 7fa2a8ff9640 10 client.16578 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.282+0000 7fa2c54da640 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a3239c0 0x55868a325de0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1f0a2a6d2e67f6 server_cookie=15238b57cc94a6c6 in_seq=0 out_seq=0 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.282+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/3544611836 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558689fc21b0 con 0x55868a3239c0 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.282+0000 7fa2a8ff9640 10 client.16578 handle_command_reply: tid=0 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.282+0000 7fa2ca3e6740 1 client.16578 shutdown 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 msgr2=0x55868a3108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3108a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005c40 tx=0x7fa2b002bc30 comp rx=0 tx=0).stop 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 shutdown_connections 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a3239c0 0x55868a325de0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a3108a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a310de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/3544611836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30a720 0x55868a313510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2a8ff9640 0 client.16578 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 >> 172.21.15.17:0/3544611836 conn(0x55868a304e50 msgr2=0x55868a306ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 shutdown_connections 2024-02-14T15:07:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/3544611836 wait complete. 2024-02-14T15:07:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.283+0000 7fa2ca3e6740 20 client.16578 trim_cache size 0 max 16384 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 Processor -- start 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 -- start start 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a30a720 0x55868a312fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a313510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a313c20 0x55868a30db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a3005a0 con 0x55868a30b040 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a320410 con 0x55868a313c20 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a300090 con 0x55868a30a720 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2c4cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a313510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2c54da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a30a720 0x55868a312fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2c5cdb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a313c20 0x55868a30db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2c54da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a30a720 0x55868a312fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:57668/0 (socket says 172.21.15.17:57668) 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.296+0000 7fa2c54da640 1 -- 172.21.15.17:0/1505882495 learned_addr learned my addr 172.21.15.17:0/1505882495 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2c5cdb640 1 -- 172.21.15.17:0/1505882495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a30a720 msgr2=0x55868a312fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a30a720 0x55868a312fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2c5cdb640 1 -- 172.21.15.17:0/1505882495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 msgr2=0x55868a313510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a313510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2c5cdb640 1 -- 172.21.15.17:0/1505882495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55868a20c820 con 0x55868a313c20 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a313510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a313c20 0x55868a30db40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80cbc60 tx=0x7fa2b8014f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0ba2ab9b9d83b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2aaffd640 1 -- 172.21.15.17:0/1505882495 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b809af30 con 0x55868a313c20 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2aaffd640 1 -- 172.21.15.17:0/1505882495 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2b809b8c0 con 0x55868a313c20 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.297+0000 7fa2aaffd640 1 -- 172.21.15.17:0/1505882495 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8099350 con 0x55868a313c20 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/1505882495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a313c20 msgr2=0x55868a30db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a313c20 0x55868a30db40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80cbc60 tx=0x7fa2b8014f40 comp rx=0 tx=0).stop 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/1505882495 shutdown_connections 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30b040 0x55868a313510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a313c20 0x55868a30db40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/1505882495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a30a720 0x55868a312fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/1505882495 >> 172.21.15.17:0/1505882495 conn(0x55868a304e50 msgr2=0x55868a306b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/1505882495 shutdown_connections 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/1505882495 wait complete. 2024-02-14T15:07:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- start start 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 Processor -- start 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30a720 0x55868a3195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30b040 0x55868a319b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a31a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a379f20 con 0x55868a30a720 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a37a090 con 0x55868a30b040 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.298+0000 7fa2ca3e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868a37a250 con 0x55868a313c20 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2ca3e6740 10 client.0 _ll_register_callbacks cb 0x7fa2bc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30a720 0x55868a3195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30a720 0x55868a3195e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35290/0 (socket says 172.21.15.17:35290) 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c4cd9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30b040 0x55868a319b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 -- 172.21.15.17:0/2194486944 learned_addr learned my addr 172.21.15.17:0/2194486944 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c5cdb640 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a31a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 -- 172.21.15.17:0/2194486944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 msgr2=0x55868a31a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a31a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 -- 172.21.15.17:0/2194486944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30b040 msgr2=0x55868a319b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30b040 0x55868a319b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 -- 172.21.15.17:0/2194486944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55868a26fd70 con 0x55868a30a720 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2c54da640 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30a720 0x55868a3195e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc01c040 tx=0x7fa2bc019e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2fe80487960f47 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/2194486944 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00fbe0 con 0x55868a30a720 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/2194486944 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa2bc0051b0 con 0x55868a30a720 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2bc00fdc0 con 0x55868a30a720 2024-02-14T15:07:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/2194486944 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2bc005390 con 0x55868a30a720 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2ca3e6740 10 client.16584 fetch_fsmap learned FSMap version 27 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.299+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa2b0009560 con 0x55868a30a720 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/2194486944 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2bc00fdc0 con 0x55868a30a720 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/2194486944 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa2bc032070 con 0x55868a30a720 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2ca3e6740 10 client.16584 fetch_fsmap finished waiting for FSMap version 27 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2ca3e6740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2ca3e6740 20 client.16584 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a37ba90 0x55868a37deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2ca3e6740 4 client.16584 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558689fae550 con 0x55868a37ba90 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a37ba90 0x55868a37deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2c4cd9640 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a37ba90 0x55868a37deb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c044246c725920ec server_cookie=dd40428ad6a3d09e in_seq=0 out_seq=0 2024-02-14T15:07:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.300+0000 7fa2a8ff9640 10 client.16584 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2a8ff9640 1 -- 172.21.15.17:0/2194486944 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558689fae550 con 0x55868a37ba90 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2a8ff9640 10 client.16584 handle_command_reply: tid=0 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 client.16584 shutdown 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30a720 msgr2=0x55868a3195e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30a720 0x55868a3195e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc01c040 tx=0x7fa2bc019e80 comp rx=0 tx=0).stop 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 shutdown_connections 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55868a37ba90 0x55868a37deb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55868a313c20 0x55868a31a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2a8ff9640 0 client.16584 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55868a30b040 0x55868a319b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 --2- 172.21.15.17:0/2194486944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55868a30a720 0x55868a3195e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 >> 172.21.15.17:0/2194486944 conn(0x55868a304e50 msgr2=0x55868a310130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 shutdown_connections 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.309+0000 7fa2ca3e6740 1 -- 172.21.15.17:0/2194486944 wait complete. 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2ca3e6740 20 client.16584 trim_cache size 0 max 16384 2024-02-14T15:07:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:07:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:07:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:07:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa2ac0a22a0 msgr2=0x7fa2ac0a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa2ac0a22a0 0x7fa2ac0a4760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009230 tx=0x7fa2b0005fd0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0197740 msgr2=0x7fa2c0197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0197740 0x7fa2c0197bc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800eb10 tx=0x7fa2b800c2d0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 shutdown_connections 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa2ac0a22a0 0x7fa2ac0a4760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2c0197740 0x7fa2c0197bc0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c0105450 0x7fa2c019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 --2- 172.21.15.17:0/3380889996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2c0100380 0x7fa2c019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 >> 172.21.15.17:0/3380889996 conn(0x7fa2c0077250 msgr2=0x7fa2c0069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 shutdown_connections 2024-02-14T15:07:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:03.310+0000 7fa2c773e640 1 -- 172.21.15.17:0/3380889996 wait complete. 2024-02-14T15:07:03.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:02 smithi050 ceph-mon[36512]: pgmap v111: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 368 op/s 2024-02-14T15:07:03.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 273653 11.47 MB/sec execute 5 sec latency 13.811 ms 2024-02-14T15:07:04.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 276757 12.50 MB/sec execute 6 sec latency 10.155 ms 2024-02-14T15:07:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:04 smithi076 ceph-mon[37057]: pgmap v112: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-02-14T15:07:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:04 smithi017 ceph-mon[32549]: pgmap v112: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-02-14T15:07:05.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:04 smithi050 ceph-mon[36512]: pgmap v112: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-02-14T15:07:05.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 279040 11.91 MB/sec execute 7 sec latency 18.322 ms 2024-02-14T15:07:06.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 281002 11.99 MB/sec execute 8 sec latency 22.979 ms 2024-02-14T15:07:06.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:06 smithi076 ceph-mon[37057]: pgmap v113: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-02-14T15:07:07.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:06 smithi017 ceph-mon[32549]: pgmap v113: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-02-14T15:07:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:06 smithi050 ceph-mon[36512]: pgmap v113: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 246 op/s 2024-02-14T15:07:07.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 283778 12.56 MB/sec execute 9 sec latency 88.370 ms 2024-02-14T15:07:08.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 286187 12.17 MB/sec execute 10 sec latency 6.737 ms 2024-02-14T15:07:08.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:08 smithi076 ceph-mon[37057]: pgmap v114: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 372 op/s 2024-02-14T15:07:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:08 smithi017 ceph-mon[32549]: pgmap v114: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 372 op/s 2024-02-14T15:07:09.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:08 smithi050 ceph-mon[36512]: pgmap v114: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 372 op/s 2024-02-14T15:07:09.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 288280 12.24 MB/sec execute 11 sec latency 13.708 ms 2024-02-14T15:07:09.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:09 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:07:10.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:09 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:07:10.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:09 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:07:10.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 291261 12.74 MB/sec execute 12 sec latency 19.405 ms 2024-02-14T15:07:10.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:10 smithi076 ceph-mon[37057]: pgmap v115: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-02-14T15:07:11.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:10 smithi017 ceph-mon[32549]: pgmap v115: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-02-14T15:07:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:10 smithi050 ceph-mon[36512]: pgmap v115: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-02-14T15:07:11.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 294243 12.88 MB/sec execute 13 sec latency 13.387 ms 2024-02-14T15:07:12.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 295912 12.65 MB/sec execute 14 sec latency 99.273 ms 2024-02-14T15:07:12.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:12 smithi076 ceph-mon[37057]: pgmap v116: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 377 op/s 2024-02-14T15:07:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:12 smithi017 ceph-mon[32549]: pgmap v116: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 377 op/s 2024-02-14T15:07:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:12 smithi050 ceph-mon[36512]: pgmap v116: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 377 op/s 2024-02-14T15:07:13.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 298439 12.75 MB/sec execute 15 sec latency 25.693 ms 2024-02-14T15:07:14.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 301513 13.04 MB/sec execute 16 sec latency 19.429 ms 2024-02-14T15:07:14.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:14 smithi076 ceph-mon[37057]: pgmap v117: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-02-14T15:07:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:14 smithi017 ceph-mon[32549]: pgmap v117: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-02-14T15:07:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:14 smithi050 ceph-mon[36512]: pgmap v117: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-02-14T15:07:15.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 303946 12.79 MB/sec execute 17 sec latency 10.955 ms 2024-02-14T15:07:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.400+0000 7f66b1efb640 1 -- 172.21.15.17:0/2342468039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac100cf0 msgr2=0x7f66ac104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.400+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2342468039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac100cf0 0x7f66ac104d30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009930 tx=0x7f66a002bc30 comp rx=0 tx=0).stop 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.400+0000 7f66b1efb640 1 -- 172.21.15.17:0/2342468039 shutdown_connections 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.400+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2342468039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac100cf0 0x7f66ac104d30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.400+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2342468039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66ac105410 0x7f66ac10cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.400+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2342468039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66ac100340 0x7f66ac100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.400+0000 7f66b1efb640 1 -- 172.21.15.17:0/2342468039 >> 172.21.15.17:0/2342468039 conn(0x7f66ac077220 msgr2=0x7f66ac077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.401+0000 7f66b1efb640 1 -- 172.21.15.17:0/2342468039 shutdown_connections 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.401+0000 7f66b1efb640 1 -- 172.21.15.17:0/2342468039 wait complete. 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.401+0000 7f66b1efb640 1 Processor -- start 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.401+0000 7f66b1efb640 1 -- start start 2024-02-14T15:07:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.401+0000 7f66b1efb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66ac100340 0x7f66ac1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.401+0000 7f66b1efb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac105410 0x7f66ac198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66b1efb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66ac1991c0 0x7f66ac1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66b1efb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1a3e10 con 0x7f66ac105410 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66b1efb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1a3f80 con 0x7f66ac100340 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66b1efb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1a4140 con 0x7f66ac1991c0 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac105410 0x7f66ac198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac105410 0x7f66ac198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49306/0 (socket says 172.21.15.17:49306) 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66ac100340 0x7f66ac1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 -- 172.21.15.17:0/2160546428 learned_addr learned my addr 172.21.15.17:0/2160546428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66abfff640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66ac1991c0 0x7f66ac1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 -- 172.21.15.17:0/2160546428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66ac1991c0 msgr2=0x7f66ac1a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66ac1991c0 0x7f66ac1a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 -- 172.21.15.17:0/2160546428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66ac100340 msgr2=0x7f66ac1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66ac100340 0x7f66ac1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 -- 172.21.15.17:0/2160546428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a0009560 con 0x7f66ac105410 2024-02-14T15:07:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66ac100340 0x7f66ac1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66aaffd640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac105410 0x7f66ac198c30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009230 tx=0x7f66a002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66a8ff9640 1 -- 172.21.15.17:0/2160546428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004350 con 0x7f66ac105410 2024-02-14T15:07:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66a8ff9640 1 -- 172.21.15.17:0/2160546428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66a00044b0 con 0x7f66ac105410 2024-02-14T15:07:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac1a43c0 con 0x7f66ac105410 2024-02-14T15:07:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66a8ff9640 1 -- 172.21.15.17:0/2160546428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0003da0 con 0x7f66ac105410 2024-02-14T15:07:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.402+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac1a48d0 con 0x7f66ac105410 2024-02-14T15:07:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66a8ff9640 1 -- 172.21.15.17:0/2160546428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f66a0004df0 con 0x7f66ac105410 2024-02-14T15:07:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66b4ba7740 1 Processor -- start 2024-02-14T15:07:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66a8ff9640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f66940a2220 0x7f66940a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66b4ba7740 1 -- start start 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66a8ff9640 1 -- 172.21.15.17:0/2160546428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f66a00f0330 con 0x7f66ac105410 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66abfff640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f66940a2220 0x7f66940a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901243830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.405+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x562901250af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x562901253420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629011b19e0 con 0x562901243450 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629011aef10 con 0x562901251030 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629011aa6f0 con 0x562901243d70 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901243830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901243830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49308/0 (socket says 172.21.15.17:49308) 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1945314110 learned_addr learned my addr 172.21.15.17:0/1945314110 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66aaffd640 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x562901250af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66abfff640 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x562901253420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66abfff640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f66940a2220 0x7f66940a46e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009ba0 tx=0x7f66a4011440 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1945314110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 msgr2=0x562901250af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x562901250af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1945314110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 msgr2=0x562901253420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x562901253420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1945314110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562900ebb920 con 0x562901243450 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66aaffd640 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x562901250af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901243830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f669c005f40 tx=0x7f669c011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38df2c52a43206a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f6691ffb640 1 -- 172.21.15.17:0/1945314110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c011330 con 0x562901243450 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f6691ffb640 1 -- 172.21.15.17:0/1945314110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f669c00fe80 con 0x562901243450 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.406+0000 7f6691ffb640 1 -- 172.21.15.17:0/1945314110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c0193e0 con 0x562901243450 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1945314110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 msgr2=0x562901243830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901243830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f669c005f40 tx=0x7f669c011d30 comp rx=0 tx=0).stop 2024-02-14T15:07:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1945314110 shutdown_connections 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x562901253420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x562901250af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1945314110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901243830 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1945314110 >> 172.21.15.17:0/1945314110 conn(0x56290123de50 msgr2=0x56290123e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1945314110 shutdown_connections 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1945314110 wait complete. 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.407+0000 7f66b4ba7740 1 -- start start 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 1 Processor -- start 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901250980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x56290124dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x56290124e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562901146fa0 con 0x562901243450 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562900e7e250 con 0x562901251030 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562901130200 con 0x562901243d70 2024-02-14T15:07:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66b4ba7740 10 client.0 _ll_register_callbacks cb 0x56290119fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901250980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901250980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49320/0 (socket says 172.21.15.17:49320) 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 -- 172.21.15.17:0/3318798039 learned_addr learned my addr 172.21.15.17:0/3318798039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66abfff640 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x56290124e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66aaffd640 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x56290124dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 -- 172.21.15.17:0/3318798039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 msgr2=0x56290124dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x56290124dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 -- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 msgr2=0x56290124e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x56290124e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.408+0000 7f66ab7fe640 1 -- 172.21.15.17:0/3318798039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c011500 con 0x562901243450 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901250980 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408606fb3c51be56 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f668affd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f668affd640 1 -- 172.21.15.17:0/3318798039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c010630 con 0x562901243450 2024-02-14T15:07:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f668affd640 1 -- 172.21.15.17:0/3318798039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f669c00f9a0 con 0x562901243450 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562900eaa600 con 0x562901243450 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f668affd640 1 -- 172.21.15.17:0/3318798039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f669c00fb60 con 0x562901243450 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 10 client.16596 fetch_fsmap learned FSMap version 28 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5629011a8d70 con 0x562901243450 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f668affd640 1 -- 172.21.15.17:0/3318798039 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562900eaa600 con 0x562901243450 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f668affd640 1 -- 172.21.15.17:0/3318798039 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f669c004050 con 0x562901243450 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 10 client.16596 fetch_fsmap finished waiting for FSMap version 28 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 20 client.16596 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562901260d50 0x562901263140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 4 client.16596 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.409+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562900efb1b0 con 0x562901260d50 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.410+0000 7f66aaffd640 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562901260d50 0x562901263140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.410+0000 7f66aaffd640 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562901260d50 0x562901263140 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83305b0b2a93444a server_cookie=b44eda9c78ac0ce1 in_seq=0 out_seq=0 2024-02-14T15:07:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.410+0000 7f668affd640 10 client.16596 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.410+0000 7f668affd640 1 -- 172.21.15.17:0/3318798039 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562900efb1b0 con 0x562901260d50 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.410+0000 7f668affd640 10 client.16596 handle_command_reply: tid=0 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 client.16596 shutdown 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 msgr2=0x562901250980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901250980 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c0113e0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 shutdown_connections 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562901260d50 0x562901263140 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901251030 0x56290124e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901243d70 0x56290124dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f668affd640 0 client.16596 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/3318798039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901250980 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 >> 172.21.15.17:0/3318798039 conn(0x56290123de50 msgr2=0x56290123fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 shutdown_connections 2024-02-14T15:07:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 1 -- 172.21.15.17:0/3318798039 wait complete. 2024-02-14T15:07:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.411+0000 7f66b4ba7740 20 client.16596 trim_cache size 0 max 16384 2024-02-14T15:07:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 Processor -- start 2024-02-14T15:07:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 -- start start 2024-02-14T15:07:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243450 0x562901250440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243d70 0x562901250980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x56290124af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562900efb1b0 con 0x562901243d70 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562900eaa600 con 0x562901243450 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562901244270 con 0x562901251030 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243d70 0x562901250980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243d70 0x562901250980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49330/0 (socket says 172.21.15.17:49330) 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 -- 172.21.15.17:0/578966309 learned_addr learned my addr 172.21.15.17:0/578966309 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243450 0x562901250440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66abfff640 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x56290124af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 -- 172.21.15.17:0/578966309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 msgr2=0x56290124af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x56290124af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 -- 172.21.15.17:0/578966309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243450 msgr2=0x562901250440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243450 0x562901250440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66aaffd640 1 -- 172.21.15.17:0/578966309 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562901145820 con 0x562901243d70 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.425+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243450 0x562901250440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66aaffd640 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243d70 0x562901250980 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006010 tx=0x7f66a004a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93596cb95ab1834 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66917fa640 1 -- 172.21.15.17:0/578966309 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00c0c20 con 0x562901243d70 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66917fa640 1 -- 172.21.15.17:0/578966309 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66a00d4070 con 0x562901243d70 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66917fa640 1 -- 172.21.15.17:0/578966309 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00bac00 con 0x562901243d70 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 -- 172.21.15.17:0/578966309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243d70 msgr2=0x562901250980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243d70 0x562901250980 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f66a0006010 tx=0x7f66a004a040 comp rx=0 tx=0).stop 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 -- 172.21.15.17:0/578966309 shutdown_connections 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243d70 0x562901250980 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x56290124af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/578966309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243450 0x562901250440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 -- 172.21.15.17:0/578966309 >> 172.21.15.17:0/578966309 conn(0x56290123de50 msgr2=0x56290124dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 -- 172.21.15.17:0/578966309 shutdown_connections 2024-02-14T15:07:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.426+0000 7f66b4ba7740 1 -- 172.21.15.17:0/578966309 wait complete. 2024-02-14T15:07:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 -- start start 2024-02-14T15:07:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 Processor -- start 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901257fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243d70 0x562901256480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x5629012569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562901256f00 con 0x562901243450 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562901257070 con 0x562901243d70 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562901257230 con 0x562901251030 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66b4ba7740 10 client.0 _ll_register_callbacks cb 0x7f669c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901257fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901257fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49342/0 (socket says 172.21.15.17:49342) 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1447377253 learned_addr learned my addr 172.21.15.17:0/1447377253 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.427+0000 7f66abfff640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x5629012569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66aaffd640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243d70 0x562901256480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1447377253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 msgr2=0x5629012569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x5629012569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243d70 msgr2=0x562901256480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243d70 0x562901256480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66ab7fe640 1 -- 172.21.15.17:0/1447377253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629011a8d70 con 0x562901243450 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66aaffd640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243d70 0x562901256480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66ab7fe640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901257fe0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f669c0092e0 tx=0x7f669c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433710d79b8a7056 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f668affd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f668affd640 1 -- 172.21.15.17:0/1447377253 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c010a40 con 0x562901243450 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f668affd640 1 -- 172.21.15.17:0/1447377253 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f669c004960 con 0x562901243450 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f669c00fe80 con 0x562901243450 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f668affd640 1 -- 172.21.15.17:0/1447377253 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f669c004b20 con 0x562901243450 2024-02-14T15:07:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66b4ba7740 10 client.16602 fetch_fsmap learned FSMap version 28 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f66a00ede70 con 0x562901243450 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f668affd640 1 -- 172.21.15.17:0/1447377253 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f669c00fe80 con 0x562901243450 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f668affd640 1 -- 172.21.15.17:0/1447377253 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f669c003b00 con 0x562901243450 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.428+0000 7f66b4ba7740 10 client.16602 fetch_fsmap finished waiting for FSMap version 28 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f66b4ba7740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f66b4ba7740 20 client.16602 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5629012b84c0 0x5629012ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f66b4ba7740 4 client.16602 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562900ee7550 con 0x5629012b84c0 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f66aaffd640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5629012b84c0 0x5629012ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f66aaffd640 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5629012b84c0 0x5629012ba8e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d5582b65ca6a35 server_cookie=91c2dd60d9a0cdb in_seq=0 out_seq=0 2024-02-14T15:07:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.429+0000 7f668affd640 10 client.16602 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f668affd640 1 -- 172.21.15.17:0/1447377253 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562900ee7550 con 0x5629012b84c0 2024-02-14T15:07:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f668affd640 10 client.16602 handle_command_reply: tid=0 2024-02-14T15:07:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 client.16602 shutdown 2024-02-14T15:07:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 msgr2=0x562901257fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901257fe0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f669c0092e0 tx=0x7f669c0096c0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 shutdown_connections 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5629012b84c0 0x5629012ba8e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562901251030 0x5629012569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562901243d70 0x562901256480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f668affd640 0 client.16602 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 --2- 172.21.15.17:0/1447377253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562901243450 0x562901257fe0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 >> 172.21.15.17:0/1447377253 conn(0x56290123de50 msgr2=0x56290123c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 shutdown_connections 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.437+0000 7f66b4ba7740 1 -- 172.21.15.17:0/1447377253 wait complete. 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b4ba7740 20 client.16602 trim_cache size 0 max 16384 2024-02-14T15:07:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:07:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:07:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:07:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f66940a2220 msgr2=0x7f66940a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f66940a2220 0x7f66940a46e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009ba0 tx=0x7f66a4011440 comp rx=0 tx=0).stop 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac105410 msgr2=0x7f66ac198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac105410 0x7f66ac198c30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009230 tx=0x7f66a002c680 comp rx=0 tx=0).stop 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 shutdown_connections 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f66940a2220 0x7f66940a46e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66ac105410 0x7f66ac198c30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66ac1991c0 0x7f66ac1a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 --2- 172.21.15.17:0/2160546428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66ac100340 0x7f66ac1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 >> 172.21.15.17:0/2160546428 conn(0x7f66ac077220 msgr2=0x7f66ac0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 shutdown_connections 2024-02-14T15:07:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:16.438+0000 7f66b1efb640 1 -- 172.21.15.17:0/2160546428 wait complete. 2024-02-14T15:07:16.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 305989 12.79 MB/sec execute 18 sec latency 11.159 ms 2024-02-14T15:07:16.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:16 smithi076 ceph-mon[37057]: pgmap v118: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-02-14T15:07:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:16 smithi017 ceph-mon[32549]: pgmap v118: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-02-14T15:07:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:16 smithi050 ceph-mon[36512]: pgmap v118: 129 pgs: 129 active+clean; 167 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-02-14T15:07:17.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 308824 13.07 MB/sec execute 19 sec latency 110.415 ms 2024-02-14T15:07:18.865 INFO:tasks.workunit.client.0.smithi017.stdout: 1 311051 12.78 MB/sec execute 20 sec latency 21.734 ms 2024-02-14T15:07:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:18 smithi076 ceph-mon[37057]: pgmap v119: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 355 op/s 2024-02-14T15:07:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:18 smithi017 ceph-mon[32549]: pgmap v119: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 355 op/s 2024-02-14T15:07:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:18 smithi050 ceph-mon[36512]: pgmap v119: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 355 op/s 2024-02-14T15:07:19.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 313099 12.78 MB/sec execute 21 sec latency 16.341 ms 2024-02-14T15:07:20.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 316091 13.04 MB/sec execute 22 sec latency 13.487 ms 2024-02-14T15:07:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:20 smithi076 ceph-mon[37057]: pgmap v120: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 230 op/s 2024-02-14T15:07:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:20 smithi017 ceph-mon[32549]: pgmap v120: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 230 op/s 2024-02-14T15:07:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:20 smithi050 ceph-mon[36512]: pgmap v120: 129 pgs: 129 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 230 op/s 2024-02-14T15:07:21.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 318644 13.01 MB/sec execute 23 sec latency 33.896 ms 2024-02-14T15:07:22.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 319944 12.76 MB/sec execute 24 sec latency 152.168 ms 2024-02-14T15:07:22.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:22 smithi076 ceph-mon[37057]: pgmap v121: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 345 op/s 2024-02-14T15:07:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:22 smithi017 ceph-mon[32549]: pgmap v121: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 345 op/s 2024-02-14T15:07:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:22 smithi050 ceph-mon[36512]: pgmap v121: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 345 op/s 2024-02-14T15:07:23.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 322024 12.68 MB/sec execute 25 sec latency 313.384 ms 2024-02-14T15:07:24.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 324198 12.70 MB/sec execute 26 sec latency 11.314 ms 2024-02-14T15:07:24.943 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:24 smithi050 ceph-mon[36512]: pgmap v122: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-02-14T15:07:24.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:24 smithi076 ceph-mon[37057]: pgmap v122: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-02-14T15:07:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:24 smithi017 ceph-mon[32549]: pgmap v122: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-02-14T15:07:25.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 326842 12.75 MB/sec execute 27 sec latency 12.279 ms 2024-02-14T15:07:26.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 329890 12.91 MB/sec execute 28 sec latency 11.179 ms 2024-02-14T15:07:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:26 smithi076 ceph-mon[37057]: pgmap v123: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-02-14T15:07:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:26 smithi017 ceph-mon[32549]: pgmap v123: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-02-14T15:07:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:26 smithi050 ceph-mon[36512]: pgmap v123: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-02-14T15:07:27.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 331404 12.70 MB/sec execute 29 sec latency 56.621 ms 2024-02-14T15:07:28.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 333966 12.75 MB/sec execute 30 sec latency 24.642 ms 2024-02-14T15:07:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:28 smithi076 ceph-mon[37057]: pgmap v124: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-02-14T15:07:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:28 smithi017 ceph-mon[32549]: pgmap v124: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-02-14T15:07:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:28 smithi050 ceph-mon[36512]: pgmap v124: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-02-14T15:07:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 -- 172.21.15.17:0/2805063778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56541023b0 msgr2=0x7f5654102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 --2- 172.21.15.17:0/2805063778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56541023b0 0x7f5654102790 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5644009900 tx=0x7f564402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 -- 172.21.15.17:0/2805063778 shutdown_connections 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 --2- 172.21.15.17:0/2805063778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565410b7d0 0x7f565410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 --2- 172.21.15.17:0/2805063778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5654102cd0 0x7f565410b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 --2- 172.21.15.17:0/2805063778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56541023b0 0x7f5654102790 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 -- 172.21.15.17:0/2805063778 >> 172.21.15.17:0/2805063778 conn(0x7f56540fbbd0 msgr2=0x7f56540fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 -- 172.21.15.17:0/2805063778 shutdown_connections 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.525+0000 7f5659132640 1 -- 172.21.15.17:0/2805063778 wait complete. 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.526+0000 7f5659132640 1 Processor -- start 2024-02-14T15:07:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.526+0000 7f5659132640 1 -- start start 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56541023b0 0x7f565419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5654102cd0 0x7f565419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565410b7d0 0x7f56541976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5654197e80 con 0x7f565410b7d0 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5654197ff0 con 0x7f5654102cd0 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541981b0 con 0x7f56541023b0 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565410b7d0 0x7f56541976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5652d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56541023b0 0x7f565419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565410b7d0 0x7f56541976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59056/0 (socket says 172.21.15.17:59056) 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 -- 172.21.15.17:0/948433189 learned_addr learned my addr 172.21.15.17:0/948433189 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5652d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56541023b0 0x7f565419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38000/0 (socket says 172.21.15.17:38000) 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565259c640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5654102cd0 0x7f565419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 -- 172.21.15.17:0/948433189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56541023b0 msgr2=0x7f565419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56541023b0 0x7f565419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 -- 172.21.15.17:0/948433189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5654102cd0 msgr2=0x7f565419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5654102cd0 0x7f565419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 -- 172.21.15.17:0/948433189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644009560 con 0x7f565410b7d0 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565259c640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5654102cd0 0x7f565419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f565359e640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565410b7d0 0x7f56541976f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f564c00e970 tx=0x7f564c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f564bfff640 1 -- 172.21.15.17:0/948433189 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c009720 con 0x7f565410b7d0 2024-02-14T15:07:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5654198490 con 0x7f565410b7d0 2024-02-14T15:07:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f564bfff640 1 -- 172.21.15.17:0/948433189 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f564c005500 con 0x7f565410b7d0 2024-02-14T15:07:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f564bfff640 1 -- 172.21.15.17:0/948433189 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c010430 con 0x7f565410b7d0 2024-02-14T15:07:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.527+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56541a45b0 con 0x7f565410b7d0 2024-02-14T15:07:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f564bfff640 1 -- 172.21.15.17:0/948433189 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f564c005020 con 0x7f565410b7d0 2024-02-14T15:07:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 Processor -- start 2024-02-14T15:07:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 -- start start 2024-02-14T15:07:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f564bfff640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f56400a2220 0x7f56400a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1b1030 0x5557dd1b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd1119e0 con 0x5557dd1a3cf0 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd10ef10 con 0x5557dd1a33d0 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.530+0000 7f565bdd6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd10a6f0 con 0x5557dd1b1030 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f5652d9d640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f56400a2220 0x7f56400a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1b1030 0x5557dd1b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:56776/0 (socket says 172.21.15.17:56776) 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565259c640 1 -- 172.21.15.17:0/984797654 learned_addr learned my addr 172.21.15.17:0/984797654 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565259c640 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f564bfff640 1 -- 172.21.15.17:0/948433189 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f564c014070 con 0x7f565410b7d0 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565359e640 1 -- 172.21.15.17:0/984797654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 msgr2=0x5557dd1a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565359e640 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565359e640 1 -- 172.21.15.17:0/984797654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 msgr2=0x5557dd1b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565359e640 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565359e640 1 -- 172.21.15.17:0/984797654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557dce1b920 con 0x5557dd1b1030 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565259c640 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f5652d9d640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f56400a2220 0x7f56400a46e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f5644005e40 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f565359e640 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1b1030 0x5557dd1b3420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f564c0cbaf0 tx=0x7f564c004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e167c91443e91520 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f56497fa640 1 -- 172.21.15.17:0/984797654 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c09a6f0 con 0x5557dd1b1030 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f56497fa640 1 -- 172.21.15.17:0/984797654 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f564c09b490 con 0x5557dd1b1030 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.531+0000 7f56497fa640 1 -- 172.21.15.17:0/984797654 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c09eee0 con 0x5557dd1b1030 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 -- 172.21.15.17:0/984797654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1b1030 msgr2=0x5557dd1b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1b1030 0x5557dd1b3420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f564c0cbaf0 tx=0x7f564c004f90 comp rx=0 tx=0).stop 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 -- 172.21.15.17:0/984797654 shutdown_connections 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1b1030 0x5557dd1b3420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 --2- 172.21.15.17:0/984797654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 -- 172.21.15.17:0/984797654 >> 172.21.15.17:0/984797654 conn(0x5557dd19de50 msgr2=0x5557dd19e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 -- 172.21.15.17:0/984797654 shutdown_connections 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.532+0000 7f565bdd6740 1 -- 172.21.15.17:0/984797654 wait complete. 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 -- start start 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 Processor -- start 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1b1030 0x5557dd1ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd0a6fa0 con 0x5557dd1b1030 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dcdde250 con 0x5557dd1a33d0 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd090200 con 0x5557dd1a3cf0 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565bdd6740 10 client.0 _ll_register_callbacks cb 0x5557dd0ffb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1b1030 0x5557dd1ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1b1030 0x5557dd1ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59084/0 (socket says 172.21.15.17:59084) 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 -- 172.21.15.17:0/1867357812 learned_addr learned my addr 172.21.15.17:0/1867357812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565259c640 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f5652d9d640 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 -- 172.21.15.17:0/1867357812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 msgr2=0x5557dd1add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 -- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 msgr2=0x5557dd1b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 -- 172.21.15.17:0/1867357812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56541a45b0 con 0x5557dd1b1030 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f565359e640 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1b1030 0x5557dd1ae300 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f564c0cbba0 tx=0x7f564c09a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac489906f34d8eef server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.533+0000 7f563a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f563a7fc640 1 -- 172.21.15.17:0/1867357812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c0046c0 con 0x5557dd1b1030 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f563a7fc640 1 -- 172.21.15.17:0/1867357812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f564c005670 con 0x5557dd1b1030 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557dce0a600 con 0x5557dd1b1030 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f563a7fc640 1 -- 172.21.15.17:0/1867357812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f564c0a3530 con 0x5557dd1b1030 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 10 client.16614 fetch_fsmap learned FSMap version 28 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5557dd108d70 con 0x5557dd1b1030 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f563a7fc640 1 -- 172.21.15.17:0/1867357812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f564c0ab080 con 0x5557dd1b1030 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f563a7fc640 1 -- 172.21.15.17:0/1867357812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f564c01f060 con 0x5557dd1b1030 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 10 client.16614 fetch_fsmap finished waiting for FSMap version 28 2024-02-14T15:07:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 20 client.16614 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd1c0be0 0x5557dd1c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 4 client.16614 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557dce5b1b0 con 0x5557dd1c0be0 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.534+0000 7f5652d9d640 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd1c0be0 0x5557dd1c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.535+0000 7f563a7fc640 10 client.16614 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.535+0000 7f5652d9d640 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd1c0be0 0x5557dd1c3000 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dac86eafd23a60 server_cookie=e2e6adc3219cbe3f in_seq=0 out_seq=0 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.535+0000 7f563a7fc640 1 -- 172.21.15.17:0/1867357812 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557dce5b1b0 con 0x5557dd1c0be0 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.535+0000 7f563a7fc640 10 client.16614 handle_command_reply: tid=0 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.535+0000 7f565bdd6740 1 client.16614 shutdown 2024-02-14T15:07:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1b1030 msgr2=0x5557dd1ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1b1030 0x5557dd1ae300 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f564c0cbba0 tx=0x7f564c09a7a0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 shutdown_connections 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd1c0be0 0x5557dd1c3000 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1b1030 0x5557dd1ae300 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f563a7fc640 0 client.16614 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1867357812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1a33d0 0x5557dd1b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 >> 172.21.15.17:0/1867357812 conn(0x5557dd19de50 msgr2=0x5557dd19fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 shutdown_connections 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 1 -- 172.21.15.17:0/1867357812 wait complete. 2024-02-14T15:07:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.536+0000 7f565bdd6740 20 client.16614 trim_cache size 0 max 16384 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 Processor -- start 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 -- start start 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a33d0 0x5557dd1acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd1b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd1ad480 con 0x5557dd1a3cf0 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd19c300 con 0x5557dd1b1030 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565bdd6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd1ae630 con 0x5557dd1a33d0 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd19f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59092/0 (socket says 172.21.15.17:59092) 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565259c640 1 -- 172.21.15.17:0/1981818858 learned_addr learned my addr 172.21.15.17:0/1981818858 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f565359e640 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd1b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.549+0000 7f5652d9d640 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a33d0 0x5557dd1acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f565259c640 1 -- 172.21.15.17:0/1981818858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a33d0 msgr2=0x5557dd1acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f565259c640 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a33d0 0x5557dd1acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f565259c640 1 -- 172.21.15.17:0/1981818858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 msgr2=0x5557dd1b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f565259c640 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd1b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f565259c640 1 -- 172.21.15.17:0/1981818858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557dd0a5820 con 0x5557dd1a3cf0 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f565359e640 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd1b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f565259c640 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd19f9b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f563c003bb0 tx=0x7f563c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c177656e056df2bc server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f5648ff9640 1 -- 172.21.15.17:0/1981818858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c004b20 con 0x5557dd1a3cf0 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f5648ff9640 1 -- 172.21.15.17:0/1981818858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f563c004c80 con 0x5557dd1a3cf0 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.550+0000 7f5648ff9640 1 -- 172.21.15.17:0/1981818858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c01d660 con 0x5557dd1a3cf0 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- 172.21.15.17:0/1981818858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 msgr2=0x5557dd19f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd19f9b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f563c003bb0 tx=0x7f563c004ab0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- 172.21.15.17:0/1981818858 shutdown_connections 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a3cf0 0x5557dd19f9b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd1b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 --2- 172.21.15.17:0/1981818858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a33d0 0x5557dd1acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- 172.21.15.17:0/1981818858 >> 172.21.15.17:0/1981818858 conn(0x5557dd19de50 msgr2=0x5557dd19f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- 172.21.15.17:0/1981818858 shutdown_connections 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- 172.21.15.17:0/1981818858 wait complete. 2024-02-14T15:07:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- start start 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 Processor -- start 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a33d0 0x5557dd1b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd2170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd1b7ec0 con 0x5557dd1a33d0 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd217620 con 0x5557dd1b1030 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557dd2177e0 con 0x5557dd1a3cf0 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f565bdd6740 10 client.0 _ll_register_callbacks cb 0x7f564c0969f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f5652d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a33d0 0x5557dd1b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f5652d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a33d0 0x5557dd1b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59102/0 (socket says 172.21.15.17:59102) 2024-02-14T15:07:29.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.551+0000 7f5652d9d640 1 -- 172.21.15.17:0/949285596 learned_addr learned my addr 172.21.15.17:0/949285596 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f565259c640 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f565359e640 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd2170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f5652d9d640 1 -- 172.21.15.17:0/949285596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 msgr2=0x5557dd1b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f5652d9d640 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f5652d9d640 1 -- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 msgr2=0x5557dd2170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f5652d9d640 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd2170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f5652d9d640 1 -- 172.21.15.17:0/949285596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557dd108d70 con 0x5557dd1a33d0 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f5652d9d640 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a33d0 0x5557dd1b7200 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5644004c50 tx=0x7f564402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fd599b4e193c30 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f563a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f563a7fc640 1 -- 172.21.15.17:0/949285596 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644054070 con 0x5557dd1a33d0 2024-02-14T15:07:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f563a7fc640 1 -- 172.21.15.17:0/949285596 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f5644047040 con 0x5557dd1a33d0 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f564c09b490 con 0x5557dd1a33d0 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f563a7fc640 1 -- 172.21.15.17:0/949285596 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5644004250 con 0x5557dd1a33d0 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f565bdd6740 10 client.16620 fetch_fsmap learned FSMap version 28 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f563c003970 con 0x5557dd1a33d0 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.552+0000 7f563a7fc640 1 -- 172.21.15.17:0/949285596 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f564c09b490 con 0x5557dd1a33d0 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f563a7fc640 1 -- 172.21.15.17:0/949285596 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5644004fb0 con 0x5557dd1a33d0 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565bdd6740 10 client.16620 fetch_fsmap finished waiting for FSMap version 28 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565bdd6740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565bdd6740 20 client.16620 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565bdd6740 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd218ee0 0x5557dd21b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565bdd6740 4 client.16620 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:07:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557dce47550 con 0x5557dd218ee0 2024-02-14T15:07:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565259c640 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd218ee0 0x5557dd21b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f565259c640 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd218ee0 0x5557dd21b300 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3d19f3cdf47cdf server_cookie=28121f7e5b5b40d2 in_seq=0 out_seq=0 2024-02-14T15:07:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.553+0000 7f563a7fc640 10 client.16620 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:29.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f563a7fc640 1 -- 172.21.15.17:0/949285596 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557dce47550 con 0x5557dd218ee0 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f563a7fc640 10 client.16620 handle_command_reply: tid=0 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 client.16620 shutdown 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a33d0 msgr2=0x5557dd1b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a33d0 0x5557dd1b7200 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5644004c50 tx=0x7f564402bcd0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 shutdown_connections 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5557dd218ee0 0x5557dd21b300 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557dd1b1030 0x5557dd2170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f563a7fc640 0 client.16620 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5557dd1a3cf0 0x5557dd1b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 --2- 172.21.15.17:0/949285596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5557dd1a33d0 0x5557dd1b7200 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 >> 172.21.15.17:0/949285596 conn(0x5557dd19de50 msgr2=0x5557dd1acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 shutdown_connections 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 1 -- 172.21.15.17:0/949285596 wait complete. 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.564+0000 7f565bdd6740 20 client.16620 trim_cache size 0 max 16384 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:07:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:07:29.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f56400a2220 msgr2=0x7f56400a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f56400a2220 0x7f56400a46e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f5644005e40 comp rx=0 tx=0).stop 2024-02-14T15:07:29.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565410b7d0 msgr2=0x7f56541976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565410b7d0 0x7f56541976f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f564c00e970 tx=0x7f564c00ed70 comp rx=0 tx=0).stop 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 shutdown_connections 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f56400a2220 0x7f56400a46e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565410b7d0 0x7f56541976f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5654102cd0 0x7f565419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 --2- 172.21.15.17:0/948433189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56541023b0 0x7f565419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 >> 172.21.15.17:0/948433189 conn(0x7f56540fbbd0 msgr2=0x7f56540fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 shutdown_connections 2024-02-14T15:07:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:29.565+0000 7f5659132640 1 -- 172.21.15.17:0/948433189 wait complete. 2024-02-14T15:07:29.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 337081 12.91 MB/sec execute 31 sec latency 6.768 ms 2024-02-14T15:07:30.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 339431 12.77 MB/sec execute 32 sec latency 13.748 ms 2024-02-14T15:07:30.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:30 smithi076 ceph-mon[37057]: pgmap v125: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 223 op/s 2024-02-14T15:07:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:30 smithi017 ceph-mon[32549]: pgmap v125: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 223 op/s 2024-02-14T15:07:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:30 smithi050 ceph-mon[36512]: pgmap v125: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 223 op/s 2024-02-14T15:07:31.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 341682 12.81 MB/sec execute 33 sec latency 10.513 ms 2024-02-14T15:07:32.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 344367 12.92 MB/sec execute 34 sec latency 80.572 ms 2024-02-14T15:07:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:32 smithi076 ceph-mon[37057]: pgmap v126: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-02-14T15:07:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:32 smithi017 ceph-mon[32549]: pgmap v126: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-02-14T15:07:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:32 smithi050 ceph-mon[36512]: pgmap v126: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-02-14T15:07:33.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 346637 12.79 MB/sec execute 35 sec latency 46.771 ms 2024-02-14T15:07:33.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:33 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:07:34.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:33 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:07:34.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:33 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:07:34.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 348774 12.80 MB/sec execute 36 sec latency 9.979 ms 2024-02-14T15:07:34.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:34 smithi076 ceph-mon[37057]: pgmap v127: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-02-14T15:07:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:34 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:07:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:34 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:07:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:34 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:07:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:34 smithi017 ceph-mon[32549]: pgmap v127: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-02-14T15:07:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:34 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:07:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:34 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:07:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:34 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:07:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:34 smithi050 ceph-mon[36512]: pgmap v127: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-02-14T15:07:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:34 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:07:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:34 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:07:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:34 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:07:35.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 351582 12.92 MB/sec execute 37 sec latency 13.394 ms 2024-02-14T15:07:36.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 354520 12.96 MB/sec execute 38 sec latency 13.221 ms 2024-02-14T15:07:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:36 smithi076 ceph-mon[37057]: pgmap v128: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-02-14T15:07:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:36 smithi017 ceph-mon[32549]: pgmap v128: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-02-14T15:07:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:36 smithi050 ceph-mon[36512]: pgmap v128: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-02-14T15:07:37.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 356188 12.88 MB/sec execute 39 sec latency 68.544 ms 2024-02-14T15:07:38.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 358679 12.91 MB/sec execute 40 sec latency 39.288 ms 2024-02-14T15:07:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:38 smithi076 ceph-mon[37057]: pgmap v129: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-02-14T15:07:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:38 smithi017 ceph-mon[32549]: pgmap v129: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-02-14T15:07:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:38 smithi050 ceph-mon[36512]: pgmap v129: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-02-14T15:07:39.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 360728 12.78 MB/sec execute 41 sec latency 262.715 ms 2024-02-14T15:07:40.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 362592 12.76 MB/sec execute 42 sec latency 265.561 ms 2024-02-14T15:07:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:40 smithi076 ceph-mon[37057]: pgmap v130: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-02-14T15:07:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:40 smithi017 ceph-mon[32549]: pgmap v130: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-02-14T15:07:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:40 smithi050 ceph-mon[36512]: pgmap v130: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-02-14T15:07:41.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 365640 12.89 MB/sec execute 43 sec latency 10.870 ms 2024-02-14T15:07:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 -- 172.21.15.17:0/4062416448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0105410 msgr2=0x7fa0e010cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 --2- 172.21.15.17:0/4062416448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0105410 0x7fa0e010cfe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009cd0 tx=0x7fa0d802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 -- 172.21.15.17:0/4062416448 shutdown_connections 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 --2- 172.21.15.17:0/4062416448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0105410 0x7fa0e010cfe0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 --2- 172.21.15.17:0/4062416448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100cf0 0x7fa0e0104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 --2- 172.21.15.17:0/4062416448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0e0100340 0x7fa0e0100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 -- 172.21.15.17:0/4062416448 >> 172.21.15.17:0/4062416448 conn(0x7fa0e0077220 msgr2=0x7fa0e0077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 -- 172.21.15.17:0/4062416448 shutdown_connections 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.659+0000 7fa0e708f640 1 -- 172.21.15.17:0/4062416448 wait complete. 2024-02-14T15:07:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 Processor -- start 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 -- start start 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100340 0x7fa0e019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0e0105410 0x7fa0e019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0197670 0x7fa0e0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0198140 con 0x7fa0e0197670 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01982b0 con 0x7fa0e0105410 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e708f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0198470 con 0x7fa0e0100340 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100340 0x7fa0e019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100340 0x7fa0e019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:52386/0 (socket says 172.21.15.17:52386) 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/3752432640 learned_addr learned my addr 172.21.15.17:0/3752432640 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0e562c640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0197670 0x7fa0e0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.660+0000 7fa0dffff640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0e0105410 0x7fa0e019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/3752432640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0e0105410 msgr2=0x7fa0e019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0e0105410 0x7fa0e019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0197670 msgr2=0x7fa0e0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0197670 0x7fa0e0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/3752432640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d4009490 con 0x7fa0e0100340 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100340 0x7fa0e019d5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400ea60 tx=0x7fa0d400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0ddffb640 1 -- 172.21.15.17:0/3752432640 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400cde0 con 0x7fa0e0100340 2024-02-14T15:07:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0ddffb640 1 -- 172.21.15.17:0/3752432640 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40035f0 con 0x7fa0e0100340 2024-02-14T15:07:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0ddffb640 1 -- 172.21.15.17:0/3752432640 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4010690 con 0x7fa0e0100340 2024-02-14T15:07:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d8009900 con 0x7fa0e0100340 2024-02-14T15:07:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.661+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e01a4710 con 0x7fa0e0100340 2024-02-14T15:07:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.663+0000 7fa0ddffb640 1 -- 172.21.15.17:0/3752432640 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa0d4003af0 con 0x7fa0e0100340 2024-02-14T15:07:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0ddffb640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa0d00a22a0 0x7fa0d00a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 Processor -- start 2024-02-14T15:07:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0dffff640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa0d00a22a0 0x7fa0d00a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 -- start start 2024-02-14T15:07:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0ddffb640 1 -- 172.21.15.17:0/3752432640 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa0d4014070 con 0x7fa0e0100340 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1abee0 0x56122e1b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1b9030 0x56122e1bb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1199e0 con 0x56122e1ab530 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e116f10 con 0x56122e1b9030 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1126f0 con 0x56122e1abee0 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0dffff640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa0d00a22a0 0x7fa0d00a4760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0198ec0 tx=0x7fa0cc005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1abee0 0x56122e1b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1abee0 0x56122e1b8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:52396/0 (socket says 172.21.15.17:52396) 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59384/0 (socket says 172.21.15.17:59384) 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.664+0000 7fa0dffff640 1 -- 172.21.15.17:0/2487855196 learned_addr learned my addr 172.21.15.17:0/2487855196 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e562c640 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1b9030 0x56122e1bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/2487855196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1abee0 msgr2=0x56122e1b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1abee0 0x56122e1b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/2487855196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1b9030 msgr2=0x56122e1bb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1b9030 0x56122e1bb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/2487855196 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56122de23920 con 0x56122e1ab530 2024-02-14T15:07:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0dffff640 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1abee0 0x56122e1b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e562c640 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1b9030 0x56122e1bb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1ab910 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40cbbe0 tx=0x7fa0d409aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a49e583ea0dd30 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0caffd640 1 -- 172.21.15.17:0/2487855196 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d409efd0 con 0x56122e1ab530 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0caffd640 1 -- 172.21.15.17:0/2487855196 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40a3620 con 0x56122e1ab530 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.665+0000 7fa0caffd640 1 -- 172.21.15.17:0/2487855196 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40ce410 con 0x56122e1ab530 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 -- 172.21.15.17:0/2487855196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 msgr2=0x56122e1ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1ab910 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40cbbe0 tx=0x7fa0d409aae0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 -- 172.21.15.17:0/2487855196 shutdown_connections 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1b9030 0x56122e1bb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1abee0 0x56122e1b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/2487855196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1ab910 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 -- 172.21.15.17:0/2487855196 >> 172.21.15.17:0/2487855196 conn(0x56122e1a5e50 msgr2=0x56122e1a6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 -- 172.21.15.17:0/2487855196 shutdown_connections 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 -- 172.21.15.17:0/2487855196 wait complete. 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 -- start start 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.666+0000 7fa0e9c66740 1 Processor -- start 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e0aefa0 con 0x56122e1ab530 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122dde6250 con 0x56122e1abee0 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e098200 con 0x56122e1b9030 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e9c66740 10 client.0 _ll_register_callbacks cb 0x56122e107b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59400/0 (socket says 172.21.15.17:59400) 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/4135508585 learned_addr learned my addr 172.21.15.17:0/4135508585 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0dffff640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e562c640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/4135508585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 msgr2=0x56122e1b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 msgr2=0x56122e1b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/4135508585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e01a4710 con 0x56122e1ab530 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0dffff640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1b8990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40cbc40 tx=0x7fa0d4097700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7be95558385ab10 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.667+0000 7fa0bffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0bffff640 1 -- 172.21.15.17:0/4135508585 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4003760 con 0x56122e1ab530 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0bffff640 1 -- 172.21.15.17:0/4135508585 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa0d40038c0 con 0x56122e1ab530 2024-02-14T15:07:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56122de12600 con 0x56122e1ab530 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0bffff640 1 -- 172.21.15.17:0/4135508585 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0d4084020 con 0x56122e1ab530 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 10 client.16632 fetch_fsmap learned FSMap version 28 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56122e110d70 con 0x56122e1ab530 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0bffff640 1 -- 172.21.15.17:0/4135508585 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40a9020 con 0x56122e1ab530 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0bffff640 1 -- 172.21.15.17:0/4135508585 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa0d40b3bb0 con 0x56122e1ab530 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 20 client.16632 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e1c89f0 0x56122e1cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 4 client.16632 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56122de631b0 con 0x56122e1c89f0 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.668+0000 7fa0dffff640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e1c89f0 0x56122e1cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0bffff640 10 client.16632 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0dffff640 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e1c89f0 0x56122e1cade0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323508bda6ce3613 server_cookie=a6961e4678e58bf2 in_seq=0 out_seq=0 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0bffff640 1 -- 172.21.15.17:0/4135508585 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56122de631b0 con 0x56122e1c89f0 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0bffff640 10 client.16632 handle_command_reply: tid=0 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 client.16632 shutdown 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 msgr2=0x56122e1b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1b8990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40cbc40 tx=0x7fa0d4097700 comp rx=0 tx=0).stop 2024-02-14T15:07:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 shutdown_connections 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e1c89f0 0x56122e1cade0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.669+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/4135508585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1ab530 0x56122e1b8990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.670+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 >> 172.21.15.17:0/4135508585 conn(0x56122e1a5e50 msgr2=0x56122e1a7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.670+0000 7fa0bffff640 0 client.16632 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.670+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 shutdown_connections 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.670+0000 7fa0e9c66740 1 -- 172.21.15.17:0/4135508585 wait complete. 2024-02-14T15:07:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.670+0000 7fa0e9c66740 20 client.16632 trim_cache size 0 max 16384 2024-02-14T15:07:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.693+0000 7fa0e9c66740 1 Processor -- start 2024-02-14T15:07:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.694+0000 7fa0e9c66740 1 -- start start 2024-02-14T15:07:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.694+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1ab530 0x56122e1b8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.694+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.694+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1b9030 0x56122e1b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.694+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1a42d0 con 0x56122e1b9030 2024-02-14T15:07:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.694+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1aaa80 con 0x56122e1abee0 2024-02-14T15:07:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.695+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1aaf50 con 0x56122e1ab530 2024-02-14T15:07:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.695+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1ab530 0x56122e1b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.695+0000 7fa0e4e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1ab530 0x56122e1b8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:52400/0 (socket says 172.21.15.17:52400) 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.695+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/197361752 learned_addr learned my addr 172.21.15.17:0/197361752 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.695+0000 7fa0e562c640 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1b9030 0x56122e1b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.696+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/197361752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 msgr2=0x56122e1b8990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.696+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b8990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.696+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/197361752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1b9030 msgr2=0x56122e1b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.696+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1b9030 0x56122e1b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.696+0000 7fa0e4e2b640 1 -- 172.21.15.17:0/197361752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56122e0ad820 con 0x56122e1ab530 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.696+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1ab530 0x56122e1b8450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c760 tx=0x7fa0d4003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef446c632d7e089 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.697+0000 7fa0ca7fc640 1 -- 172.21.15.17:0/197361752 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d409aa00 con 0x56122e1ab530 2024-02-14T15:07:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.697+0000 7fa0ca7fc640 1 -- 172.21.15.17:0/197361752 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40a9030 con 0x56122e1ab530 2024-02-14T15:07:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.697+0000 7fa0ca7fc640 1 -- 172.21.15.17:0/197361752 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40b3d20 con 0x56122e1ab530 2024-02-14T15:07:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.699+0000 7fa0e9c66740 1 -- 172.21.15.17:0/197361752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1ab530 msgr2=0x56122e1b8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.699+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1ab530 0x56122e1b8450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c760 tx=0x7fa0d4003ef0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.700+0000 7fa0e9c66740 1 -- 172.21.15.17:0/197361752 shutdown_connections 2024-02-14T15:07:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.700+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1b9030 0x56122e1b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.700+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1abee0 0x56122e1b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.700+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/197361752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1ab530 0x56122e1b8450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.700+0000 7fa0e9c66740 1 -- 172.21.15.17:0/197361752 >> 172.21.15.17:0/197361752 conn(0x56122e1a5e50 msgr2=0x56122e1b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.708+0000 7fa0e9c66740 1 -- 172.21.15.17:0/197361752 shutdown_connections 2024-02-14T15:07:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.709+0000 7fa0e9c66740 1 -- 172.21.15.17:0/197361752 wait complete. 2024-02-14T15:07:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.709+0000 7fa0e9c66740 1 -- start start 2024-02-14T15:07:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.709+0000 7fa0e9c66740 1 Processor -- start 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1ab530 0x56122e1bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1abee0 0x56122e1c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e9c66740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1be8c0 con 0x56122e1abee0 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1bea30 con 0x56122e1ab530 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e9c66740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122e1bebf0 con 0x56122e1b9030 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e9c66740 10 client.0 _ll_register_callbacks cb 0x7fa0d4096a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e562c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e562c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1be300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:52402/0 (socket says 172.21.15.17:52402) 2024-02-14T15:07:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0e562c640 1 -- 172.21.15.17:0/3645837976 learned_addr learned my addr 172.21.15.17:0/3645837976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.710+0000 7fa0dffff640 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1abee0 0x56122e1c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1ab530 0x56122e1bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0dffff640 1 -- 172.21.15.17:0/3645837976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 msgr2=0x56122e1be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0dffff640 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1be300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0dffff640 1 -- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1ab530 msgr2=0x56122e1bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0dffff640 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1ab530 0x56122e1bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0dffff640 1 -- 172.21.15.17:0/3645837976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56122e110d70 con 0x56122e1abee0 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0dffff640 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1abee0 0x56122e1c01b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc011ce0 tx=0x7fa0cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3856580978030c55 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.711+0000 7fa0bffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.712+0000 7fa0bffff640 1 -- 172.21.15.17:0/3645837976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc024040 con 0x56122e1abee0 2024-02-14T15:07:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.712+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0d40a3620 con 0x56122e1abee0 2024-02-14T15:07:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.712+0000 7fa0bffff640 1 -- 172.21.15.17:0/3645837976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa0cc029620 con 0x56122e1abee0 2024-02-14T15:07:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.714+0000 7fa0bffff640 1 -- 172.21.15.17:0/3645837976 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0cc0297e0 con 0x56122e1abee0 2024-02-14T15:07:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.714+0000 7fa0e9c66740 10 client.16638 fetch_fsmap learned FSMap version 28 2024-02-14T15:07:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.714+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56122e0d3720 con 0x56122e1abee0 2024-02-14T15:07:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0bffff640 1 -- 172.21.15.17:0/3645837976 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40a3620 con 0x56122e1abee0 2024-02-14T15:07:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0bffff640 1 -- 172.21.15.17:0/3645837976 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa0cc01e540 con 0x56122e1abee0 2024-02-14T15:07:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0e9c66740 10 client.16638 fetch_fsmap finished waiting for FSMap version 28 2024-02-14T15:07:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0e9c66740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0e9c66740 20 client.16638 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e220240 0x56122e222660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0e9c66740 4 client.16638 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:07:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.715+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56122de4f550 con 0x56122e220240 2024-02-14T15:07:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.716+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e220240 0x56122e222660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.718+0000 7fa0e4e2b640 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e220240 0x56122e222660 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91a218520b87046 server_cookie=a6badd6dd155c25d in_seq=0 out_seq=0 2024-02-14T15:07:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.718+0000 7fa0bffff640 10 client.16638 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0bffff640 1 -- 172.21.15.17:0/3645837976 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56122de4f550 con 0x56122e220240 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0bffff640 10 client.16638 handle_command_reply: tid=0 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 client.16638 shutdown 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1abee0 msgr2=0x56122e1c01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1abee0 0x56122e1c01b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc011ce0 tx=0x7fa0cc004730 comp rx=0 tx=0).stop 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 shutdown_connections 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56122e220240 0x56122e222660 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56122e1abee0 0x56122e1c01b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56122e1b9030 0x56122e1be300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 --2- 172.21.15.17:0/3645837976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56122e1ab530 0x56122e1bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 >> 172.21.15.17:0/3645837976 conn(0x56122e1a5e50 msgr2=0x56122e1b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0bffff640 0 client.16638 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 shutdown_connections 2024-02-14T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 1 -- 172.21.15.17:0/3645837976 wait complete. 2024-02-14T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.754+0000 7fa0e9c66740 20 client.16638 trim_cache size 0 max 16384 2024-02-14T15:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:07:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:07:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:07:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa0d00a22a0 msgr2=0x7fa0d00a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa0d00a22a0 0x7fa0d00a4760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0198ec0 tx=0x7fa0cc005f90 comp rx=0 tx=0).stop 2024-02-14T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100340 msgr2=0x7fa0e019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100340 0x7fa0e019d5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400ea60 tx=0x7fa0d400ef20 comp rx=0 tx=0).stop 2024-02-14T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 shutdown_connections 2024-02-14T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa0d00a22a0 0x7fa0d00a4760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e0197670 0x7fa0e0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0e0105410 0x7fa0e019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 --2- 172.21.15.17:0/3752432640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e0100340 0x7fa0e019d5b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 >> 172.21.15.17:0/3752432640 conn(0x7fa0e0077220 msgr2=0x7fa0e0068d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 shutdown_connections 2024-02-14T15:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:42.755+0000 7fa0e708f640 1 -- 172.21.15.17:0/3752432640 wait complete. 2024-02-14T15:07:42.866 INFO:tasks.workunit.client.0.smithi017.stdout: 1 367893 12.78 MB/sec execute 44 sec latency 56.785 ms 2024-02-14T15:07:42.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:42 smithi076 ceph-mon[37057]: pgmap v131: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-02-14T15:07:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:42 smithi017 ceph-mon[32549]: pgmap v131: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-02-14T15:07:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:42 smithi050 ceph-mon[36512]: pgmap v131: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-02-14T15:07:43.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 369686 12.76 MB/sec execute 45 sec latency 51.097 ms 2024-02-14T15:07:44.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 372657 12.88 MB/sec execute 46 sec latency 66.464 ms 2024-02-14T15:07:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:44 smithi076 ceph-mon[37057]: pgmap v132: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-02-14T15:07:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:44 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:07:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:44 smithi017 ceph-mon[32549]: pgmap v132: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-02-14T15:07:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:44 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:07:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:44 smithi050 ceph-mon[36512]: pgmap v132: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-02-14T15:07:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:44 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:07:45.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 375189 12.84 MB/sec execute 47 sec latency 14.141 ms 2024-02-14T15:07:46.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 377129 12.79 MB/sec execute 48 sec latency 14.732 ms 2024-02-14T15:07:46.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:46 smithi076 ceph-mon[37057]: pgmap v133: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-02-14T15:07:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:46 smithi017 ceph-mon[32549]: pgmap v133: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-02-14T15:07:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:46 smithi050 ceph-mon[36512]: pgmap v133: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-02-14T15:07:47.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 379928 12.88 MB/sec execute 49 sec latency 43.614 ms 2024-02-14T15:07:48.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 382049 12.77 MB/sec execute 50 sec latency 54.624 ms 2024-02-14T15:07:48.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:48 smithi076 ceph-mon[37057]: pgmap v134: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 347 op/s 2024-02-14T15:07:49.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:48 smithi017 ceph-mon[32549]: pgmap v134: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 347 op/s 2024-02-14T15:07:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:48 smithi050 ceph-mon[36512]: pgmap v134: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 347 op/s 2024-02-14T15:07:49.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 384292 12.79 MB/sec execute 51 sec latency 14.273 ms 2024-02-14T15:07:50.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 387131 12.87 MB/sec execute 52 sec latency 14.141 ms 2024-02-14T15:07:50.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:50 smithi076 ceph-mon[37057]: pgmap v135: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-02-14T15:07:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:50 smithi017 ceph-mon[32549]: pgmap v135: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-02-14T15:07:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:50 smithi050 ceph-mon[36512]: pgmap v135: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-02-14T15:07:51.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 389646 12.86 MB/sec execute 53 sec latency 105.890 ms 2024-02-14T15:07:52.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 391828 12.84 MB/sec execute 54 sec latency 14.662 ms 2024-02-14T15:07:52.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:52 smithi076 ceph-mon[37057]: pgmap v136: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-02-14T15:07:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:52 smithi017 ceph-mon[32549]: pgmap v136: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-02-14T15:07:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:52 smithi050 ceph-mon[36512]: pgmap v136: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.2 MiB/s wr, 356 op/s 2024-02-14T15:07:53.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 394085 12.86 MB/sec execute 55 sec latency 50.756 ms 2024-02-14T15:07:54.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 396761 12.86 MB/sec execute 56 sec latency 10.843 ms 2024-02-14T15:07:54.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:54 smithi076 ceph-mon[37057]: pgmap v137: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-02-14T15:07:55.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:54 smithi017 ceph-mon[32549]: pgmap v137: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-02-14T15:07:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:54 smithi050 ceph-mon[36512]: pgmap v137: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-02-14T15:07:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 -- 172.21.15.17:0/3235694022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880100380 msgr2=0x7f0880100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 --2- 172.21.15.17:0/3235694022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880100380 0x7f0880100760 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0870009cd0 tx=0x7f087002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 -- 172.21.15.17:0/3235694022 shutdown_connections 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 --2- 172.21.15.17:0/3235694022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0880105450 0x7f088010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 --2- 172.21.15.17:0/3235694022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880100d30 0x7f0880104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 --2- 172.21.15.17:0/3235694022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880100380 0x7f0880100760 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 -- 172.21.15.17:0/3235694022 >> 172.21.15.17:0/3235694022 conn(0x7f0880077250 msgr2=0x7f0880077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 -- 172.21.15.17:0/3235694022 shutdown_connections 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 -- 172.21.15.17:0/3235694022 wait complete. 2024-02-14T15:07:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.844+0000 7f08854d4640 1 Processor -- start 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f08854d4640 1 -- start start 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f08854d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0880100380 0x7f0880101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f08854d4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880105450 0x7f0880102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f08854d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880102890 0x7f08801a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f08854d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801a6090 con 0x7f0880102890 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f08854d4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801a6200 con 0x7f0880105450 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f08854d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801a63c0 con 0x7f0880100380 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880102890 0x7f08801a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880105450 0x7f0880102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0880100380 0x7f0880101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880105450 0x7f0880102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:41498/0 (socket says 172.21.15.17:41498) 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087e7fc640 1 -- 172.21.15.17:0/1516540900 learned_addr learned my addr 172.21.15.17:0/1516540900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880102890 0x7f08801a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45836/0 (socket says 172.21.15.17:45836) 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087effd640 1 -- 172.21.15.17:0/1516540900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880105450 msgr2=0x7f0880102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087effd640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880105450 0x7f0880102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087effd640 1 -- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880102890 msgr2=0x7f08801a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087effd640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880102890 0x7f08801a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087effd640 1 -- 172.21.15.17:0/1516540900 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0870009900 con 0x7f0880100380 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087e7fc640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880105450 0x7f0880102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087f7fe640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880102890 0x7f08801a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.845+0000 7f087effd640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0880100380 0x7f0880101e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f087002c700 tx=0x7f08700325d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.846+0000 7f0867fff640 1 -- 172.21.15.17:0/1516540900 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700328b0 con 0x7f0880100380 2024-02-14T15:07:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.846+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08801a6640 con 0x7f0880100380 2024-02-14T15:07:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.846+0000 7f0867fff640 1 -- 172.21.15.17:0/1516540900 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0870003e50 con 0x7f0880100380 2024-02-14T15:07:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.846+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08801a6b10 con 0x7f0880100380 2024-02-14T15:07:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.846+0000 7f0867fff640 1 -- 172.21.15.17:0/1516540900 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870004800 con 0x7f0880100380 2024-02-14T15:07:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0867fff640 1 -- 172.21.15.17:0/1516540900 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f087000a040 con 0x7f0880100380 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 Processor -- start 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 -- start start 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab7720 0x55ff63ab7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab8040 0x55ff63ac06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63ac3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63a259e0 con 0x55ff63ab8040 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63a22f10 con 0x55ff63ab7720 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.848+0000 7f0888184740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63a1e6f0 con 0x55ff63ac0c20 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f0867fff640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08680a2250 0x7f08680a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab8040 0x55ff63ac06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63ac3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab8040 0x55ff63ac06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45848/0 (socket says 172.21.15.17:45848) 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 -- 172.21.15.17:0/140947441 learned_addr learned my addr 172.21.15.17:0/140947441 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087f7fe640 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab7720 0x55ff63ab7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087f7fe640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08680a2250 0x7f08680a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f0867fff640 1 -- 172.21.15.17:0/1516540900 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f08700f0e20 con 0x7f0880100380 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 -- 172.21.15.17:0/140947441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 msgr2=0x55ff63ac3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63ac3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 -- 172.21.15.17:0/140947441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab7720 msgr2=0x55ff63ab7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab7720 0x55ff63ab7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 -- 172.21.15.17:0/140947441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6372f920 con 0x55ff63ab8040 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087f7fe640 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab7720 0x55ff63ab7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087effd640 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab8040 0x55ff63ac06e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f08700c5830 tx=0x7f0870004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ca8d5086e0789b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f087f7fe640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08680a2250 0x7f08680a4710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0878009fd0 tx=0x7f0878005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f08657fa640 1 -- 172.21.15.17:0/140947441 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700bdd50 con 0x55ff63ab8040 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.849+0000 7f08657fa640 1 -- 172.21.15.17:0/140947441 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08700bd650 con 0x55ff63ab8040 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f08657fa640 1 -- 172.21.15.17:0/140947441 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870004040 con 0x55ff63ab8040 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 -- 172.21.15.17:0/140947441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab8040 msgr2=0x55ff63ac06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab8040 0x55ff63ac06e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f08700c5830 tx=0x7f0870004300 comp rx=0 tx=0).stop 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 -- 172.21.15.17:0/140947441 shutdown_connections 2024-02-14T15:07:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab8040 0x55ff63ac06e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63ac3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 --2- 172.21.15.17:0/140947441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab7720 0x55ff63ab7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 -- 172.21.15.17:0/140947441 >> 172.21.15.17:0/140947441 conn(0x55ff63ab1e50 msgr2=0x55ff63ab2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 -- 172.21.15.17:0/140947441 shutdown_connections 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.850+0000 7f0888184740 1 -- 172.21.15.17:0/140947441 wait complete. 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 -- start start 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 Processor -- start 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63abd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff639bafa0 con 0x55ff63ab7720 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff636f2250 con 0x55ff63ac0c20 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff639a4200 con 0x55ff63ab8040 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0888184740 10 client.0 _ll_register_callbacks cb 0x55ff63a13b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45850/0 (socket says 172.21.15.17:45850) 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63abd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 -- 172.21.15.17:0/76747856 learned_addr learned my addr 172.21.15.17:0/76747856 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087f7fe640 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 -- 172.21.15.17:0/76747856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 msgr2=0x55ff63abd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63abd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 -- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 msgr2=0x55ff63abde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 -- 172.21.15.17:0/76747856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08801a6b10 con 0x55ff63ab7720 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f087effd640 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0540 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0870005cc0 tx=0x7f08700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bda1708c6d5b8d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.851+0000 7f0862ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0862ff5640 1 -- 172.21.15.17:0/76747856 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700c17a0 con 0x55ff63ab7720 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0862ff5640 1 -- 172.21.15.17:0/76747856 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f08700bdd50 con 0x55ff63ab7720 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff6371e600 con 0x55ff63ab7720 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0862ff5640 1 -- 172.21.15.17:0/76747856 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08700b9cc0 con 0x55ff63ab7720 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 10 client.16650 fetch_fsmap learned FSMap version 29 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff63a1cd70 con 0x55ff63ab7720 2024-02-14T15:07:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0862ff5640 1 -- 172.21.15.17:0/76747856 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ff6371e600 con 0x55ff63ab7720 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0862ff5640 1 -- 172.21.15.17:0/76747856 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f08700c58d0 con 0x55ff63ab7720 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 20 client.16650 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63ad0210 0x55ff63ad2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 4 client.16650 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff6376f1b0 con 0x55ff63ad0210 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.852+0000 7f087e7fc640 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63ad0210 0x55ff63ad2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.853+0000 7f087e7fc640 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63ad0210 0x55ff63ad2600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79168ea78bb1cc5 server_cookie=7b4bdd967653d7b8 in_seq=0 out_seq=0 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.853+0000 7f0862ff5640 10 client.16650 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.853+0000 7f0862ff5640 1 -- 172.21.15.17:0/76747856 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff6376f1b0 con 0x55ff63ad0210 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.853+0000 7f0862ff5640 10 client.16650 handle_command_reply: tid=0 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.853+0000 7f0888184740 1 client.16650 shutdown 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.853+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 msgr2=0x55ff63ac0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.853+0000 7f0888184740 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0540 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0870005cc0 tx=0x7f08700095c0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 shutdown_connections 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63ad0210 0x55ff63ad2600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63abd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0862ff5640 0 client.16650 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 --2- 172.21.15.17:0/76747856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0540 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 >> 172.21.15.17:0/76747856 conn(0x55ff63ab1e50 msgr2=0x55ff63ab3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 shutdown_connections 2024-02-14T15:07:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 1 -- 172.21.15.17:0/76747856 wait complete. 2024-02-14T15:07:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.854+0000 7f0888184740 20 client.16650 trim_cache size 0 max 16384 2024-02-14T15:07:55.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 398759 12.84 MB/sec execute 57 sec latency 16.418 ms 2024-02-14T15:07:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 Processor -- start 2024-02-14T15:07:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 -- start start 2024-02-14T15:07:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63ac0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63aad5d0 con 0x55ff63ab7720 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63ab2e80 con 0x55ff63ac0c20 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0888184740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63ab1d00 con 0x55ff63ab8040 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45864/0 (socket says 172.21.15.17:45864) 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 -- 172.21.15.17:0/1465939781 learned_addr learned my addr 172.21.15.17:0/1465939781 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087e7fc640 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63ac0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087f7fe640 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 -- 172.21.15.17:0/1465939781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 msgr2=0x55ff63ac0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63ac0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 -- 172.21.15.17:0/1465939781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 msgr2=0x55ff63abab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 -- 172.21.15.17:0/1465939781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff639b9820 con 0x55ff63ab7720 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087f7fe640 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f087effd640 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f08700ee810 tx=0x7f08700bd550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c540c3f82903e882 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.869+0000 7f0864ff9640 1 -- 172.21.15.17:0/1465939781 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700c17a0 con 0x55ff63ab7720 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0864ff9640 1 -- 172.21.15.17:0/1465939781 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08700d6120 con 0x55ff63ab7720 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0864ff9640 1 -- 172.21.15.17:0/1465939781 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700c5d60 con 0x55ff63ab7720 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 -- 172.21.15.17:0/1465939781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 msgr2=0x55ff63ac0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f08700ee810 tx=0x7f08700bd550 comp rx=0 tx=0).stop 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 -- 172.21.15.17:0/1465939781 shutdown_connections 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ac0c20 0x55ff63abab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ab8040 0x55ff63ac0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 --2- 172.21.15.17:0/1465939781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac0000 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 -- 172.21.15.17:0/1465939781 >> 172.21.15.17:0/1465939781 conn(0x55ff63ab1e50 msgr2=0x55ff63ab0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 -- 172.21.15.17:0/1465939781 shutdown_connections 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 -- 172.21.15.17:0/1465939781 wait complete. 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.870+0000 7f0888184740 1 -- start start 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 1 Processor -- start 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab8040 0x55ff63ac6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63b266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63ac7620 con 0x55ff63ab7720 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63b26c10 con 0x55ff63ab8040 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63b26d50 con 0x55ff63ac0c20 2024-02-14T15:07:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0888184740 10 client.0 _ll_register_callbacks cb 0x7f08700bd4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45876/0 (socket says 172.21.15.17:45876) 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 -- 172.21.15.17:0/15868865 learned_addr learned my addr 172.21.15.17:0/15868865 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087f7fe640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63b266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087e7fc640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab8040 0x55ff63ac6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 -- 172.21.15.17:0/15868865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 msgr2=0x55ff63b266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63b266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 -- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab8040 msgr2=0x55ff63ac6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab8040 0x55ff63ac6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 -- 172.21.15.17:0/15868865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff63a1cd70 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087e7fc640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab8040 0x55ff63ac6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f087effd640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac69e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f08700ba020 tx=0x7f0870004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709d4ab40ab36fca server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.871+0000 7f0862ff5640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0862ff5640 1 -- 172.21.15.17:0/15868865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700a6c60 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0862ff5640 1 -- 172.21.15.17:0/15868865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f08700bdd50 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08700bd650 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0862ff5640 1 -- 172.21.15.17:0/15868865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08700b98f0 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 10 client.16656 fetch_fsmap learned FSMap version 29 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff639df720 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0862ff5640 1 -- 172.21.15.17:0/15868865 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08700bd650 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0862ff5640 1 -- 172.21.15.17:0/15868865 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0870004bd0 con 0x55ff63ab7720 2024-02-14T15:07:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 10 client.16656 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 20 client.16656 populate_metadata read hostname 'smithi017' 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63b28000 0x55ff63b2a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 4 client.16656 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff6375b550 con 0x55ff63b28000 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.872+0000 7f087e7fc640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63b28000 0x55ff63b2a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.873+0000 7f087e7fc640 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63b28000 0x55ff63b2a3f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bee2ae4b2d8e6c server_cookie=a1ba75d52a34c1a7 in_seq=0 out_seq=0 2024-02-14T15:07:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.873+0000 7f0862ff5640 10 client.16656 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0862ff5640 1 -- 172.21.15.17:0/15868865 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff6375b550 con 0x55ff63b28000 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0862ff5640 10 client.16656 handle_command_reply: tid=0 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 client.16656 shutdown 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 msgr2=0x55ff63ac69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac69e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f08700ba020 tx=0x7f0870004300 comp rx=0 tx=0).stop 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 shutdown_connections 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ff63b28000 0x55ff63b2a3f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ff63ac0c20 0x55ff63b266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ff63ab8040 0x55ff63ac6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 --2- 172.21.15.17:0/15868865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff63ab7720 0x55ff63ac69e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0862ff5640 0 client.16656 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 >> 172.21.15.17:0/15868865 conn(0x55ff63ab1e50 msgr2=0x55ff63ab8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 shutdown_connections 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 1 -- 172.21.15.17:0/15868865 wait complete. 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.882+0000 7f0888184740 20 client.16656 trim_cache size 0 max 16384 2024-02-14T15:07:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:07:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:07:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:07:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08680a2250 msgr2=0x7f08680a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08680a2250 0x7f08680a4710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0878009fd0 tx=0x7f0878005f90 comp rx=0 tx=0).stop 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0880100380 msgr2=0x7f0880101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0880100380 0x7f0880101e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f087002c700 tx=0x7f08700325d0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 shutdown_connections 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08680a2250 0x7f08680a4710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0880102890 0x7f08801a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0880105450 0x7f0880102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 --2- 172.21.15.17:0/1516540900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0880100380 0x7f0880101e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:07:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 >> 172.21.15.17:0/1516540900 conn(0x7f0880077250 msgr2=0x7f0880069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:07:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 shutdown_connections 2024-02-14T15:07:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:07:55.883+0000 7f08854d4640 1 -- 172.21.15.17:0/1516540900 wait complete. 2024-02-14T15:07:56.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 401289 12.86 MB/sec execute 58 sec latency 88.435 ms 2024-02-14T15:07:56.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:56 smithi076 ceph-mon[37057]: pgmap v138: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-02-14T15:07:57.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:56 smithi017 ceph-mon[32549]: pgmap v138: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-02-14T15:07:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:56 smithi050 ceph-mon[36512]: pgmap v138: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-02-14T15:07:57.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 404045 12.87 MB/sec execute 59 sec latency 75.142 ms 2024-02-14T15:07:58.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 405472 12.78 MB/sec execute 60 sec latency 43.854 ms 2024-02-14T15:07:58.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:07:58 smithi076 ceph-mon[37057]: pgmap v139: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2024-02-14T15:07:59.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:07:58 smithi017 ceph-mon[32549]: pgmap v139: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2024-02-14T15:07:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:07:58 smithi050 ceph-mon[36512]: pgmap v139: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2024-02-14T15:07:59.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 408394 12.85 MB/sec execute 61 sec latency 7.511 ms 2024-02-14T15:08:00.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 411182 12.86 MB/sec execute 62 sec latency 37.240 ms 2024-02-14T15:08:00.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:00 smithi076 ceph-mon[37057]: pgmap v140: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-02-14T15:08:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:00 smithi017 ceph-mon[32549]: pgmap v140: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-02-14T15:08:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:00 smithi050 ceph-mon[36512]: pgmap v140: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-02-14T15:08:01.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 411829 12.75 MB/sec execute 63 sec latency 59.917 ms 2024-02-14T15:08:02.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 414412 12.74 MB/sec execute 64 sec latency 23.563 ms 2024-02-14T15:08:02.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:02 smithi076 ceph-mon[37057]: pgmap v141: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.9 MiB/s wr, 367 op/s 2024-02-14T15:08:03.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:02 smithi017 ceph-mon[32549]: pgmap v141: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.9 MiB/s wr, 367 op/s 2024-02-14T15:08:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:02 smithi050 ceph-mon[36512]: pgmap v141: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.9 MiB/s wr, 367 op/s 2024-02-14T15:08:03.867 INFO:tasks.workunit.client.0.smithi017.stdout: 1 415990 12.66 MB/sec execute 65 sec latency 96.636 ms 2024-02-14T15:08:04.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 418855 12.74 MB/sec execute 66 sec latency 24.287 ms 2024-02-14T15:08:04.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:04 smithi076 ceph-mon[37057]: pgmap v142: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-02-14T15:08:05.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:04 smithi017 ceph-mon[32549]: pgmap v142: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-02-14T15:08:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:04 smithi050 ceph-mon[36512]: pgmap v142: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-02-14T15:08:05.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 421342 12.72 MB/sec execute 67 sec latency 18.842 ms 2024-02-14T15:08:06.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 423055 12.67 MB/sec execute 68 sec latency 72.660 ms 2024-02-14T15:08:06.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:06 smithi076 ceph-mon[37057]: pgmap v143: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-02-14T15:08:07.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:06 smithi017 ceph-mon[32549]: pgmap v143: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-02-14T15:08:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:06 smithi050 ceph-mon[36512]: pgmap v143: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-02-14T15:08:07.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 426037 12.75 MB/sec execute 69 sec latency 17.352 ms 2024-02-14T15:08:08.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 427058 12.64 MB/sec execute 70 sec latency 298.625 ms 2024-02-14T15:08:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 -- 172.21.15.17:0/1655892664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c1062c0 msgr2=0x7fc26c1066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 --2- 172.21.15.17:0/1655892664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c1062c0 0x7fc26c1066a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009930 tx=0x7fc25c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:08:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 -- 172.21.15.17:0/1655892664 shutdown_connections 2024-02-14T15:08:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 --2- 172.21.15.17:0/1655892664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c100d40 0x7fc26c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 --2- 172.21.15.17:0/1655892664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc26c100380 0x7fc26c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 --2- 172.21.15.17:0/1655892664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c1062c0 0x7fc26c1066a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 -- 172.21.15.17:0/1655892664 >> 172.21.15.17:0/1655892664 conn(0x7fc26c076ef0 msgr2=0x7fc26c077300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 -- 172.21.15.17:0/1655892664 shutdown_connections 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.972+0000 7fc270c23640 1 -- 172.21.15.17:0/1655892664 wait complete. 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 Processor -- start 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 -- start start 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c100380 0x7fc26c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc26c100d40 0x7fc26c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c1062c0 0x7fc26c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c197c60 con 0x7fc26c100380 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c197dd0 con 0x7fc26c1062c0 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc270c23640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c197f90 con 0x7fc26c100d40 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c100380 0x7fc26c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c100380 0x7fc26c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42950/0 (socket says 172.21.15.17:42950) 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 -- 172.21.15.17:0/695006002 learned_addr learned my addr 172.21.15.17:0/695006002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c1062c0 0x7fc26c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc263fff640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc26c100d40 0x7fc26c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 -- 172.21.15.17:0/695006002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc26c100d40 msgr2=0x7fc26c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc26c100d40 0x7fc26c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 -- 172.21.15.17:0/695006002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c1062c0 msgr2=0x7fc26c197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c1062c0 0x7fc26c197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 -- 172.21.15.17:0/695006002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c009560 con 0x7fc26c100380 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.973+0000 7fc26a59c640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c100380 0x7fc26c19d460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009230 tx=0x7fc25c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.974+0000 7fc2637fe640 1 -- 172.21.15.17:0/695006002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c004440 con 0x7fc26c100380 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.974+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c198210 con 0x7fc26c100380 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.974+0000 7fc2637fe640 1 -- 172.21.15.17:0/695006002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc25c0052d0 con 0x7fc26c100380 2024-02-14T15:08:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.974+0000 7fc2637fe640 1 -- 172.21.15.17:0/695006002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c003d70 con 0x7fc26c100380 2024-02-14T15:08:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.974+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c1a4500 con 0x7fc26c100380 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.976+0000 7fc2637fe640 1 -- 172.21.15.17:0/695006002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc25c005440 con 0x7fc26c100380 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.976+0000 7fc2737f4740 1 Processor -- start 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.976+0000 7fc2737f4740 1 -- start start 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.976+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc50720 0x55942bc50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.976+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc59c20 0x55942bc5c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc2737f4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bbbe9e0 con 0x55942bc50720 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc2737f4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bbbbf10 con 0x55942bc51040 2024-02-14T15:08:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc2737f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bbb76f0 con 0x55942bc59c20 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc2637fe640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc2580a2220 0x7fc2580a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc50720 0x55942bc50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc50720 0x55942bc50b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42958/0 (socket says 172.21.15.17:42958) 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 -- 172.21.15.17:0/941855996 learned_addr learned my addr 172.21.15.17:0/941855996 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc2580a2220 0x7fc2580a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc263fff640 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc59c20 0x55942bc5c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26a59c640 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc2637fe640 1 -- 172.21.15.17:0/695006002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc25c0f04b0 con 0x7fc26c100380 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 -- 172.21.15.17:0/941855996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc59c20 msgr2=0x55942bc5c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc59c20 0x55942bc5c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 -- 172.21.15.17:0/941855996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 msgr2=0x55942bc596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 -- 172.21.15.17:0/941855996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55942b8c8920 con 0x55942bc50720 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc50720 0x55942bc50b00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc264014f10 tx=0x7fc2640117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a2b0334b34b60a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc2580a2220 0x7fc2580a46e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc264005c40 tx=0x7fc264016000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc260ff9640 1 -- 172.21.15.17:0/941855996 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26400cdf0 con 0x55942bc50720 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc260ff9640 1 -- 172.21.15.17:0/941855996 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc26401a040 con 0x55942bc50720 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.977+0000 7fc260ff9640 1 -- 172.21.15.17:0/941855996 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264005790 con 0x55942bc50720 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 -- 172.21.15.17:0/941855996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc50720 msgr2=0x55942bc50b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc50720 0x55942bc50b00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc264014f10 tx=0x7fc2640117b0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 -- 172.21.15.17:0/941855996 shutdown_connections 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc59c20 0x55942bc5c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 --2- 172.21.15.17:0/941855996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc50720 0x55942bc50b00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 -- 172.21.15.17:0/941855996 >> 172.21.15.17:0/941855996 conn(0x55942bc4ae50 msgr2=0x55942bc4b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 -- 172.21.15.17:0/941855996 shutdown_connections 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.978+0000 7fc2737f4740 1 -- 172.21.15.17:0/941855996 wait complete. 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 -- start start 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 Processor -- start 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc59640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc56fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bb53fa0 con 0x55942bc59c20 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b88b250 con 0x55942bc50720 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bb3d200 con 0x55942bc51040 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc2737f4740 10 client.0 _ll_register_callbacks cb 0x55942bbacb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc56fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc56fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42962/0 (socket says 172.21.15.17:42962) 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1686358767 learned_addr learned my addr 172.21.15.17:0/1686358767 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26a59c640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc59640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc263fff640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1686358767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 msgr2=0x55942bc569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 msgr2=0x55942bc59640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc59640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1686358767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc264014af0 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.979+0000 7fc26a59c640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc59640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc56fe0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc264005220 tx=0x7fc264005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdcb2c1c7475e86 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc251ffb640 1 -- 172.21.15.17:0/1686358767 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264005710 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc251ffb640 1 -- 172.21.15.17:0/1686358767 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc264003970 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55942b8b7600 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc251ffb640 1 -- 172.21.15.17:0/1686358767 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc264003b30 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 10 client.16668 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55942bbb5d70 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc251ffb640 1 -- 172.21.15.17:0/1686358767 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55942b8b7600 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc251ffb640 1 -- 172.21.15.17:0/1686358767 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc26401d070 con 0x55942bc59c20 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 20 client.16668 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bc69a30 0x55942bc6be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 4 client.16668 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.980+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55942b9081b0 con 0x55942bc69a30 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.981+0000 7fc26a59c640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bc69a30 0x55942bc6be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.981+0000 7fc26a59c640 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bc69a30 0x55942bc6be50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170dc45635735dce server_cookie=f68e9c463f350eec in_seq=0 out_seq=0 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.981+0000 7fc251ffb640 10 client.16668 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.981+0000 7fc251ffb640 1 -- 172.21.15.17:0/1686358767 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55942b9081b0 con 0x55942bc69a30 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.981+0000 7fc251ffb640 10 client.16668 handle_command_reply: tid=0 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.981+0000 7fc2737f4740 1 client.16668 shutdown 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 msgr2=0x55942bc56fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc56fe0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc264005220 tx=0x7fc264005980 comp rx=0 tx=0).stop 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 shutdown_connections 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bc69a30 0x55942bc6be50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc56fe0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc251ffb640 0 client.16668 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1686358767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc59640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 >> 172.21.15.17:0/1686358767 conn(0x55942bc4ae50 msgr2=0x55942bc4cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 shutdown_connections 2024-02-14T15:08:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 1 -- 172.21.15.17:0/1686358767 wait complete. 2024-02-14T15:08:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.982+0000 7fc2737f4740 20 client.16668 trim_cache size 0 max 16384 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 Processor -- start 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 -- start start 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc50720 0x55942bc55910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc59100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc59640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bc465a0 con 0x55942bc59c20 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bc4be50 con 0x55942bc51040 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc2737f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bc664d0 con 0x55942bc50720 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc59640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc59640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42964/0 (socket says 172.21.15.17:42964) 2024-02-14T15:08:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 -- 172.21.15.17:0/448046699 learned_addr learned my addr 172.21.15.17:0/448046699 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc263fff640 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc59100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26a59c640 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc50720 0x55942bc55910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 -- 172.21.15.17:0/448046699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc50720 msgr2=0x55942bc55910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc50720 0x55942bc55910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 -- 172.21.15.17:0/448046699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 msgr2=0x55942bc59100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc59100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 -- 172.21.15.17:0/448046699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55942bb52820 con 0x55942bc59c20 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.995+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc59640 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc264008010 tx=0x7fc264007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a186d5b0b3fbff server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc253fff640 1 -- 172.21.15.17:0/448046699 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264005250 con 0x55942bc59c20 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc253fff640 1 -- 172.21.15.17:0/448046699 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc26400ce80 con 0x55942bc59c20 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc253fff640 1 -- 172.21.15.17:0/448046699 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264003970 con 0x55942bc59c20 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 -- 172.21.15.17:0/448046699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 msgr2=0x55942bc59640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc59640 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc264008010 tx=0x7fc264007740 comp rx=0 tx=0).stop 2024-02-14T15:08:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 -- 172.21.15.17:0/448046699 shutdown_connections 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc59640 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc51040 0x55942bc59100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 --2- 172.21.15.17:0/448046699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc50720 0x55942bc55910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 -- 172.21.15.17:0/448046699 >> 172.21.15.17:0/448046699 conn(0x55942bc4ae50 msgr2=0x55942bc4cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 -- 172.21.15.17:0/448046699 shutdown_connections 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 -- 172.21.15.17:0/448046699 wait complete. 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.996+0000 7fc2737f4740 1 -- start start 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 1 Processor -- start 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc60750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc60c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc5ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bc5f3e0 con 0x55942bc59c20 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bc5f550 con 0x55942bc50720 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942bc5f710 con 0x55942bc51040 2024-02-14T15:08:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc2737f4740 10 client.0 _ll_register_callbacks cb 0x7fc264007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc5ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc60750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc5ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42980/0 (socket says 172.21.15.17:42980) 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc263fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc60c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1766700740 learned_addr learned my addr 172.21.15.17:0/1766700740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc60750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:59772/0 (socket says 172.21.15.17:59772) 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1766700740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 msgr2=0x55942bc60c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc60c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1766700740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 msgr2=0x55942bc60750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc60750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 -- 172.21.15.17:0/1766700740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55942bbb5d70 con 0x55942bc59c20 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc26ad9d640 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc5ee20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc264014ac0 tx=0x7fc264004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6351b17b83712e52 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.997+0000 7fc251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc251ffb640 1 -- 172.21.15.17:0/1766700740 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264003b70 con 0x55942bc59c20 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc251ffb640 1 -- 172.21.15.17:0/1766700740 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc264003cd0 con 0x55942bc59c20 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55942b8f4550 con 0x55942bc59c20 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc251ffb640 1 -- 172.21.15.17:0/1766700740 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc26400ce80 con 0x55942bc59c20 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 10 client.16674 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55942bb78720 con 0x55942bc59c20 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc251ffb640 1 -- 172.21.15.17:0/1766700740 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55942b8f4550 con 0x55942bc59c20 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc251ffb640 1 -- 172.21.15.17:0/1766700740 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc264047070 con 0x55942bc59c20 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 10 client.16674 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 20 client.16674 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bcc0c20 0x55942bcc3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 4 client.16674 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:08:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55942b863520 con 0x55942bcc0c20 2024-02-14T15:08:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.998+0000 7fc26a59c640 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bcc0c20 0x55942bcc3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.999+0000 7fc26a59c640 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bcc0c20 0x55942bcc3010 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f09b8f6f2763de3 server_cookie=440d39b0f0598c91 in_seq=0 out_seq=0 2024-02-14T15:08:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:08.999+0000 7fc251ffb640 10 client.16674 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.007+0000 7fc251ffb640 1 -- 172.21.15.17:0/1766700740 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55942b863520 con 0x55942bcc0c20 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.007+0000 7fc251ffb640 10 client.16674 handle_command_reply: tid=0 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.007+0000 7fc2737f4740 1 client.16674 shutdown 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.007+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 msgr2=0x55942bc5ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.007+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc5ee20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc264014ac0 tx=0x7fc264004c30 comp rx=0 tx=0).stop 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 shutdown_connections 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55942bcc0c20 0x55942bcc3010 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55942bc59c20 0x55942bc5ee20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55942bc51040 0x55942bc60c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc251ffb640 0 client.16674 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 --2- 172.21.15.17:0/1766700740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55942bc50720 0x55942bc60750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 >> 172.21.15.17:0/1766700740 conn(0x55942bc4ae50 msgr2=0x55942bc49630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 shutdown_connections 2024-02-14T15:08:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 1 -- 172.21.15.17:0/1766700740 wait complete. 2024-02-14T15:08:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc2737f4740 20 client.16674 trim_cache size 0 max 16384 2024-02-14T15:08:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:08:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:08:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:08:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc2580a2220 msgr2=0x7fc2580a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc2580a2220 0x7fc2580a46e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc264005c40 tx=0x7fc264016000 comp rx=0 tx=0).stop 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c100380 msgr2=0x7fc26c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c100380 0x7fc26c19d460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009230 tx=0x7fc25c004040 comp rx=0 tx=0).stop 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 shutdown_connections 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc2580a2220 0x7fc2580a46e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c1062c0 0x7fc26c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc26c100d40 0x7fc26c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 --2- 172.21.15.17:0/695006002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26c100380 0x7fc26c19d460 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 >> 172.21.15.17:0/695006002 conn(0x7fc26c076ef0 msgr2=0x7fc26c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 shutdown_connections 2024-02-14T15:08:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:09.008+0000 7fc270c23640 1 -- 172.21.15.17:0/695006002 wait complete. 2024-02-14T15:08:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:08 smithi017 ceph-mon[32549]: pgmap v144: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 370 op/s 2024-02-14T15:08:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:08 smithi050 ceph-mon[36512]: pgmap v144: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 370 op/s 2024-02-14T15:08:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:08 smithi076 ceph-mon[37057]: pgmap v144: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 370 op/s 2024-02-14T15:08:09.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 427058 12.46 MB/sec execute 71 sec latency 1298.740 ms 2024-02-14T15:08:10.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 427058 12.29 MB/sec execute 72 sec latency 2298.788 ms 2024-02-14T15:08:11.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:10 smithi017 ceph-mon[32549]: pgmap v145: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-02-14T15:08:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:10 smithi050 ceph-mon[36512]: pgmap v145: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-02-14T15:08:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:10 smithi076 ceph-mon[37057]: pgmap v145: 129 pgs: 129 active+clean; 245 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-02-14T15:08:11.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 427058 12.12 MB/sec execute 73 sec latency 3298.905 ms 2024-02-14T15:08:12.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 427058 11.95 MB/sec execute 74 sec latency 4299.020 ms 2024-02-14T15:08:13.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:12 smithi017 ceph-mon[32549]: pgmap v146: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-02-14T15:08:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:12 smithi050 ceph-mon[36512]: pgmap v146: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-02-14T15:08:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:12 smithi076 ceph-mon[37057]: pgmap v146: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-02-14T15:08:13.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 427058 11.79 MB/sec execute 75 sec latency 5299.083 ms 2024-02-14T15:08:14.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 429444 11.82 MB/sec execute 76 sec latency 5331.408 ms 2024-02-14T15:08:15.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:14 smithi017 ceph-mon[32549]: pgmap v147: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-02-14T15:08:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:14 smithi050 ceph-mon[36512]: pgmap v147: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-02-14T15:08:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:14 smithi076 ceph-mon[37057]: pgmap v147: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-02-14T15:08:15.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 431520 11.76 MB/sec execute 77 sec latency 19.450 ms 2024-02-14T15:08:16.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 433505 11.77 MB/sec execute 78 sec latency 19.403 ms 2024-02-14T15:08:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:16 smithi017 ceph-mon[32549]: pgmap v148: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-02-14T15:08:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:16 smithi050 ceph-mon[36512]: pgmap v148: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-02-14T15:08:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:16 smithi076 ceph-mon[37057]: pgmap v148: 129 pgs: 129 active+clean; 248 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-02-14T15:08:17.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 436372 11.84 MB/sec execute 79 sec latency 23.641 ms 2024-02-14T15:08:18.868 INFO:tasks.workunit.client.0.smithi017.stdout: 1 437468 11.74 MB/sec execute 80 sec latency 119.737 ms 2024-02-14T15:08:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:18 smithi017 ceph-mon[32549]: pgmap v149: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-02-14T15:08:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:18 smithi050 ceph-mon[36512]: pgmap v149: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-02-14T15:08:19.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:18 smithi076 ceph-mon[37057]: pgmap v149: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-02-14T15:08:19.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 440015 11.79 MB/sec execute 81 sec latency 19.991 ms 2024-02-14T15:08:20.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 441681 11.72 MB/sec execute 82 sec latency 18.018 ms 2024-02-14T15:08:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:20 smithi017 ceph-mon[32549]: pgmap v150: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 150 op/s 2024-02-14T15:08:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:20 smithi050 ceph-mon[36512]: pgmap v150: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 150 op/s 2024-02-14T15:08:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:20 smithi076 ceph-mon[37057]: pgmap v150: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 150 op/s 2024-02-14T15:08:21.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 444083 11.75 MB/sec execute 83 sec latency 14.932 ms 2024-02-14T15:08:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 -- 172.21.15.17:0/1739736158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698106510 msgr2=0x7fb69810e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 --2- 172.21.15.17:0/1739736158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698106510 0x7fb69810e0e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb690009930 tx=0x7fb69002bc30 comp rx=0 tx=0).stop 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 -- 172.21.15.17:0/1739736158 shutdown_connections 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 --2- 172.21.15.17:0/1739736158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698106510 0x7fb69810e0e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 --2- 172.21.15.17:0/1739736158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6980696f0 0x7fb698105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 --2- 172.21.15.17:0/1739736158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698068d40 0x7fb698069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 -- 172.21.15.17:0/1739736158 >> 172.21.15.17:0/1739736158 conn(0x7fb698077220 msgr2=0x7fb698077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 -- 172.21.15.17:0/1739736158 shutdown_connections 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.104+0000 7fb69c90c640 1 -- 172.21.15.17:0/1739736158 wait complete. 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 Processor -- start 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 -- start start 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb698068d40 0x7fb69819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698106510 0x7fb69819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698197600 0x7fb698197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698197ff0 con 0x7fb698197600 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698198160 con 0x7fb698106510 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69c90c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698198320 con 0x7fb698068d40 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698197600 0x7fb698197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698197600 0x7fb698197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56326/0 (socket says 172.21.15.17:56326) 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb69659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb698068d40 0x7fb69819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 -- 172.21.15.17:0/3482068984 learned_addr learned my addr 172.21.15.17:0/3482068984 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb695d9b640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698106510 0x7fb69819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 -- 172.21.15.17:0/3482068984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb698068d40 msgr2=0x7fb69819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb698068d40 0x7fb69819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 -- 172.21.15.17:0/3482068984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698106510 msgr2=0x7fb69819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698106510 0x7fb69819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.105+0000 7fb696d9d640 1 -- 172.21.15.17:0/3482068984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690009560 con 0x7fb698197600 2024-02-14T15:08:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.106+0000 7fb696d9d640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698197600 0x7fb698197ab0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb690005940 tx=0x7fb69002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.106+0000 7fb6877fe640 1 -- 172.21.15.17:0/3482068984 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690004350 con 0x7fb698197600 2024-02-14T15:08:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.106+0000 7fb6877fe640 1 -- 172.21.15.17:0/3482068984 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6900044b0 con 0x7fb698197600 2024-02-14T15:08:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.106+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6981a4310 con 0x7fb698197600 2024-02-14T15:08:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.106+0000 7fb6877fe640 1 -- 172.21.15.17:0/3482068984 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690003cc0 con 0x7fb698197600 2024-02-14T15:08:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.106+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6981a47c0 con 0x7fb698197600 2024-02-14T15:08:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.108+0000 7fb6877fe640 1 -- 172.21.15.17:0/3482068984 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb690005580 con 0x7fb698197600 2024-02-14T15:08:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 Processor -- start 2024-02-14T15:08:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 -- start start 2024-02-14T15:08:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb6877fe640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb6800a2220 0x7fb6800a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda14ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb695d9b640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb6800a2220 0x7fb6800a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb6877fe640 1 -- 172.21.15.17:0/3482068984 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb6900f03a0 con 0x7fb698197600 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda0b89e0 con 0x558eda14b040 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda0b5f10 con 0x558eda153c20 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda0b16f0 con 0x558eda14a720 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb695d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda14ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda14ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:34928/0 (socket says 172.21.15.17:34928) 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb695d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda1536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56338/0 (socket says 172.21.15.17:56338) 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.109+0000 7fb69659c640 1 -- 172.21.15.17:0/1582602769 learned_addr learned my addr 172.21.15.17:0/1582602769 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb696d9d640 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb69659c640 1 -- 172.21.15.17:0/1582602769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 msgr2=0x558eda156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb69659c640 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb69659c640 1 -- 172.21.15.17:0/1582602769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 msgr2=0x558eda1536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb69659c640 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda1536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb69659c640 1 -- 172.21.15.17:0/1582602769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ed9dc2920 con 0x558eda14a720 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb696d9d640 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb695d9b640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb6800a2220 0x7fb6800a46e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6981079c0 tx=0x7fb68c013830 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb69659c640 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda14ab00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb688007f90 tx=0x7fb68800f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f658e5a6b33aa61a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb684ff9640 1 -- 172.21.15.17:0/1582602769 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688004d20 con 0x558eda14a720 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb684ff9640 1 -- 172.21.15.17:0/1582602769 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb688004e80 con 0x558eda14a720 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.110+0000 7fb684ff9640 1 -- 172.21.15.17:0/1582602769 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688005680 con 0x558eda14a720 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1582602769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 msgr2=0x558eda14ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda14ab00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb688007f90 tx=0x7fb68800f850 comp rx=0 tx=0).stop 2024-02-14T15:08:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1582602769 shutdown_connections 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda1536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1582602769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda14ab00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1582602769 >> 172.21.15.17:0/1582602769 conn(0x558eda144e50 msgr2=0x558eda145230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1582602769 shutdown_connections 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1582602769 wait complete. 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- start start 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 Processor -- start 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda150920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda153c20 0x558eda150e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda04dfa0 con 0x558eda14b040 2024-02-14T15:08:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed9d85250 con 0x558eda14a720 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.111+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda037200 con 0x558eda153c20 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb69f5ba740 10 client.0 _ll_register_callbacks cb 0x558eda0a6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56352/0 (socket says 172.21.15.17:56352) 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb696d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda153c20 0x558eda150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 -- 172.21.15.17:0/1858299634 learned_addr learned my addr 172.21.15.17:0/1858299634 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb69659c640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 -- 172.21.15.17:0/1858299634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda153c20 msgr2=0x558eda150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda153c20 0x558eda150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 -- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 msgr2=0x558eda153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 -- 172.21.15.17:0/1858299634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb688007bc0 con 0x558eda14b040 2024-02-14T15:08:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb69659c640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb696d9d640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda153c20 0x558eda150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb695d9b640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda150920 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0043f0 tx=0x7fb68c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383f03c401251fa6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb67dffb640 1 -- 172.21.15.17:0/1858299634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c005220 con 0x558eda14b040 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb67dffb640 1 -- 172.21.15.17:0/1858299634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb68c026030 con 0x558eda14b040 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ed9e021b0 con 0x558eda14b040 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb67dffb640 1 -- 172.21.15.17:0/1858299634 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb68c005380 con 0x558eda14b040 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb69f5ba740 10 client.16686 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.112+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558eda0afd70 con 0x558eda14b040 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb67dffb640 1 -- 172.21.15.17:0/1858299634 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558ed9e021b0 con 0x558eda14b040 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb67dffb640 1 -- 172.21.15.17:0/1858299634 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb68c00f7b0 con 0x558eda14b040 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb69f5ba740 10 client.16686 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb69f5ba740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb69f5ba740 20 client.16686 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda165d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb69f5ba740 4 client.16686 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ed9dee550 con 0x558eda163910 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.113+0000 7fb69659c640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69659c640 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda165d30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5cd5adbe309dba server_cookie=245abecdf1df3a23 in_seq=0 out_seq=0 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb67dffb640 10 client.16686 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb67dffb640 1 -- 172.21.15.17:0/1858299634 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ed9dee550 con 0x558eda163910 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb67dffb640 10 client.16686 handle_command_reply: tid=0 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69f5ba740 1 client.16686 shutdown 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 msgr2=0x558eda150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda150920 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0043f0 tx=0x7fb68c004730 comp rx=0 tx=0).stop 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 shutdown_connections 2024-02-14T15:08:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda165d30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda150920 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb67dffb640 0 client.16686 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.114+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda153c20 0x558eda150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.115+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/1858299634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.115+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 >> 172.21.15.17:0/1858299634 conn(0x558eda144e50 msgr2=0x558eda146ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.115+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 shutdown_connections 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.115+0000 7fb69f5ba740 1 -- 172.21.15.17:0/1858299634 wait complete. 2024-02-14T15:08:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.115+0000 7fb69f5ba740 20 client.16686 trim_cache size 0 max 16384 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 Processor -- start 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 -- start start 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14b040 0x558eda153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda153c20 0x558eda14f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed9dee580 con 0x558eda153c20 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed9e021e0 con 0x558eda14a720 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.129+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda160470 con 0x558eda14b040 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda153c20 0x558eda14f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb695d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14b040 0x558eda153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda153c20 0x558eda14f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56366/0 (socket says 172.21.15.17:56366) 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 -- 172.21.15.17:0/369950626 learned_addr learned my addr 172.21.15.17:0/369950626 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb69659c640 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 -- 172.21.15.17:0/369950626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14b040 msgr2=0x558eda153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14b040 0x558eda153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 -- 172.21.15.17:0/369950626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 msgr2=0x558eda153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 -- 172.21.15.17:0/369950626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eda04c820 con 0x558eda153c20 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb696d9d640 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda153c20 0x558eda14f1f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b8fa0 tx=0x7fb69002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a555026a4b2cf3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb67ffff640 1 -- 172.21.15.17:0/369950626 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900bd240 con 0x558eda153c20 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb67ffff640 1 -- 172.21.15.17:0/369950626 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6900bcb40 con 0x558eda153c20 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.130+0000 7fb67ffff640 1 -- 172.21.15.17:0/369950626 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900c4e50 con 0x558eda153c20 2024-02-14T15:08:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 -- 172.21.15.17:0/369950626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda153c20 msgr2=0x558eda14f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda153c20 0x558eda14f1f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b8fa0 tx=0x7fb69002bc30 comp rx=0 tx=0).stop 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 -- 172.21.15.17:0/369950626 shutdown_connections 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda153c20 0x558eda14f1f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14b040 0x558eda153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/369950626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda14a720 0x558eda153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 -- 172.21.15.17:0/369950626 >> 172.21.15.17:0/369950626 conn(0x558eda144e50 msgr2=0x558eda16a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 -- 172.21.15.17:0/369950626 shutdown_connections 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 -- 172.21.15.17:0/369950626 wait complete. 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 -- start start 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.131+0000 7fb69f5ba740 1 Processor -- start 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda1599a0 con 0x558eda14b040 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda159b10 con 0x558eda153c20 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eda159cd0 con 0x558eda14a720 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 10 client.0 _ll_register_callbacks cb 0x7fb6880079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56378/0 (socket says 172.21.15.17:56378) 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 -- 172.21.15.17:0/2956032132 learned_addr learned my addr 172.21.15.17:0/2956032132 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69659c640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb696d9d640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 -- 172.21.15.17:0/2956032132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 msgr2=0x558eda15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 -- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 msgr2=0x558eda1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 -- 172.21.15.17:0/2956032132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6981a47c0 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb696d9d640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb695d9b640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda15afb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x558eda151ef0 tx=0x7fb68c00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d274248a445faaa server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb67dffb640 1 -- 172.21.15.17:0/2956032132 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c009b80 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb67dffb640 1 -- 172.21.15.17:0/2956032132 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb68c026070 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.132+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ed9db1600 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb67dffb640 1 -- 172.21.15.17:0/2956032132 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb68c00f8a0 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 10 client.16692 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558eda072720 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb67dffb640 1 -- 172.21.15.17:0/2956032132 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558ed9db1600 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb67dffb640 1 -- 172.21.15.17:0/2956032132 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb68c0053b0 con 0x558eda14b040 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 10 client.16692 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 20 client.16692 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda1bd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 4 client.16692 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:08:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ed9d5d520 con 0x558eda163910 2024-02-14T15:08:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.133+0000 7fb69659c640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda1bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.134+0000 7fb67dffb640 10 client.16692 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.134+0000 7fb69659c640 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda1bd130 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a222f9ce43f3530c server_cookie=697832547e817e5b in_seq=0 out_seq=0 2024-02-14T15:08:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb67dffb640 1 -- 172.21.15.17:0/2956032132 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ed9d5d520 con 0x558eda163910 2024-02-14T15:08:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb67dffb640 10 client.16692 handle_command_reply: tid=0 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 client.16692 shutdown 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 msgr2=0x558eda15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda15afb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x558eda151ef0 tx=0x7fb68c00ff60 comp rx=0 tx=0).stop 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 shutdown_connections 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558eda163910 0x558eda1bd130 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558eda14b040 0x558eda15afb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558eda153c20 0x558eda1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 --2- 172.21.15.17:0/2956032132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558eda14a720 0x558eda15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb67dffb640 0 client.16692 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 >> 172.21.15.17:0/2956032132 conn(0x558eda144e50 msgr2=0x558eda1501b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 shutdown_connections 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.142+0000 7fb69f5ba740 1 -- 172.21.15.17:0/2956032132 wait complete. 2024-02-14T15:08:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69f5ba740 20 client.16692 trim_cache size 0 max 16384 2024-02-14T15:08:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:08:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:08:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:08:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb6800a2220 msgr2=0x7fb6800a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb6800a2220 0x7fb6800a46e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6981079c0 tx=0x7fb68c013830 comp rx=0 tx=0).stop 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698197600 msgr2=0x7fb698197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698197600 0x7fb698197ab0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb690005940 tx=0x7fb69002c650 comp rx=0 tx=0).stop 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 shutdown_connections 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb6800a2220 0x7fb6800a46e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb698197600 0x7fb698197ab0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698106510 0x7fb69819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 --2- 172.21.15.17:0/3482068984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb698068d40 0x7fb69819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 >> 172.21.15.17:0/3482068984 conn(0x7fb698077220 msgr2=0x7fb6980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 shutdown_connections 2024-02-14T15:08:22.147 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:22.143+0000 7fb69c90c640 1 -- 172.21.15.17:0/3482068984 wait complete. 2024-02-14T15:08:22.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 446753 11.78 MB/sec execute 84 sec latency 35.358 ms 2024-02-14T15:08:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:22 smithi017 ceph-mon[32549]: pgmap v151: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 254 op/s 2024-02-14T15:08:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:22 smithi050 ceph-mon[36512]: pgmap v151: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 254 op/s 2024-02-14T15:08:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:22 smithi076 ceph-mon[37057]: pgmap v151: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 254 op/s 2024-02-14T15:08:23.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 448054 11.72 MB/sec execute 85 sec latency 114.488 ms 2024-02-14T15:08:24.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 450613 11.77 MB/sec execute 86 sec latency 30.135 ms 2024-02-14T15:08:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:24 smithi017 ceph-mon[32549]: pgmap v152: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:08:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:24 smithi050 ceph-mon[36512]: pgmap v152: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:08:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:24 smithi076 ceph-mon[37057]: pgmap v152: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:08:25.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 452271 11.71 MB/sec execute 87 sec latency 17.412 ms 2024-02-14T15:08:26.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 454646 11.73 MB/sec execute 88 sec latency 17.910 ms 2024-02-14T15:08:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:26 smithi017 ceph-mon[32549]: pgmap v153: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:08:27.035 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:26 smithi076 ceph-mon[37057]: pgmap v153: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:08:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:26 smithi050 ceph-mon[36512]: pgmap v153: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:08:27.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 457400 11.76 MB/sec execute 89 sec latency 17.621 ms 2024-02-14T15:08:28.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 458509 11.76 MB/sec execute 90 sec latency 109.185 ms 2024-02-14T15:08:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:28 smithi017 ceph-mon[32549]: pgmap v154: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 282 op/s 2024-02-14T15:08:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:28 smithi050 ceph-mon[36512]: pgmap v154: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 282 op/s 2024-02-14T15:08:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:28 smithi076 ceph-mon[37057]: pgmap v154: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 282 op/s 2024-02-14T15:08:29.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 459987 11.85 MB/sec execute 91 sec latency 199.830 ms 2024-02-14T15:08:30.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 462495 11.87 MB/sec execute 92 sec latency 15.513 ms 2024-02-14T15:08:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:30 smithi017 ceph-mon[32549]: pgmap v155: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 213 op/s 2024-02-14T15:08:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:30 smithi050 ceph-mon[36512]: pgmap v155: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 213 op/s 2024-02-14T15:08:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:30 smithi076 ceph-mon[37057]: pgmap v155: 129 pgs: 129 active+clean; 267 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 213 op/s 2024-02-14T15:08:31.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 465277 11.90 MB/sec execute 93 sec latency 16.691 ms 2024-02-14T15:08:32.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 467222 11.88 MB/sec execute 94 sec latency 16.214 ms 2024-02-14T15:08:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:32 smithi017 ceph-mon[32549]: pgmap v156: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 318 op/s 2024-02-14T15:08:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:32 smithi050 ceph-mon[36512]: pgmap v156: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 318 op/s 2024-02-14T15:08:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:32 smithi076 ceph-mon[37057]: pgmap v156: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 318 op/s 2024-02-14T15:08:33.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 469528 11.90 MB/sec execute 95 sec latency 62.648 ms 2024-02-14T15:08:34.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 471426 11.85 MB/sec execute 96 sec latency 60.601 ms 2024-02-14T15:08:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:34 smithi017 ceph-mon[32549]: pgmap v157: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:08:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:34 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:08:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:34 smithi050 ceph-mon[36512]: pgmap v157: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:08:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:34 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.232+0000 7f08e71d0640 1 -- 172.21.15.17:0/88753607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0106240 msgr2=0x7f08e0106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.232+0000 7f08e71d0640 1 --2- 172.21.15.17:0/88753607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0106240 0x7f08e0106620 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009900 tx=0x7f08d402bc80 comp rx=0 tx=0).stop 2024-02-14T15:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- 172.21.15.17:0/88753607 shutdown_connections 2024-02-14T15:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 --2- 172.21.15.17:0/88753607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08e0100d40 0x7f08e010cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 --2- 172.21.15.17:0/88753607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e0100380 0x7f08e0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 --2- 172.21.15.17:0/88753607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0106240 0x7f08e0106620 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- 172.21.15.17:0/88753607 >> 172.21.15.17:0/88753607 conn(0x7f08e0076f80 msgr2=0x7f08e0077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- 172.21.15.17:0/88753607 shutdown_connections 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- 172.21.15.17:0/88753607 wait complete. 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 Processor -- start 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- start start 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e0100380 0x7f08e019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0100d40 0x7f08e019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08e0106240 0x7f08e0196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0196e20 con 0x7f08e0100d40 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0196f90 con 0x7f08e0106240 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.233+0000 7f08e71d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0197150 con 0x7f08e0100380 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0100d40 0x7f08e019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08e4f6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e0100380 0x7f08e019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08e4f6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e0100380 0x7f08e019c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37180/0 (socket says 172.21.15.17:37180) 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08e4f6c640 1 -- 172.21.15.17:0/2747865159 learned_addr learned my addr 172.21.15.17:0/2747865159 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08e576d640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08e0106240 0x7f08e0196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08dffff640 1 -- 172.21.15.17:0/2747865159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e0100380 msgr2=0x7f08e019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08dffff640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e0100380 0x7f08e019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08dffff640 1 -- 172.21.15.17:0/2747865159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08e0106240 msgr2=0x7f08e0196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08dffff640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08e0106240 0x7f08e0196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08dffff640 1 -- 172.21.15.17:0/2747865159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d4009560 con 0x7f08e0100d40 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08dffff640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0100d40 0x7f08e019cb10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00e9f0 tx=0x7f08cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08ddffb640 1 -- 172.21.15.17:0/2747865159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00ccc0 con 0x7f08e0100d40 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08ddffb640 1 -- 172.21.15.17:0/2747865159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08cc00ce20 con 0x7f08e0100d40 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e0197430 con 0x7f08e0100d40 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.234+0000 7f08ddffb640 1 -- 172.21.15.17:0/2747865159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc004a00 con 0x7f08e0100d40 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.235+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e01a3550 con 0x7f08e0100d40 2024-02-14T15:08:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.236+0000 7f08ddffb640 1 -- 172.21.15.17:0/2747865159 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f08cc0050b0 con 0x7f08e0100d40 2024-02-14T15:08:35.241 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:34 smithi076 ceph-mon[37057]: pgmap v157: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:08:35.241 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:34 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:08:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08ddffb640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08d00a22a0 0x7f08d00a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08d00a22a0 0x7f08d00a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08ddffb640 1 -- 172.21.15.17:0/2747865159 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f08cc014070 con 0x7f08e0100d40 2024-02-14T15:08:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 Processor -- start 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 -- start start 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8120 0x5633532a8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8ad0 0x5633532b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633532169e0 con 0x5633532a8120 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563353213f10 con 0x5633532a8ad0 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.237+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56335320f6f0 con 0x5633532b3e30 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e4f6c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8120 0x5633532a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08d00a22a0 0x7f08d00a4760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000c00 tx=0x7f08d40058d0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37194/0 (socket says 172.21.15.17:37194) 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 -- 172.21.15.17:0/2427605273 learned_addr learned my addr 172.21.15.17:0/2427605273 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08dffff640 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8ad0 0x5633532b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 -- 172.21.15.17:0/2427605273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8ad0 msgr2=0x5633532b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8ad0 0x5633532b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 -- 172.21.15.17:0/2427605273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8120 msgr2=0x5633532a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8120 0x5633532a8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 -- 172.21.15.17:0/2427605273 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563352f20920 con 0x5633532b3e30 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8120 0x5633532a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:08:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08e576d640 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532b6220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009f40 tx=0x7f08d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc9b1a602358742c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08caffd640 1 -- 172.21.15.17:0/2427605273 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8004d20 con 0x5633532b3e30 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08caffd640 1 -- 172.21.15.17:0/2427605273 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08d8004e80 con 0x5633532b3e30 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.238+0000 7f08caffd640 1 -- 172.21.15.17:0/2427605273 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8005280 con 0x5633532b3e30 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 -- 172.21.15.17:0/2427605273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 msgr2=0x5633532b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532b6220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009f40 tx=0x7f08d80118d0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 -- 172.21.15.17:0/2427605273 shutdown_connections 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532b6220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8ad0 0x5633532b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/2427605273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8120 0x5633532a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 -- 172.21.15.17:0/2427605273 >> 172.21.15.17:0/2427605273 conn(0x5633532a2e50 msgr2=0x5633532a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 -- 172.21.15.17:0/2427605273 shutdown_connections 2024-02-14T15:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 -- 172.21.15.17:0/2427605273 wait complete. 2024-02-14T15:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 -- start start 2024-02-14T15:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.239+0000 7f08e9d9f740 1 Processor -- start 2024-02-14T15:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633531abfa0 con 0x5633532b3e30 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563352ee3250 con 0x5633532a8120 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563353195200 con 0x5633532a8ad0 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e9d9f740 10 client.0 _ll_register_callbacks cb 0x563353204b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42786/0 (socket says 172.21.15.17:42786) 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e4f6c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 -- 172.21.15.17:0/4201156520 learned_addr learned my addr 172.21.15.17:0/4201156520 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 -- 172.21.15.17:0/4201156520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 msgr2=0x5633532b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 -- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 msgr2=0x5633532b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e576d640 1 -- 172.21.15.17:0/4201156520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d8009bf0 con 0x5633532b3e30 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08dffff640 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.240+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e576d640 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b1070 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f08d8004660 tx=0x7f08d8004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9655cc63dc66a6a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08bffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08bffff640 1 -- 172.21.15.17:0/4201156520 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8004410 con 0x5633532b3e30 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563352f601b0 con 0x5633532b3e30 2024-02-14T15:08:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08bffff640 1 -- 172.21.15.17:0/4201156520 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f08d8005000 con 0x5633532b3e30 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08bffff640 1 -- 172.21.15.17:0/4201156520 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d80051c0 con 0x5633532b3e30 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 10 client.16704 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56335320dd70 con 0x5633532b3e30 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08bffff640 1 -- 172.21.15.17:0/4201156520 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563352f601b0 con 0x5633532b3e30 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08bffff640 1 -- 172.21.15.17:0/4201156520 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f08d800f8e0 con 0x5633532b3e30 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 10 client.16704 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 20 client.16704 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633532c3ac0 0x5633532c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 4 client.16704 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563352f4c550 con 0x5633532c3ac0 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.241+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633532c3ac0 0x5633532c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.242+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633532c3ac0 0x5633532c5eb0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c872ae5fa064a434 server_cookie=54979dc2898f7f17 in_seq=0 out_seq=0 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.242+0000 7f08bffff640 10 client.16704 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.242+0000 7f08bffff640 1 -- 172.21.15.17:0/4201156520 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563352f4c550 con 0x5633532c3ac0 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.242+0000 7f08bffff640 10 client.16704 handle_command_reply: tid=0 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.242+0000 7f08e9d9f740 1 client.16704 shutdown 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 msgr2=0x5633532b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b1070 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f08d8004660 tx=0x7f08d8004c70 comp rx=0 tx=0).stop 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 shutdown_connections 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633532c3ac0 0x5633532c5eb0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b1070 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08bffff640 0 client.16704 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/4201156520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 >> 172.21.15.17:0/4201156520 conn(0x5633532a2e50 msgr2=0x5633532a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 shutdown_connections 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 1 -- 172.21.15.17:0/4201156520 wait complete. 2024-02-14T15:08:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.243+0000 7f08e9d9f740 20 client.16704 trim_cache size 0 max 16384 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 Processor -- start 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 -- start start 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633532b67f0 con 0x5633532b3e30 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633532a7a90 con 0x5633532a8120 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633532a37c0 con 0x5633532a8ad0 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e576d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e4f6c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e576d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42800/0 (socket says 172.21.15.17:42800) 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e4f6c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532a1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:40404/0 (socket says 172.21.15.17:40404) 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.257+0000 7f08e576d640 1 -- 172.21.15.17:0/1855304037 learned_addr learned my addr 172.21.15.17:0/1855304037 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08e576d640 1 -- 172.21.15.17:0/1855304037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 msgr2=0x5633532b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08e576d640 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08e576d640 1 -- 172.21.15.17:0/1855304037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 msgr2=0x5633532a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08e576d640 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08e576d640 1 -- 172.21.15.17:0/1855304037 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633531aa820 con 0x5633532b3e30 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08dffff640 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08e576d640 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b3780 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08d8004020 tx=0x7f08d800ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806fa58941fd43e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08ca7fc640 1 -- 172.21.15.17:0/1855304037 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8009e10 con 0x5633532b3e30 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08ca7fc640 1 -- 172.21.15.17:0/1855304037 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08d8004ce0 con 0x5633532b3e30 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.258+0000 7f08ca7fc640 1 -- 172.21.15.17:0/1855304037 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80047f0 con 0x5633532b3e30 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- 172.21.15.17:0/1855304037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 msgr2=0x5633532b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b3780 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08d8004020 tx=0x7f08d800ffb0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- 172.21.15.17:0/1855304037 shutdown_connections 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532b3e30 0x5633532b3780 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532a8ad0 0x5633532b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/1855304037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x5633532a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- 172.21.15.17:0/1855304037 >> 172.21.15.17:0/1855304037 conn(0x5633532a2e50 msgr2=0x5633532a1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- 172.21.15.17:0/1855304037 shutdown_connections 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- 172.21.15.17:0/1855304037 wait complete. 2024-02-14T15:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- start start 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 Processor -- start 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x563353313350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8ad0 0x563353313890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633532bac80 con 0x5633532a8ad0 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633532badf0 con 0x5633532a8120 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633532bafb0 con 0x5633532b3e30 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.259+0000 7f08e9d9f740 10 client.0 _ll_register_callbacks cb 0x7f08d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8ad0 0x563353313890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8ad0 0x563353313890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42804/0 (socket says 172.21.15.17:42804) 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 -- 172.21.15.17:0/816720002 learned_addr learned my addr 172.21.15.17:0/816720002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08e576d640 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x563353313350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 -- 172.21.15.17:0/816720002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 msgr2=0x5633532ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532ba710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 -- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 msgr2=0x563353313350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x563353313350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 -- 172.21.15.17:0/816720002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc0cce70 con 0x5633532a8ad0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08dffff640 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8ad0 0x563353313890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0cb7c0 tx=0x7f08cc09a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835ae7af7cd81983 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08bffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08bffff640 1 -- 172.21.15.17:0/816720002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0b3460 con 0x5633532a8ad0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08bffff640 1 -- 172.21.15.17:0/816720002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f08cc0a3150 con 0x5633532a8ad0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563352f0f600 con 0x5633532a8ad0 2024-02-14T15:08:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08bffff640 1 -- 172.21.15.17:0/816720002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08cc0a2cc0 con 0x5633532a8ad0 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08e9d9f740 10 client.16710 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56335320dd70 con 0x5633532a8ad0 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.260+0000 7f08bffff640 1 -- 172.21.15.17:0/816720002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563352f0f600 con 0x5633532a8ad0 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08bffff640 1 -- 172.21.15.17:0/816720002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f08cc004b60 con 0x5633532a8ad0 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e9d9f740 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e9d9f740 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e9d9f740 20 client.16710 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56335331b2f0 0x56335331d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e9d9f740 4 client.16710 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563352ebb520 con 0x56335331b2f0 2024-02-14T15:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56335331b2f0 0x56335331d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08bffff640 10 client.16710 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.261+0000 7f08e4f6c640 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56335331b2f0 0x56335331d710 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258a5a68a6459b0 server_cookie=c9e75adcc06278d4 in_seq=0 out_seq=0 2024-02-14T15:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08bffff640 1 -- 172.21.15.17:0/816720002 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563352ebb520 con 0x56335331b2f0 2024-02-14T15:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08bffff640 10 client.16710 handle_command_reply: tid=0 2024-02-14T15:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 client.16710 shutdown 2024-02-14T15:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8ad0 msgr2=0x563353313890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8ad0 0x563353313890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0cb7c0 tx=0x7f08cc09a3d0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 shutdown_connections 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56335331b2f0 0x56335331d710 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633532a8ad0 0x563353313890 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08bffff640 0 client.16710 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633532b3e30 0x5633532ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 --2- 172.21.15.17:0/816720002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633532a8120 0x563353313350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 >> 172.21.15.17:0/816720002 conn(0x5633532a2e50 msgr2=0x5633532ab890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 shutdown_connections 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 1 -- 172.21.15.17:0/816720002 wait complete. 2024-02-14T15:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.270+0000 7f08e9d9f740 20 client.16710 trim_cache size 0 max 16384 2024-02-14T15:08:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:08:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:08:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:08:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:08:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08d00a22a0 msgr2=0x7f08d00a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08d00a22a0 0x7f08d00a4760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000c00 tx=0x7f08d40058d0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0100d40 msgr2=0x7f08e019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0100d40 0x7f08e019cb10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00e9f0 tx=0x7f08cc00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 shutdown_connections 2024-02-14T15:08:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f08d00a22a0 0x7f08d00a4760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08e0100d40 0x7f08e019cb10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08e0106240 0x7f08e0196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 --2- 172.21.15.17:0/2747865159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08e0100380 0x7f08e019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 >> 172.21.15.17:0/2747865159 conn(0x7f08e0076f80 msgr2=0x7f08e00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 shutdown_connections 2024-02-14T15:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:35.271+0000 7f08e71d0640 1 -- 172.21.15.17:0/2747865159 wait complete. 2024-02-14T15:08:35.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 473649 11.87 MB/sec execute 97 sec latency 14.398 ms 2024-02-14T15:08:36.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:35 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:08:36.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:35 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:08:36.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:35 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:08:36.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:35 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:08:36.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:35 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:08:36.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:35 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:08:36.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:35 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:08:36.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:35 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:08:36.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:35 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:08:36.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 476454 11.92 MB/sec execute 98 sec latency 12.411 ms 2024-02-14T15:08:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:36 smithi050 ceph-mon[36512]: pgmap v158: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:08:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:36 smithi076 ceph-mon[37057]: pgmap v158: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:08:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:36 smithi017 ceph-mon[32549]: pgmap v158: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:08:37.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 479003 11.92 MB/sec execute 99 sec latency 15.278 ms 2024-02-14T15:08:38.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 481093 11.93 MB/sec execute 100 sec latency 25.419 ms 2024-02-14T15:08:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:38 smithi050 ceph-mon[36512]: pgmap v159: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 313 op/s 2024-02-14T15:08:39.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:38 smithi076 ceph-mon[37057]: pgmap v159: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 313 op/s 2024-02-14T15:08:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:38 smithi017 ceph-mon[32549]: pgmap v159: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 313 op/s 2024-02-14T15:08:39.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 483361 11.94 MB/sec execute 101 sec latency 83.557 ms 2024-02-14T15:08:40.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 484828 11.88 MB/sec execute 102 sec latency 193.496 ms 2024-02-14T15:08:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:40 smithi050 ceph-mon[36512]: pgmap v160: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-02-14T15:08:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:40 smithi076 ceph-mon[37057]: pgmap v160: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-02-14T15:08:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:40 smithi017 ceph-mon[32549]: pgmap v160: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-02-14T15:08:41.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 487313 11.90 MB/sec execute 103 sec latency 12.803 ms 2024-02-14T15:08:42.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 489774 11.91 MB/sec execute 104 sec latency 16.162 ms 2024-02-14T15:08:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:42 smithi050 ceph-mon[36512]: pgmap v161: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 299 op/s 2024-02-14T15:08:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:42 smithi076 ceph-mon[37057]: pgmap v161: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 299 op/s 2024-02-14T15:08:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:42 smithi017 ceph-mon[32549]: pgmap v161: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 299 op/s 2024-02-14T15:08:43.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 491796 11.91 MB/sec execute 105 sec latency 50.893 ms 2024-02-14T15:08:44.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 494222 11.93 MB/sec execute 106 sec latency 28.752 ms 2024-02-14T15:08:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:44 smithi050 ceph-mon[36512]: pgmap v162: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-02-14T15:08:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:44 smithi076 ceph-mon[37057]: pgmap v162: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-02-14T15:08:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:44 smithi017 ceph-mon[32549]: pgmap v162: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-02-14T15:08:45.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 496187 11.88 MB/sec execute 107 sec latency 94.949 ms 2024-02-14T15:08:46.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 498283 11.89 MB/sec execute 108 sec latency 20.746 ms 2024-02-14T15:08:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:46 smithi050 ceph-mon[36512]: pgmap v163: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-02-14T15:08:47.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:46 smithi076 ceph-mon[37057]: pgmap v163: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-02-14T15:08:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:46 smithi017 ceph-mon[32549]: pgmap v163: 129 pgs: 129 active+clean; 288 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-02-14T15:08:47.869 INFO:tasks.workunit.client.0.smithi017.stdout: 1 501187 11.95 MB/sec execute 109 sec latency 12.682 ms 2024-02-14T15:08:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 -- 172.21.15.17:0/3363457208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 msgr2=0x7eff5810cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/3363457208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5810cfe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7eff50009cd0 tx=0x7eff5002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 -- 172.21.15.17:0/3363457208 shutdown_connections 2024-02-14T15:08:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/3363457208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5810cfe0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/3363457208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff58100cf0 0x7eff58104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/3363457208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff58100340 0x7eff58100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 -- 172.21.15.17:0/3363457208 >> 172.21.15.17:0/3363457208 conn(0x7eff58077220 msgr2=0x7eff58077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 -- 172.21.15.17:0/3363457208 shutdown_connections 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.365+0000 7eff5f3ac640 1 -- 172.21.15.17:0/3363457208 wait complete. 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 Processor -- start 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 -- start start 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff58100340 0x7eff5819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff58197690 0x7eff58197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff58198160 con 0x7eff58105410 2024-02-14T15:08:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff581982d0 con 0x7eff58197690 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5f3ac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff58198490 con 0x7eff58100340 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5c947640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5c947640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54596/0 (socket says 172.21.15.17:54596) 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5c947640 1 -- 172.21.15.17:0/682095798 learned_addr learned my addr 172.21.15.17:0/682095798 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.366+0000 7eff5d148640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff58100340 0x7eff5819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5d949640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff58197690 0x7eff58197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5c947640 1 -- 172.21.15.17:0/682095798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff58100340 msgr2=0x7eff5819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5c947640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff58100340 0x7eff5819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5c947640 1 -- 172.21.15.17:0/682095798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff58197690 msgr2=0x7eff58197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5c947640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff58197690 0x7eff58197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5c947640 1 -- 172.21.15.17:0/682095798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff54009490 con 0x7eff58105410 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5c947640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5819dac0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9c0 tx=0x7eff4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff4e7fc640 1 -- 172.21.15.17:0/682095798 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48009700 con 0x7eff58105410 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff4e7fc640 1 -- 172.21.15.17:0/682095798 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff4800cd80 con 0x7eff58105410 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff50009900 con 0x7eff58105410 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff4e7fc640 1 -- 172.21.15.17:0/682095798 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48010430 con 0x7eff58105410 2024-02-14T15:08:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.367+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff581a4730 con 0x7eff58105410 2024-02-14T15:08:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.369+0000 7eff4e7fc640 1 -- 172.21.15.17:0/682095798 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7eff48005000 con 0x7eff58105410 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 Processor -- start 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 -- start start 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc697b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc6a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc6059e0 con 0x5591bc6a0c20 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc602f10 con 0x5591bc698040 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff61f78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc5fe6f0 con 0x5591bc697720 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.370+0000 7eff4e7fc640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7eff440a2220 0x7eff440a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d949640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc697b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5c947640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc6a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d949640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc697b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:34030/0 (socket says 172.21.15.17:34030) 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d949640 1 -- 172.21.15.17:0/4166459933 learned_addr learned my addr 172.21.15.17:0/4166459933 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:37516/0 (socket says 172.21.15.17:37516) 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d949640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7eff440a2220 0x7eff440a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff4e7fc640 1 -- 172.21.15.17:0/682095798 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7eff48014070 con 0x7eff58105410 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 -- 172.21.15.17:0/4166459933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 msgr2=0x5591bc697b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc697b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 -- 172.21.15.17:0/4166459933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 msgr2=0x5591bc6a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc6a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 -- 172.21.15.17:0/4166459933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591bc30f920 con 0x5591bc698040 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5c947640 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc6a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d949640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7eff440a2220 0x7eff440a46e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff50005fa0 tx=0x7eff500095c0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff5d148640 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a06e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7eff54009db0 tx=0x7eff54011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ffddfbb290eae server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff437fe640 1 -- 172.21.15.17:0/4166459933 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54019410 con 0x5591bc698040 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff437fe640 1 -- 172.21.15.17:0/4166459933 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff54019a50 con 0x5591bc698040 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.371+0000 7eff437fe640 1 -- 172.21.15.17:0/4166459933 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54018ac0 con 0x5591bc698040 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 -- 172.21.15.17:0/4166459933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 msgr2=0x5591bc6a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a06e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7eff54009db0 tx=0x7eff54011810 comp rx=0 tx=0).stop 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 -- 172.21.15.17:0/4166459933 shutdown_connections 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc6a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a06e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 --2- 172.21.15.17:0/4166459933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc697b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 -- 172.21.15.17:0/4166459933 >> 172.21.15.17:0/4166459933 conn(0x5591bc691e50 msgr2=0x5591bc692230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 -- 172.21.15.17:0/4166459933 shutdown_connections 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 -- 172.21.15.17:0/4166459933 wait complete. 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.372+0000 7eff61f78740 1 -- start start 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 1 Processor -- start 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc697720 0x5591bc6a0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc698040 0x5591bc69d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc69de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc59afa0 con 0x5591bc6a0c20 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc2d2250 con 0x5591bc697720 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc584200 con 0x5591bc698040 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff61f78740 10 client.0 _ll_register_callbacks cb 0x5591bc5f3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc69de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc69de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54608/0 (socket says 172.21.15.17:54608) 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 -- 172.21.15.17:0/815796253 learned_addr learned my addr 172.21.15.17:0/815796253 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d148640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc697720 0x5591bc6a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5c947640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc698040 0x5591bc69d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 -- 172.21.15.17:0/815796253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc698040 msgr2=0x5591bc69d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc698040 0x5591bc69d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 -- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc697720 msgr2=0x5591bc6a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc697720 0x5591bc6a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d949640 1 -- 172.21.15.17:0/815796253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff540099e0 con 0x5591bc6a0c20 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.373+0000 7eff5d148640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc697720 0x5591bc6a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff5d949640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc69de60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7eff500348c0 tx=0x7eff5002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e935ce3ff4a08f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff40ff9640 1 -- 172.21.15.17:0/815796253 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50003d80 con 0x5591bc6a0c20 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff40ff9640 1 -- 172.21.15.17:0/815796253 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7eff50049040 con 0x5591bc6a0c20 2024-02-14T15:08:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591bc2fe600 con 0x5591bc6a0c20 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff40ff9640 1 -- 172.21.15.17:0/815796253 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff50032d70 con 0x5591bc6a0c20 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 10 client.16722 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5591bc5fcd70 con 0x5591bc6a0c20 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff40ff9640 1 -- 172.21.15.17:0/815796253 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5591bc2fe600 con 0x5591bc6a0c20 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff40ff9640 1 -- 172.21.15.17:0/815796253 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7eff5003b070 con 0x5591bc6a0c20 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 10 client.16722 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 20 client.16722 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc6b0960 0x5591bc6b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 4 client.16722 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.374+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591bc34f1b0 con 0x5591bc6b0960 2024-02-14T15:08:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.375+0000 7eff5d148640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc6b0960 0x5591bc6b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.375+0000 7eff40ff9640 10 client.16722 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.375+0000 7eff5d148640 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc6b0960 0x5591bc6b2d50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fab5e2e38e14ab server_cookie=f5ed48b182cda634 in_seq=0 out_seq=0 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff40ff9640 1 -- 172.21.15.17:0/815796253 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5591bc34f1b0 con 0x5591bc6b0960 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff40ff9640 10 client.16722 handle_command_reply: tid=0 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 client.16722 shutdown 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 msgr2=0x5591bc69de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc69de60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7eff500348c0 tx=0x7eff5002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 shutdown_connections 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc6b0960 0x5591bc6b2d50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc6a0c20 0x5591bc69de60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc698040 0x5591bc69d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 --2- 172.21.15.17:0/815796253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc697720 0x5591bc6a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 >> 172.21.15.17:0/815796253 conn(0x5591bc691e50 msgr2=0x5591bc693ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff40ff9640 0 client.16722 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 shutdown_connections 2024-02-14T15:08:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 1 -- 172.21.15.17:0/815796253 wait complete. 2024-02-14T15:08:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.376+0000 7eff61f78740 20 client.16722 trim_cache size 0 max 16384 2024-02-14T15:08:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 Processor -- start 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 -- start start 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc697720 0x5591bc6a0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc6a0c20 0x5591bc69d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc696a10 con 0x5591bc697720 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc6ad470 con 0x5591bc698040 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff61f78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc68d080 con 0x5591bc6a0c20 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc697720 0x5591bc6a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc697720 0x5591bc6a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54610/0 (socket says 172.21.15.17:54610) 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 -- 172.21.15.17:0/1703207821 learned_addr learned my addr 172.21.15.17:0/1703207821 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d949640 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc6a0c20 0x5591bc69d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5c947640 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 -- 172.21.15.17:0/1703207821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc6a0c20 msgr2=0x5591bc69d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc6a0c20 0x5591bc69d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 -- 172.21.15.17:0/1703207821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 msgr2=0x5591bc6a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.391+0000 7eff5d148640 1 -- 172.21.15.17:0/1703207821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591bc599820 con 0x5591bc697720 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff5d148640 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc697720 0x5591bc6a0030 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7eff54009fd0 tx=0x7eff54009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2992bb561cc660 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff42ffd640 1 -- 172.21.15.17:0/1703207821 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54029ea0 con 0x5591bc697720 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff42ffd640 1 -- 172.21.15.17:0/1703207821 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff54005d50 con 0x5591bc697720 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff42ffd640 1 -- 172.21.15.17:0/1703207821 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54029430 con 0x5591bc697720 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 -- 172.21.15.17:0/1703207821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc697720 msgr2=0x5591bc6a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc697720 0x5591bc6a0030 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7eff54009fd0 tx=0x7eff54009f60 comp rx=0 tx=0).stop 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 -- 172.21.15.17:0/1703207821 shutdown_connections 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc6a0c20 0x5591bc69d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc698040 0x5591bc6a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 --2- 172.21.15.17:0/1703207821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc697720 0x5591bc6a0030 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 -- 172.21.15.17:0/1703207821 >> 172.21.15.17:0/1703207821 conn(0x5591bc691e50 msgr2=0x5591bc69d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 -- 172.21.15.17:0/1703207821 shutdown_connections 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.392+0000 7eff61f78740 1 -- 172.21.15.17:0/1703207821 wait complete. 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 -- start start 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 Processor -- start 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc6a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc698040 0x5591bc6a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc6a0c20 0x5591bc6a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc6a6790 con 0x5591bc698040 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc6a6900 con 0x5591bc6a0c20 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591bc6a6ac0 con 0x5591bc697720 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff61f78740 10 client.0 _ll_register_callbacks cb 0x7eff54009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff5c947640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc698040 0x5591bc6a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff5d148640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc6a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff5c947640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc698040 0x5591bc6a7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54622/0 (socket says 172.21.15.17:54622) 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff5c947640 1 -- 172.21.15.17:0/1708864968 learned_addr learned my addr 172.21.15.17:0/1708864968 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:08:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.393+0000 7eff5d949640 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc6a0c20 0x5591bc6a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff5c947640 1 -- 172.21.15.17:0/1708864968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 msgr2=0x5591bc6a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff5c947640 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc6a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff5c947640 1 -- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc6a0c20 msgr2=0x5591bc6a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff5c947640 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc6a0c20 0x5591bc6a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff5c947640 1 -- 172.21.15.17:0/1708864968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591bc30f920 con 0x5591bc698040 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff5d148640 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc6a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff5c947640 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc698040 0x5591bc6a7fc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c6a0 tx=0x7eff4806a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466f2258bab485ed server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff40ff9640 1 -- 172.21.15.17:0/1708864968 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48099c00 con 0x5591bc698040 2024-02-14T15:08:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff40ff9640 1 -- 172.21.15.17:0/1708864968 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7eff48066430 con 0x5591bc698040 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff54019a50 con 0x5591bc698040 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff40ff9640 1 -- 172.21.15.17:0/1708864968 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff4809b5d0 con 0x5591bc698040 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff61f78740 10 client.16728 fetch_fsmap learned FSMap version 29 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5591bc5bf720 con 0x5591bc698040 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.394+0000 7eff40ff9640 1 -- 172.21.15.17:0/1708864968 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff54019a50 con 0x5591bc698040 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff40ff9640 1 -- 172.21.15.17:0/1708864968 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7eff4801f060 con 0x5591bc698040 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff61f78740 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff61f78740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff61f78740 20 client.16728 populate_metadata read hostname 'smithi017' 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff61f78740 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc708230 0x5591bc70a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff61f78740 4 client.16728 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:08:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5591bc33b550 con 0x5591bc708230 2024-02-14T15:08:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff5d148640 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc708230 0x5591bc70a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:08:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff5d148640 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc708230 0x5591bc70a650 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb1984260430cd server_cookie=b7a172f9b283f3d1 in_seq=0 out_seq=0 2024-02-14T15:08:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.395+0000 7eff40ff9640 10 client.16728 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff40ff9640 1 -- 172.21.15.17:0/1708864968 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5591bc33b550 con 0x5591bc708230 2024-02-14T15:08:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff40ff9640 10 client.16728 handle_command_reply: tid=0 2024-02-14T15:08:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff61f78740 1 client.16728 shutdown 2024-02-14T15:08:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc698040 msgr2=0x5591bc6a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff61f78740 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc698040 0x5591bc6a7fc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c6a0 tx=0x7eff4806a070 comp rx=0 tx=0).stop 2024-02-14T15:08:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 shutdown_connections 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff61f78740 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5591bc708230 0x5591bc70a650 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff61f78740 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5591bc698040 0x5591bc6a7fc0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff40ff9640 0 client.16728 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.404+0000 7eff61f78740 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591bc6a0c20 0x5591bc6a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff61f78740 1 --2- 172.21.15.17:0/1708864968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591bc697720 0x5591bc6a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 >> 172.21.15.17:0/1708864968 conn(0x5591bc691e50 msgr2=0x5591bc69d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 shutdown_connections 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff61f78740 1 -- 172.21.15.17:0/1708864968 wait complete. 2024-02-14T15:08:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff61f78740 20 client.16728 trim_cache size 0 max 16384 2024-02-14T15:08:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:08:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:08:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:08:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7eff440a2220 msgr2=0x7eff440a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7eff440a2220 0x7eff440a46e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff50005fa0 tx=0x7eff500095c0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 msgr2=0x7eff5819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5819dac0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9c0 tx=0x7eff4800ee80 comp rx=0 tx=0).stop 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 shutdown_connections 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7eff440a2220 0x7eff440a46e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff58105410 0x7eff5819dac0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff58197690 0x7eff58197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 --2- 172.21.15.17:0/682095798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff58100340 0x7eff5819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:08:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.405+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 >> 172.21.15.17:0/682095798 conn(0x7eff58077220 msgr2=0x7eff58068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:08:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.406+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 shutdown_connections 2024-02-14T15:08:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:08:48.406+0000 7eff5f3ac640 1 -- 172.21.15.17:0/682095798 wait complete. 2024-02-14T15:08:48.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 503383 11.90 MB/sec execute 110 sec latency 17.417 ms 2024-02-14T15:08:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:48 smithi050 ceph-mon[36512]: pgmap v164: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-02-14T15:08:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:48 smithi076 ceph-mon[37057]: pgmap v164: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-02-14T15:08:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:48 smithi017 ceph-mon[32549]: pgmap v164: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-02-14T15:08:49.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 505213 11.91 MB/sec execute 111 sec latency 51.934 ms 2024-02-14T15:08:50.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 507469 11.91 MB/sec execute 112 sec latency 63.673 ms 2024-02-14T15:08:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:50 smithi050 ceph-mon[36512]: pgmap v165: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-02-14T15:08:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:50 smithi076 ceph-mon[37057]: pgmap v165: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-02-14T15:08:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:50 smithi017 ceph-mon[32549]: pgmap v165: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-02-14T15:08:51.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 509543 11.91 MB/sec execute 113 sec latency 30.557 ms 2024-02-14T15:08:52.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 512032 11.93 MB/sec execute 114 sec latency 13.397 ms 2024-02-14T15:08:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:52 smithi050 ceph-mon[36512]: pgmap v166: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-02-14T15:08:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:52 smithi076 ceph-mon[37057]: pgmap v166: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-02-14T15:08:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:52 smithi017 ceph-mon[32549]: pgmap v166: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-02-14T15:08:53.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 514189 11.89 MB/sec execute 115 sec latency 19.450 ms 2024-02-14T15:08:54.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 515932 11.89 MB/sec execute 116 sec latency 83.369 ms 2024-02-14T15:08:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:54 smithi050 ceph-mon[36512]: pgmap v167: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-02-14T15:08:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:54 smithi076 ceph-mon[37057]: pgmap v167: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-02-14T15:08:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:54 smithi017 ceph-mon[32549]: pgmap v167: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-02-14T15:08:55.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 518765 11.94 MB/sec execute 117 sec latency 21.126 ms 2024-02-14T15:08:56.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 520814 11.90 MB/sec execute 118 sec latency 19.505 ms 2024-02-14T15:08:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:56 smithi050 ceph-mon[36512]: pgmap v168: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-02-14T15:08:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:56 smithi076 ceph-mon[37057]: pgmap v168: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-02-14T15:08:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:56 smithi017 ceph-mon[32549]: pgmap v168: 129 pgs: 129 active+clean; 303 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-02-14T15:08:57.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 523040 11.91 MB/sec execute 119 sec latency 20.929 ms 2024-02-14T15:08:58.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 525978 11.97 MB/sec execute 120 sec latency 14.599 ms 2024-02-14T15:08:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:08:58 smithi050 ceph-mon[36512]: pgmap v169: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 276 op/s 2024-02-14T15:08:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:08:58 smithi076 ceph-mon[37057]: pgmap v169: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 276 op/s 2024-02-14T15:08:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:08:58 smithi017 ceph-mon[32549]: pgmap v169: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 276 op/s 2024-02-14T15:08:59.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 527459 11.91 MB/sec execute 121 sec latency 95.450 ms 2024-02-14T15:09:00.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 529716 11.93 MB/sec execute 122 sec latency 26.396 ms 2024-02-14T15:09:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:00 smithi050 ceph-mon[36512]: pgmap v170: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-02-14T15:09:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:00 smithi076 ceph-mon[37057]: pgmap v170: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-02-14T15:09:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:00 smithi017 ceph-mon[32549]: pgmap v170: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 200 op/s 2024-02-14T15:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 -- 172.21.15.17:0/618359526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 msgr2=0x7f2f4410e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/618359526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4410e0e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009cd0 tx=0x7f2f3c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 -- 172.21.15.17:0/618359526 shutdown_connections 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/618359526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4410e0e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/618359526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f440696f0 0x7f2f44105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/618359526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f44068d40 0x7f2f44069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 -- 172.21.15.17:0/618359526 >> 172.21.15.17:0/618359526 conn(0x7f2f44077220 msgr2=0x7f2f44077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 -- 172.21.15.17:0/618359526 shutdown_connections 2024-02-14T15:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 -- 172.21.15.17:0/618359526 wait complete. 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.511+0000 7f2f48b4f640 1 Processor -- start 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f48b4f640 1 -- start start 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f48b4f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f44068d40 0x7f2f4419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f48b4f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f48b4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f441976a0 0x7f2f44197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f48b4f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f44198170 con 0x7f2f44106510 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f48b4f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441982e0 con 0x7f2f44068d40 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f48b4f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441984a0 con 0x7f2f441976a0 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f42d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f441976a0 0x7f2f44197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35782/0 (socket says 172.21.15.17:35782) 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 -- 172.21.15.17:0/113234326 learned_addr learned my addr 172.21.15.17:0/113234326 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f4259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f44068d40 0x7f2f4419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f441976a0 0x7f2f44197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:49498/0 (socket says 172.21.15.17:49498) 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 -- 172.21.15.17:0/113234326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f441976a0 msgr2=0x7f2f44197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f441976a0 0x7f2f44197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 -- 172.21.15.17:0/113234326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f44068d40 msgr2=0x7f2f4419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f44068d40 0x7f2f4419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.512+0000 7f2f39d9b640 1 -- 172.21.15.17:0/113234326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c009900 con 0x7f2f44106510 2024-02-14T15:09:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.513+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4419dad0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9f0 tx=0x7f2f2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.513+0000 7f2f3bfff640 1 -- 172.21.15.17:0/113234326 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0096c0 con 0x7f2f44106510 2024-02-14T15:09:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.513+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f441a43f0 con 0x7f2f44106510 2024-02-14T15:09:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.513+0000 7f2f3bfff640 1 -- 172.21.15.17:0/113234326 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f2c005500 con 0x7f2f44106510 2024-02-14T15:09:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.513+0000 7f2f3bfff640 1 -- 172.21.15.17:0/113234326 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c010430 con 0x7f2f44106510 2024-02-14T15:09:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.513+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f441a48d0 con 0x7f2f44106510 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.515+0000 7f2f3bfff640 1 -- 172.21.15.17:0/113234326 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2f2c005020 con 0x7f2f44106510 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 Processor -- start 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 -- start start 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f3bfff640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2f300a2220 0x7f2f300a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db28269e0 con 0x564db28b9040 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f3bfff640 1 -- 172.21.15.17:0/113234326 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2f2c014070 con 0x7f2f44106510 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db2823f10 con 0x564db28c1c20 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db281f6f0 con 0x564db28b8720 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2f300a2220 0x7f2f300a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:49508/0 (socket says 172.21.15.17:49508) 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f4259c640 1 -- 172.21.15.17:0/2232509962 learned_addr learned my addr 172.21.15.17:0/2232509962 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.516+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f4259c640 1 -- 172.21.15.17:0/2232509962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 msgr2=0x564db28c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f4259c640 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f4259c640 1 -- 172.21.15.17:0/2232509962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 msgr2=0x564db28c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f4259c640 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28c16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f4259c640 1 -- 172.21.15.17:0/2232509962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db2530920 con 0x564db28b8720 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2f300a2220 0x7f2f300a46e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c02c630 tx=0x7f2f3c005f60 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f4259c640 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28b8b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009cf0 tx=0x7f2f34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c422857a5c4b5cf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f38d99640 1 -- 172.21.15.17:0/2232509962 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34004020 con 0x564db28b8720 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f38d99640 1 -- 172.21.15.17:0/2232509962 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f340035f0 con 0x564db28b8720 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.517+0000 7f2f38d99640 1 -- 172.21.15.17:0/2232509962 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34018630 con 0x564db28b8720 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/2232509962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 msgr2=0x564db28b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28b8b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009cf0 tx=0x7f2f34011810 comp rx=0 tx=0).stop 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/2232509962 shutdown_connections 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/2232509962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28b8b00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/2232509962 >> 172.21.15.17:0/2232509962 conn(0x564db28b2e50 msgr2=0x564db28b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/2232509962 shutdown_connections 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/2232509962 wait complete. 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 -- start start 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.518+0000 7f2f4b7f6740 1 Processor -- start 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db27bbfa0 con 0x564db28b9040 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db24f3250 con 0x564db28c1c20 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db27a5200 con 0x564db28b8720 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 10 client.0 _ll_register_callbacks cb 0x564db2814b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28be900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35810/0 (socket says 172.21.15.17:35810) 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 -- 172.21.15.17:0/3975838632 learned_addr learned my addr 172.21.15.17:0/3975838632 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 -- 172.21.15.17:0/3975838632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 msgr2=0x564db28c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 -- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 msgr2=0x564db28bee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 -- 172.21.15.17:0/3975838632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f34009950 con 0x564db28b9040 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28be900 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c6a0 tx=0x7f2f2c09a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2c950b552b59bf server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f29ffb640 1 -- 172.21.15.17:0/3975838632 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c09eee0 con 0x564db28b9040 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f29ffb640 1 -- 172.21.15.17:0/3975838632 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2f2c0a3530 con 0x564db28b9040 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.519+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564db251f600 con 0x564db28b9040 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f29ffb640 1 -- 172.21.15.17:0/3975838632 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2c099090 con 0x564db28b9040 2024-02-14T15:09:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 10 client.16740 fetch_fsmap learned FSMap version 29 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564db281dd70 con 0x564db28b9040 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f29ffb640 1 -- 172.21.15.17:0/3975838632 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f2c0b2080 con 0x564db28b9040 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f29ffb640 1 -- 172.21.15.17:0/3975838632 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f2c01f060 con 0x564db28b9040 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 10 client.16740 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 20 client.16740 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db28d19e0 0x564db28d3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 4 client.16740 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db25701b0 con 0x564db28d19e0 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.520+0000 7f2f4259c640 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db28d19e0 0x564db28d3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.521+0000 7f2f4259c640 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db28d19e0 0x564db28d3df0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab311c579dbe2b0 server_cookie=1f6fb85741a071b1 in_seq=0 out_seq=0 2024-02-14T15:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.521+0000 7f2f29ffb640 10 client.16740 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.521+0000 7f2f29ffb640 1 -- 172.21.15.17:0/3975838632 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564db25701b0 con 0x564db28d19e0 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.521+0000 7f2f29ffb640 10 client.16740 handle_command_reply: tid=0 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.521+0000 7f2f4b7f6740 1 client.16740 shutdown 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 msgr2=0x564db28be900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28be900 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c6a0 tx=0x7f2f2c09a820 comp rx=0 tx=0).stop 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 shutdown_connections 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db28d19e0 0x564db28d3df0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b9040 0x564db28be900 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/3975838632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 >> 172.21.15.17:0/3975838632 conn(0x564db28b2e50 msgr2=0x564db28b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f29ffb640 0 client.16740 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 shutdown_connections 2024-02-14T15:09:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/3975838632 wait complete. 2024-02-14T15:09:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.522+0000 7f2f4b7f6740 20 client.16740 trim_cache size 0 max 16384 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.535+0000 7f2f4b7f6740 1 Processor -- start 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.535+0000 7f2f4b7f6740 1 -- start start 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b8720 0x564db28c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28c0c70 0x564db28bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db28b83b0 con 0x564db28b8720 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db28b3e50 con 0x564db28c1c20 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db28b12d0 con 0x564db28c0c70 2024-02-14T15:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b8720 0x564db28c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b8720 0x564db28c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35822/0 (socket says 172.21.15.17:35822) 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 -- 172.21.15.17:0/1088298510 learned_addr learned my addr 172.21.15.17:0/1088298510 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28c0c70 0x564db28bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 -- 172.21.15.17:0/1088298510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28c0c70 msgr2=0x564db28bc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28c0c70 0x564db28bc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 -- 172.21.15.17:0/1088298510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 msgr2=0x564db28bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 -- 172.21.15.17:0/1088298510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db27ba820 con 0x564db28b8720 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f4259c640 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b8720 0x564db28c0730 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f34000c00 tx=0x7f2f34005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3096151ad862e9b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f2bfff640 1 -- 172.21.15.17:0/1088298510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34005390 con 0x564db28b8720 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f2bfff640 1 -- 172.21.15.17:0/1088298510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f340054f0 con 0x564db28b8720 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.536+0000 7f2f2bfff640 1 -- 172.21.15.17:0/1088298510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34026770 con 0x564db28b8720 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1088298510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b8720 msgr2=0x564db28c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b8720 0x564db28c0730 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f34000c00 tx=0x7f2f34005d30 comp rx=0 tx=0).stop 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1088298510 shutdown_connections 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28c0c70 0x564db28bc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c1c20 0x564db28bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1088298510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28b8720 0x564db28c0730 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1088298510 >> 172.21.15.17:0/1088298510 conn(0x564db28b2e50 msgr2=0x564db28c0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1088298510 shutdown_connections 2024-02-14T15:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.537+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1088298510 wait complete. 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 -- start start 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 Processor -- start 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c0c70 0x564db28c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28c1c20 0x564db2922010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db2922550 con 0x564db28c1c20 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db2922690 con 0x564db28c0c70 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db2922850 con 0x564db28b8720 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4b7f6740 10 client.0 _ll_register_callbacks cb 0x7f2f340097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c0c70 0x564db28c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f39d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c0c70 0x564db28c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:35744/0 (socket says 172.21.15.17:35744) 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f39d9b640 1 -- 172.21.15.17:0/1630568571 learned_addr learned my addr 172.21.15.17:0/1630568571 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28c1c20 0x564db2922010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4259c640 1 -- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c0c70 msgr2=0x564db28c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4259c640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c0c70 0x564db28c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4259c640 1 -- 172.21.15.17:0/1630568571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28c1c20 msgr2=0x564db2922010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4259c640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28c1c20 0x564db2922010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.538+0000 7f2f4259c640 1 -- 172.21.15.17:0/1630568571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564db281dd70 con 0x564db28b8720 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c0c70 0x564db28c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f42d9d640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28c1c20 0x564db2922010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f4259c640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c7e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f340041e0 tx=0x7f2f34005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f333ac61be3a72 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f29ffb640 1 -- 172.21.15.17:0/1630568571 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34011dc0 con 0x564db28b8720 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f340035f0 con 0x564db28b8720 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f29ffb640 1 -- 172.21.15.17:0/1630568571 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2f34003760 con 0x564db28b8720 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f29ffb640 1 -- 172.21.15.17:0/1630568571 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f34005660 con 0x564db28b8720 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f4b7f6740 10 client.26269 fetch_fsmap learned FSMap version 29 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2f3401d040 con 0x564db28b8720 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.539+0000 7f2f29ffb640 1 -- 172.21.15.17:0/1630568571 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f340035f0 con 0x564db28b8720 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f29ffb640 1 -- 172.21.15.17:0/1630568571 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f34033070 con 0x564db28b8720 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f4b7f6740 10 client.26269 fetch_fsmap finished waiting for FSMap version 29 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f4b7f6740 10 client.26269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f4b7f6740 20 client.26269 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db2929df0 0x564db292c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f4b7f6740 4 client.26269 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564db255c550 con 0x564db2929df0 2024-02-14T15:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db2929df0 0x564db292c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f29ffb640 10 client.26269 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:01.540+0000 7f2f39d9b640 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db2929df0 0x564db292c1e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736466fc3bbab919 server_cookie=14374906a259c5c4 in_seq=0 out_seq=0 2024-02-14T15:09:01.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 530399 11.85 MB/sec execute 123 sec latency 58.647 ms 2024-02-14T15:09:02.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 533096 11.89 MB/sec execute 124 sec latency 33.125 ms 2024-02-14T15:09:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:02 smithi050 ceph-mon[36512]: pgmap v171: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 313 op/s 2024-02-14T15:09:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.172+0000 7f2f29ffb640 1 -- 172.21.15.17:0/1630568571 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564db255c550 con 0x564db2929df0 2024-02-14T15:09:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.172+0000 7f2f29ffb640 10 client.26269 handle_command_reply: tid=0 2024-02-14T15:09:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.172+0000 7f2f4b7f6740 1 client.26269 shutdown 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 msgr2=0x564db28c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c7e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f340041e0 tx=0x7f2f34005f10 comp rx=0 tx=0).stop 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 shutdown_connections 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x564db2929df0 0x564db292c1e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564db28c1c20 0x564db2922010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564db28c0c70 0x564db28c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 --2- 172.21.15.17:0/1630568571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564db28b8720 0x564db28c7e70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f29ffb640 0 client.26269 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 >> 172.21.15.17:0/1630568571 conn(0x564db28b2e50 msgr2=0x564db28b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 shutdown_connections 2024-02-14T15:09:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 1 -- 172.21.15.17:0/1630568571 wait complete. 2024-02-14T15:09:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f4b7f6740 20 client.26269 trim_cache size 0 max 16384 2024-02-14T15:09:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:09:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:09:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:09:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2f300a2220 msgr2=0x7f2f300a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2f300a2220 0x7f2f300a46e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c02c630 tx=0x7f2f3c005f60 comp rx=0 tx=0).stop 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 msgr2=0x7f2f4419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4419dad0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9f0 tx=0x7f2f2c00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 shutdown_connections 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2f300a2220 0x7f2f300a46e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f44106510 0x7f2f4419dad0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f441976a0 0x7f2f44197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 --2- 172.21.15.17:0/113234326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f44068d40 0x7f2f4419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.173+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 >> 172.21.15.17:0/113234326 conn(0x7f2f44077220 msgr2=0x7f2f440fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.174+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 shutdown_connections 2024-02-14T15:09:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:03.174+0000 7f2f48b4f640 1 -- 172.21.15.17:0/113234326 wait complete. 2024-02-14T15:09:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:02 smithi076 ceph-mon[37057]: pgmap v171: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 313 op/s 2024-02-14T15:09:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:02 smithi017 ceph-mon[32549]: pgmap v171: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 313 op/s 2024-02-14T15:09:03.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 534804 11.84 MB/sec execute 125 sec latency 20.630 ms 2024-02-14T15:09:03.893 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: *** Caught signal (Segmentation fault) ** 2024-02-14T15:09:03.893 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: in thread 7f63a93eb700 thread_name:md_log_replay 2024-02-14T15:09:03.893 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:09:03.893 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f63ba7acd20] 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55ebf1081d5f] 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ebf1083f93] 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 4: (MDLog::_replay_thread()+0x647) [0x55ebf10e07b7] 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ebf10ea651] 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f63ba7a21ca] 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 7: clone() 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2024-02-14T15:09:03.432+0000 7f63a93eb700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: in thread 7f63a93eb700 thread_name:md_log_replay 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2024-02-14T15:09:03.894 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f63ba7acd20] 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55ebf1081d5f] 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ebf1083f93] 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 4: (MDLog::_replay_thread()+0x647) [0x55ebf10e07b7] 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ebf10ea651] 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f63ba7a21ca] 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 7: clone() 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2024-02-14T15:09:03.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 0> 2024-02-14T15:09:03.432+0000 7f63a93eb700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: in thread 7f63a93eb700 thread_name:md_log_replay 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f63ba7acd20] 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55ebf1081d5f] 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ebf1083f93] 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 4: (MDLog::_replay_thread()+0x647) [0x55ebf10e07b7] 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ebf10ea651] 2024-02-14T15:09:03.896 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f63ba7a21ca] 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 7: clone() 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 0> 2024-02-14T15:09:03.432+0000 7f63a93eb700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: in thread 7f63a93eb700 thread_name:md_log_replay 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f63ba7acd20] 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55ebf1081d5f] 2024-02-14T15:09:03.897 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ebf1083f93] 2024-02-14T15:09:03.898 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 4: (MDLog::_replay_thread()+0x647) [0x55ebf10e07b7] 2024-02-14T15:09:03.898 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ebf10ea651] 2024-02-14T15:09:03.898 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f63ba7a21ca] 2024-02-14T15:09:03.898 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 7: clone() 2024-02-14T15:09:03.898 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:09:03.898 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[66897]: 2024-02-14T15:09:03.898 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:03 smithi050 podman[71133]: 2024-02-14 15:09:03.504693478 +0000 UTC m=+0.020858978 container died 62c016a31def367d2ea291198dba761de1da3779c686af7340a1bdd0390f4038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, org.label-schema.build-date=20240131) 2024-02-14T15:09:04.642 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:04 smithi050 podman[71133]: 2024-02-14 15:09:04.388129932 +0000 UTC m=+0.904295420 container remove 62c016a31def367d2ea291198dba761de1da3779c686af7340a1bdd0390f4038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2) 2024-02-14T15:09:04.643 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:04 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-02-14T15:09:04.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 536879 11.85 MB/sec execute 126 sec latency 47.558 ms 2024-02-14T15:09:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:04 smithi050 ceph-mon[36512]: pgmap v172: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 210 op/s 2024-02-14T15:09:05.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:04 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-02-14T15:09:05.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:04 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Consumed 1.710s CPU time. 2024-02-14T15:09:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:04 smithi076 ceph-mon[37057]: pgmap v172: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 210 op/s 2024-02-14T15:09:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:04 smithi017 ceph-mon[32549]: pgmap v172: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 210 op/s 2024-02-14T15:09:05.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 538866 11.82 MB/sec execute 127 sec latency 43.343 ms 2024-02-14T15:09:06.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 540789 11.82 MB/sec execute 128 sec latency 18.941 ms 2024-02-14T15:09:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:06 smithi050 ceph-mon[36512]: pgmap v173: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 210 op/s 2024-02-14T15:09:07.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:06 smithi076 ceph-mon[37057]: pgmap v173: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 210 op/s 2024-02-14T15:09:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:06 smithi017 ceph-mon[32549]: pgmap v173: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 210 op/s 2024-02-14T15:09:07.870 INFO:tasks.workunit.client.0.smithi017.stdout: 1 543348 11.84 MB/sec execute 129 sec latency 66.517 ms 2024-02-14T15:09:08.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 545039 11.82 MB/sec execute 130 sec latency 14.222 ms 2024-02-14T15:09:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:08 smithi050 ceph-mon[36512]: pgmap v174: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 325 op/s 2024-02-14T15:09:09.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:08 smithi076 ceph-mon[37057]: pgmap v174: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 325 op/s 2024-02-14T15:09:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:08 smithi017 ceph-mon[32549]: pgmap v174: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 325 op/s 2024-02-14T15:09:09.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 547322 11.84 MB/sec execute 131 sec latency 69.150 ms 2024-02-14T15:09:10.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 549161 11.79 MB/sec execute 132 sec latency 24.243 ms 2024-02-14T15:09:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:10 smithi050 ceph-mon[36512]: pgmap v175: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 228 op/s 2024-02-14T15:09:11.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:10 smithi076 ceph-mon[37057]: pgmap v175: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 228 op/s 2024-02-14T15:09:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:10 smithi017 ceph-mon[32549]: pgmap v175: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 228 op/s 2024-02-14T15:09:11.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 551404 11.81 MB/sec execute 133 sec latency 19.705 ms 2024-02-14T15:09:12.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 553985 11.83 MB/sec execute 134 sec latency 80.635 ms 2024-02-14T15:09:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:12 smithi050 ceph-mon[36512]: pgmap v176: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 327 op/s 2024-02-14T15:09:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:12 smithi076 ceph-mon[37057]: pgmap v176: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 327 op/s 2024-02-14T15:09:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:12 smithi017 ceph-mon[32549]: pgmap v176: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 327 op/s 2024-02-14T15:09:13.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 555219 11.79 MB/sec execute 135 sec latency 146.486 ms 2024-02-14T15:09:14.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 557816 11.82 MB/sec execute 136 sec latency 19.468 ms 2024-02-14T15:09:15.119 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:14 smithi050 ceph-mon[36512]: pgmap v177: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:09:15.120 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:14 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-02-14T15:09:15.120 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:14 smithi050 systemd[1]: Stopped Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:09:15.120 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:14 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Consumed 1.710s CPU time. 2024-02-14T15:09:15.120 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:14 smithi050 systemd[1]: Starting Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:09:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:14 smithi076 ceph-mon[37057]: pgmap v177: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:09:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:14 smithi017 ceph-mon[32549]: pgmap v177: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:09:15.386 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 podman[71241]: 2024-02-14 15:09:15.022033063 +0000 UTC m=+0.019582189 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:09:15.386 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 podman[71241]: 2024-02-14 15:09:15.123049163 +0000 UTC m=+0.120598307 container create 17713ed48b61dfb22bb8baba90322cb37101ba3e9a21cce306a28b0793a7c9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:09:15.386 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 podman[71241]: 2024-02-14 15:09:15.359767271 +0000 UTC m=+0.357316400 container init 17713ed48b61dfb22bb8baba90322cb37101ba3e9a21cce306a28b0793a7c9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=) 2024-02-14T15:09:15.386 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 podman[71241]: 2024-02-14 15:09:15.362685542 +0000 UTC m=+0.360234659 container start 17713ed48b61dfb22bb8baba90322cb37101ba3e9a21cce306a28b0793a7c9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:09:15.386 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 ceph-mds[71260]: -- 172.21.15.50:0/1529998192 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560ed67d520 con 0x5560ed652c00 2024-02-14T15:09:15.642 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: starting mds.h at 2024-02-14T15:09:15.642 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 bash[71241]: 17713ed48b61dfb22bb8baba90322cb37101ba3e9a21cce306a28b0793a7c9e0 2024-02-14T15:09:15.642 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:09:15 smithi050 systemd[1]: Started Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:09:15.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 559158 11.78 MB/sec execute 137 sec latency 47.117 ms 2024-02-14T15:09:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:15 smithi050 ceph-mon[36512]: daemon mds.h restarted 2024-02-14T15:09:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:15 smithi050 ceph-mon[36512]: osdmap e95: 12 total, 12 up, 12 in 2024-02-14T15:09:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:15 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6836/575364744,v1:172.21.15.50:6837/575364744] up:boot 2024-02-14T15:09:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:15 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:09:16.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:15 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:09:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:15 smithi076 ceph-mon[37057]: daemon mds.h restarted 2024-02-14T15:09:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:15 smithi076 ceph-mon[37057]: osdmap e95: 12 total, 12 up, 12 in 2024-02-14T15:09:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:15 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6836/575364744,v1:172.21.15.50:6837/575364744] up:boot 2024-02-14T15:09:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:15 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:09:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:15 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:09:16.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:15 smithi017 ceph-mon[32549]: daemon mds.h restarted 2024-02-14T15:09:16.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:15 smithi017 ceph-mon[32549]: osdmap e95: 12 total, 12 up, 12 in 2024-02-14T15:09:16.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:15 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6836/575364744,v1:172.21.15.50:6837/575364744] up:boot 2024-02-14T15:09:16.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:15 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:09:16.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:15 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 -- 172.21.15.17:0/700266546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10105410 msgr2=0x7f3c1010cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/700266546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10105410 0x7f3c1010cfe0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009cd0 tx=0x7f3c0802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 -- 172.21.15.17:0/700266546 shutdown_connections 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/700266546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10105410 0x7f3c1010cfe0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/700266546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c10100cf0 0x7f3c10104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/700266546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c10100340 0x7f3c10100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 -- 172.21.15.17:0/700266546 >> 172.21.15.17:0/700266546 conn(0x7f3c10077220 msgr2=0x7f3c10077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 -- 172.21.15.17:0/700266546 shutdown_connections 2024-02-14T15:09:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 -- 172.21.15.17:0/700266546 wait complete. 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 Processor -- start 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 -- start start 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c10100340 0x7f3c10198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c10105410 0x7f3c10198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.269+0000 7f3c16d3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10199120 0x7f3c101a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c16d3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101a3d70 con 0x7f3c10199120 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c16d3e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101a3ee0 con 0x7f3c10105410 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c16d3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101a40a0 con 0x7f3c10100340 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10199120 0x7f3c101a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10199120 0x7f3c101a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51324/0 (socket says 172.21.15.17:51324) 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 -- 172.21.15.17:0/3439243198 learned_addr learned my addr 172.21.15.17:0/3439243198 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c14ada640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c10100340 0x7f3c10198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c07fff640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c10105410 0x7f3c10198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 -- 172.21.15.17:0/3439243198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c10100340 msgr2=0x7f3c10198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c10100340 0x7f3c10198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 -- 172.21.15.17:0/3439243198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c10105410 msgr2=0x7f3c10198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c10105410 0x7f3c10198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 -- 172.21.15.17:0/3439243198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c08009900 con 0x7f3c10199120 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c14ada640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c10100340 0x7f3c10198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c152db640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10199120 0x7f3c101a3680 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3c08004af0 tx=0x7f3c08032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.270+0000 7f3c05ffb640 1 -- 172.21.15.17:0/3439243198 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c080328b0 con 0x7f3c10199120 2024-02-14T15:09:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.271+0000 7f3c05ffb640 1 -- 172.21.15.17:0/3439243198 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c08003e50 con 0x7f3c10199120 2024-02-14T15:09:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.271+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c101a4320 con 0x7f3c10199120 2024-02-14T15:09:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.271+0000 7f3c05ffb640 1 -- 172.21.15.17:0/3439243198 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08004740 con 0x7f3c10199120 2024-02-14T15:09:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.271+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c101a47d0 con 0x7f3c10199120 2024-02-14T15:09:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.273+0000 7f3c05ffb640 1 -- 172.21.15.17:0/3439243198 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3c0800a040 con 0x7f3c10199120 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.273+0000 7f3c199e1740 1 Processor -- start 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.273+0000 7f3c199e1740 1 -- start start 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc876720 0x55fabc876b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc877040 0x55fabc87f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc87fc20 0x55fabc882010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c199e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc7e49e0 con 0x55fabc876720 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c199e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc7e1f10 con 0x55fabc877040 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c199e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc7dd6f0 con 0x55fabc87fc20 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c05ffb640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3bf80a2220 0x7f3bf80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc876720 0x55fabc876b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c14ada640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc877040 0x55fabc87f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc876720 0x55fabc876b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51328/0 (socket says 172.21.15.17:51328) 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c07fff640 1 -- 172.21.15.17:0/3579237590 learned_addr learned my addr 172.21.15.17:0/3579237590 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c05ffb640 1 -- 172.21.15.17:0/3439243198 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f3c080f0e70 con 0x7f3c10199120 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c07fff640 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc87fc20 0x55fabc882010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c152db640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3bf80a2220 0x7f3bf80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c14ada640 1 -- 172.21.15.17:0/3579237590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc87fc20 msgr2=0x55fabc882010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c14ada640 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc87fc20 0x55fabc882010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c14ada640 1 -- 172.21.15.17:0/3579237590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc876720 msgr2=0x55fabc876b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c14ada640 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc876720 0x55fabc876b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c14ada640 1 -- 172.21.15.17:0/3579237590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fabc4ee920 con 0x55fabc877040 2024-02-14T15:09:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c07fff640 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc876720 0x55fabc876b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c152db640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3bf80a2220 0x7f3bf80a46e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c080b9a70 tx=0x7f3c08004920 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3c14ada640 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc877040 0x55fabc87f6e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009f50 tx=0x7f3c0c013c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11754ebc93338b8c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.274+0000 7f3bf6ffd640 1 -- 172.21.15.17:0/3579237590 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c018070 con 0x55fabc877040 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3bf6ffd640 1 -- 172.21.15.17:0/3579237590 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c004020 con 0x55fabc877040 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3bf6ffd640 1 -- 172.21.15.17:0/3579237590 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c003b60 con 0x55fabc877040 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 -- 172.21.15.17:0/3579237590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc877040 msgr2=0x55fabc87f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc877040 0x55fabc87f6e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009f50 tx=0x7f3c0c013c60 comp rx=0 tx=0).stop 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 -- 172.21.15.17:0/3579237590 shutdown_connections 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc87fc20 0x55fabc882010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc877040 0x55fabc87f6e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 --2- 172.21.15.17:0/3579237590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc876720 0x55fabc876b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 -- 172.21.15.17:0/3579237590 >> 172.21.15.17:0/3579237590 conn(0x55fabc870e50 msgr2=0x55fabc871230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 -- 172.21.15.17:0/3579237590 shutdown_connections 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.275+0000 7f3c199e1740 1 -- 172.21.15.17:0/3579237590 wait complete. 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 -- start start 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 Processor -- start 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc779fa0 con 0x55fabc877040 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc4b1250 con 0x55fabc87fc20 2024-02-14T15:09:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc763200 con 0x55fabc876720 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c199e1740 10 client.0 _ll_register_callbacks cb 0x55fabc7d2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c14ada640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c14ada640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51332/0 (socket says 172.21.15.17:51332) 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c14ada640 1 -- 172.21.15.17:0/2395060572 learned_addr learned my addr 172.21.15.17:0/2395060572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c14ada640 1 -- 172.21.15.17:0/2395060572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 msgr2=0x55fabc87f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c14ada640 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c14ada640 1 -- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 msgr2=0x55fabc87ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.276+0000 7f3c14ada640 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3c14ada640 1 -- 172.21.15.17:0/2395060572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c009c00 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3c14ada640 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87c920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0036a0 tx=0x7f3c0c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f30e374688d595 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3beffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3beffff640 1 -- 172.21.15.17:0/2395060572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c018040 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3beffff640 1 -- 172.21.15.17:0/2395060572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f3c0c005050 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fabc4dd600 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3beffff640 1 -- 172.21.15.17:0/2395060572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c0c005210 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3c199e1740 10 client.16764 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fabc7dbd70 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3beffff640 1 -- 172.21.15.17:0/2395060572 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fabc4dd600 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.277+0000 7f3beffff640 1 -- 172.21.15.17:0/2395060572 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3c0c02e070 con 0x55fabc877040 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c199e1740 10 client.16764 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c199e1740 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c199e1740 20 client.16764 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc88f920 0x55fabc891d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c199e1740 4 client.16764 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fabc52e1b0 con 0x55fabc88f920 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc88f920 0x55fabc891d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc88f920 0x55fabc891d40 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4174b484a41d458a server_cookie=6b35140826999886 in_seq=0 out_seq=0 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.278+0000 7f3beffff640 10 client.16764 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3beffff640 1 -- 172.21.15.17:0/2395060572 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fabc52e1b0 con 0x55fabc88f920 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3beffff640 10 client.16764 handle_command_reply: tid=0 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 client.16764 shutdown 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 msgr2=0x55fabc87c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87c920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0036a0 tx=0x7f3c0c0045e0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 shutdown_connections 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc88f920 0x55fabc891d40 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87c920 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3beffff640 0 client.16764 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2395060572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 >> 172.21.15.17:0/2395060572 conn(0x55fabc870e50 msgr2=0x55fabc872ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 shutdown_connections 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 1 -- 172.21.15.17:0/2395060572 wait complete. 2024-02-14T15:09:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.279+0000 7f3c199e1740 20 client.16764 trim_cache size 0 max 16384 2024-02-14T15:09:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 Processor -- start 2024-02-14T15:09:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 -- start start 2024-02-14T15:09:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc86c0f0 con 0x55fabc877040 2024-02-14T15:09:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc872040 con 0x55fabc87fc20 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.294+0000 7f3c199e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc896ef0 con 0x55fabc876720 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51346/0 (socket says 172.21.15.17:51346) 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 -- 172.21.15.17:0/1822356234 learned_addr learned my addr 172.21.15.17:0/1822356234 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c07fff640 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c07fff640 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 -- 172.21.15.17:0/1822356234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 msgr2=0x55fabc87f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 -- 172.21.15.17:0/1822356234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 msgr2=0x55fabc87c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 -- 172.21.15.17:0/1822356234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fabc778820 con 0x55fabc877040 2024-02-14T15:09:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3c14ada640 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87f590 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0059f0 tx=0x7f3c0c003750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90bd6054bf41d37 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3bf67fc640 1 -- 172.21.15.17:0/1822356234 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c00e800 con 0x55fabc877040 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3bf67fc640 1 -- 172.21.15.17:0/1822356234 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c003ca0 con 0x55fabc877040 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.295+0000 7f3bf67fc640 1 -- 172.21.15.17:0/1822356234 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c004f60 con 0x55fabc877040 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 -- 172.21.15.17:0/1822356234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 msgr2=0x55fabc87f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87f590 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0059f0 tx=0x7f3c0c003750 comp rx=0 tx=0).stop 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 -- 172.21.15.17:0/1822356234 shutdown_connections 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc877040 0x55fabc87f590 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc87fc20 0x55fabc87c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 --2- 172.21.15.17:0/1822356234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc876720 0x55fabc87f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 -- 172.21.15.17:0/1822356234 >> 172.21.15.17:0/1822356234 conn(0x55fabc870e50 msgr2=0x55fabc87b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 -- 172.21.15.17:0/1822356234 shutdown_connections 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.296+0000 7f3c199e1740 1 -- 172.21.15.17:0/1822356234 wait complete. 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 -- start start 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 Processor -- start 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc876720 0x55fabc886b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc877040 0x55fabc885010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc87fc20 0x55fabc885550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc8873f0 con 0x55fabc87fc20 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc885a90 con 0x55fabc876720 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fabc885c50 con 0x55fabc877040 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c199e1740 10 client.0 _ll_register_callbacks cb 0x7f3c0c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc87fc20 0x55fabc885550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c14ada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc877040 0x55fabc885010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc87fc20 0x55fabc885550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51358/0 (socket says 172.21.15.17:51358) 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.297+0000 7f3c07fff640 1 -- 172.21.15.17:0/2345743953 learned_addr learned my addr 172.21.15.17:0/2345743953 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc876720 0x55fabc886b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c07fff640 1 -- 172.21.15.17:0/2345743953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc877040 msgr2=0x55fabc885010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc877040 0x55fabc885010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c07fff640 1 -- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc876720 msgr2=0x55fabc886b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc876720 0x55fabc886b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c07fff640 1 -- 172.21.15.17:0/2345743953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fabc7dbd70 con 0x55fabc87fc20 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c14ada640 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc877040 0x55fabc885010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c07fff640 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc87fc20 0x55fabc885550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3c000058c0 tx=0x7f3c0c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16352ba6a0b52a0f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3beffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3beffff640 1 -- 172.21.15.17:0/2345743953 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00013e30 con 0x55fabc87fc20 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3beffff640 1 -- 172.21.15.17:0/2345743953 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f3c000035f0 con 0x55fabc87fc20 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c0c004020 con 0x55fabc87fc20 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3beffff640 1 -- 172.21.15.17:0/2345743953 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c000037b0 con 0x55fabc87fc20 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c199e1740 10 client.16770 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fabc79e720 con 0x55fabc87fc20 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.298+0000 7f3beffff640 1 -- 172.21.15.17:0/2345743953 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c004020 con 0x55fabc87fc20 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3beffff640 1 -- 172.21.15.17:0/2345743953 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3c00033820 con 0x55fabc87fc20 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c199e1740 10 client.16770 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c199e1740 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c199e1740 20 client.16770 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc8e72c0 0x55fabc8e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c199e1740 4 client.16770 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:09:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fabc51a550 con 0x55fabc8e72c0 2024-02-14T15:09:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c14ada640 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc8e72c0 0x55fabc8e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3c14ada640 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc8e72c0 0x55fabc8e96e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60190d2a9934200 server_cookie=ddf8308faa9b2dcd in_seq=0 out_seq=0 2024-02-14T15:09:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.299+0000 7f3beffff640 10 client.16770 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3beffff640 1 -- 172.21.15.17:0/2345743953 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fabc51a550 con 0x55fabc8e72c0 2024-02-14T15:09:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3beffff640 10 client.16770 handle_command_reply: tid=0 2024-02-14T15:09:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 client.16770 shutdown 2024-02-14T15:09:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc87fc20 msgr2=0x55fabc885550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc87fc20 0x55fabc885550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3c000058c0 tx=0x7f3c0c0044c0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 shutdown_connections 2024-02-14T15:09:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55fabc8e72c0 0x55fabc8e96e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fabc87fc20 0x55fabc885550 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fabc877040 0x55fabc885010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 --2- 172.21.15.17:0/2345743953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fabc876720 0x55fabc886b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3beffff640 0 client.16770 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 >> 172.21.15.17:0/2345743953 conn(0x55fabc870e50 msgr2=0x55fabc87c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 shutdown_connections 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 1 -- 172.21.15.17:0/2345743953 wait complete. 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c199e1740 20 client.16770 trim_cache size 0 max 16384 2024-02-14T15:09:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:09:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:09:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:09:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:09:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3bf80a2220 msgr2=0x7f3bf80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3bf80a2220 0x7f3bf80a46e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c080b9a70 tx=0x7f3c08004920 comp rx=0 tx=0).stop 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10199120 msgr2=0x7f3c101a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.310+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10199120 0x7f3c101a3680 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3c08004af0 tx=0x7f3c08032570 comp rx=0 tx=0).stop 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 shutdown_connections 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f3bf80a2220 0x7f3bf80a46e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c10199120 0x7f3c101a3680 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c10105410 0x7f3c10198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 --2- 172.21.15.17:0/3439243198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c10100340 0x7f3c10198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 >> 172.21.15.17:0/3439243198 conn(0x7f3c10077220 msgr2=0x7f3c100fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 shutdown_connections 2024-02-14T15:09:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:16.311+0000 7f3c16d3e640 1 -- 172.21.15.17:0/3439243198 wait complete. 2024-02-14T15:09:16.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 561659 11.79 MB/sec execute 138 sec latency 15.201 ms 2024-02-14T15:09:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:16 smithi050 ceph-mon[36512]: pgmap v178: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:09:17.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:16 smithi076 ceph-mon[37057]: pgmap v178: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:09:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:16 smithi017 ceph-mon[32549]: pgmap v178: 129 pgs: 129 active+clean; 351 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-02-14T15:09:17.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 563836 11.77 MB/sec execute 139 sec latency 35.121 ms 2024-02-14T15:09:18.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 565898 11.78 MB/sec execute 140 sec latency 16.662 ms 2024-02-14T15:09:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:18 smithi050 ceph-mon[36512]: pgmap v180: 129 pgs: 129 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:09:19.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:18 smithi076 ceph-mon[37057]: pgmap v180: 129 pgs: 129 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:09:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:18 smithi017 ceph-mon[32549]: pgmap v180: 129 pgs: 129 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:09:19.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 568569 11.81 MB/sec execute 141 sec latency 16.604 ms 2024-02-14T15:09:20.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 569993 11.77 MB/sec execute 142 sec latency 77.443 ms 2024-02-14T15:09:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:20 smithi050 ceph-mon[36512]: pgmap v181: 129 pgs: 129 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:09:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:20 smithi076 ceph-mon[37057]: pgmap v181: 129 pgs: 129 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:09:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:20 smithi017 ceph-mon[32549]: pgmap v181: 129 pgs: 129 active+clean; 361 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:09:21.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 572426 11.78 MB/sec execute 143 sec latency 12.932 ms 2024-02-14T15:09:22.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 574908 11.79 MB/sec execute 144 sec latency 28.770 ms 2024-02-14T15:09:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:22 smithi050 ceph-mon[36512]: pgmap v182: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:23.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:22 smithi076 ceph-mon[37057]: pgmap v182: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:22 smithi017 ceph-mon[32549]: pgmap v182: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:23.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 576927 11.79 MB/sec execute 145 sec latency 19.371 ms 2024-02-14T15:09:24.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 579194 11.80 MB/sec execute 146 sec latency 160.009 ms 2024-02-14T15:09:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:24 smithi050 ceph-mon[36512]: pgmap v183: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:24 smithi076 ceph-mon[37057]: pgmap v183: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:24 smithi017 ceph-mon[32549]: pgmap v183: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:25.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 580486 11.76 MB/sec execute 147 sec latency 48.421 ms 2024-02-14T15:09:26.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 582972 11.77 MB/sec execute 148 sec latency 13.405 ms 2024-02-14T15:09:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:26 smithi050 ceph-mon[36512]: pgmap v184: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:26 smithi076 ceph-mon[37057]: pgmap v184: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:26 smithi017 ceph-mon[32549]: pgmap v184: 129 pgs: 129 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-02-14T15:09:27.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 585353 11.77 MB/sec execute 149 sec latency 23.027 ms 2024-02-14T15:09:28.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 587250 11.76 MB/sec execute 150 sec latency 16.185 ms 2024-02-14T15:09:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:28 smithi050 ceph-mon[36512]: pgmap v185: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 330 op/s 2024-02-14T15:09:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:28 smithi076 ceph-mon[37057]: pgmap v185: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 330 op/s 2024-02-14T15:09:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:28 smithi017 ceph-mon[32549]: pgmap v185: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 330 op/s 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 -- 172.21.15.17:0/2207493323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc100cf0 msgr2=0x7fd9dc104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2207493323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc100cf0 0x7fd9dc104d30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009930 tx=0x7fd9d002bc30 comp rx=0 tx=0).stop 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 -- 172.21.15.17:0/2207493323 shutdown_connections 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2207493323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc100cf0 0x7fd9dc104d30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2207493323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc105410 0x7fd9dc10cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2207493323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9dc100340 0x7fd9dc100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 -- 172.21.15.17:0/2207493323 >> 172.21.15.17:0/2207493323 conn(0x7fd9dc077220 msgr2=0x7fd9dc077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 -- 172.21.15.17:0/2207493323 shutdown_connections 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 -- 172.21.15.17:0/2207493323 wait complete. 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 Processor -- start 2024-02-14T15:09:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.405+0000 7fd9e253e640 1 -- start start 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e253e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc100340 0x7fd9dc101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e253e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9dc105410 0x7fd9dc102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e253e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc102850 0x7fd9dc1a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e253e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a6030 con 0x7fd9dc102850 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e253e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a61a0 con 0x7fd9dc105410 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e253e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a6360 con 0x7fd9dc100340 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e0adb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc102850 0x7fd9dc1a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc100340 0x7fd9dc101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e0adb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc102850 0x7fd9dc1a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43320/0 (socket says 172.21.15.17:43320) 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc100340 0x7fd9dc101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:48804/0 (socket says 172.21.15.17:48804) 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e0adb640 1 -- 172.21.15.17:0/2699689764 learned_addr learned my addr 172.21.15.17:0/2699689764 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9dc105410 0x7fd9dc102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 -- 172.21.15.17:0/2699689764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9dc105410 msgr2=0x7fd9dc102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9dc105410 0x7fd9dc102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 -- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc102850 msgr2=0x7fd9dc1a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc102850 0x7fd9dc1a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 -- 172.21.15.17:0/2699689764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0009560 con 0x7fd9dc100340 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9dc105410 0x7fd9dc102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e0adb640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc102850 0x7fd9dc1a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc100340 0x7fd9dc101dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9f0 tx=0x7fd9cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9d97fa640 1 -- 172.21.15.17:0/2699689764 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00ccc0 con 0x7fd9dc100340 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.406+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc1a6640 con 0x7fd9dc100340 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.407+0000 7fd9d97fa640 1 -- 172.21.15.17:0/2699689764 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc00ce20 con 0x7fd9dc100340 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.407+0000 7fd9d97fa640 1 -- 172.21.15.17:0/2699689764 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc018920 con 0x7fd9dc100340 2024-02-14T15:09:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.407+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc1a6b70 con 0x7fd9dc100340 2024-02-14T15:09:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.409+0000 7fd9d97fa640 1 -- 172.21.15.17:0/2699689764 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd9cc018a80 con 0x7fd9dc100340 2024-02-14T15:09:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.409+0000 7fd9e51ec740 1 Processor -- start 2024-02-14T15:09:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.409+0000 7fd9e51ec740 1 -- start start 2024-02-14T15:09:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.409+0000 7fd9d97fa640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9c40a2220 0x7fd9c40a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5c4040 0x563cbd5cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5ccc20 0x563cbd5cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd5319e0 con 0x563cbd5ccc20 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd52ef10 con 0x563cbd5c4040 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd52a6f0 con 0x563cbd5c3720 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e0adb640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9c40a2220 0x7fd9c40a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e0adb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5ccc20 0x563cbd5cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5c4040 0x563cbd5cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e0adb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5ccc20 0x563cbd5cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43332/0 (socket says 172.21.15.17:43332) 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:48810/0 (socket says 172.21.15.17:48810) 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5c4040 0x563cbd5cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:41986/0 (socket says 172.21.15.17:41986) 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e0adb640 1 -- 172.21.15.17:0/3786822018 learned_addr learned my addr 172.21.15.17:0/3786822018 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9d97fa640 1 -- 172.21.15.17:0/2699689764 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd9cc014070 con 0x7fd9dc100340 2024-02-14T15:09:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 -- 172.21.15.17:0/3786822018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5c4040 msgr2=0x563cbd5cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5c4040 0x563cbd5cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 -- 172.21.15.17:0/3786822018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5ccc20 msgr2=0x563cbd5cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5ccc20 0x563cbd5cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 -- 172.21.15.17:0/3786822018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbd23b920 con 0x563cbd5c3720 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e0adb640 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5ccc20 0x563cbd5cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9e0adb640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9c40a2220 0x7fd9c40a46e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4013570 tx=0x7fd9d4011770 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c3b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0cbb40 tx=0x7fd9cc09a740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c12eb262bccb905 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9c27fc640 1 -- 172.21.15.17:0/3786822018 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc06f210 con 0x563cbd5c3720 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9c27fc640 1 -- 172.21.15.17:0/3786822018 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc0665d0 con 0x563cbd5c3720 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.410+0000 7fd9c27fc640 1 -- 172.21.15.17:0/3786822018 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc097450 con 0x563cbd5c3720 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 -- 172.21.15.17:0/3786822018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 msgr2=0x563cbd5c3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c3b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0cbb40 tx=0x7fd9cc09a740 comp rx=0 tx=0).stop 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 -- 172.21.15.17:0/3786822018 shutdown_connections 2024-02-14T15:09:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5ccc20 0x563cbd5cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5c4040 0x563cbd5cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/3786822018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c3b00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 -- 172.21.15.17:0/3786822018 >> 172.21.15.17:0/3786822018 conn(0x563cbd5bde50 msgr2=0x563cbd5be230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 -- 172.21.15.17:0/3786822018 shutdown_connections 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 -- 172.21.15.17:0/3786822018 wait complete. 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.411+0000 7fd9e51ec740 1 -- start start 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 1 Processor -- start 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c3720 0x563cbd5cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c4040 0x563cbd5c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd4c6fa0 con 0x563cbd5c3720 2024-02-14T15:09:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd1fe250 con 0x563cbd5ccc20 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd4b0200 con 0x563cbd5c4040 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e51ec740 10 client.0 _ll_register_callbacks cb 0x563cbd51fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c3720 0x563cbd5cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c3720 0x563cbd5cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43336/0 (socket says 172.21.15.17:43336) 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 -- 172.21.15.17:0/2383550481 learned_addr learned my addr 172.21.15.17:0/2383550481 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c4040 0x563cbd5c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9e0adb640 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 -- 172.21.15.17:0/2383550481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c4040 msgr2=0x563cbd5c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c4040 0x563cbd5c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 -- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 msgr2=0x563cbd5c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 -- 172.21.15.17:0/2383550481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc1a6b70 con 0x563cbd5c3720 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.412+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c3720 0x563cbd5cc5e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9c0 tx=0x7fd9cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c692480441263954 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2383550481 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0cbfc0 con 0x563cbd5c3720 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2383550481 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd9cc067140 con 0x563cbd5c3720 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cbd22a600 con 0x563cbd5c3720 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2383550481 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9cc06f210 con 0x563cbd5c3720 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 10 client.16782 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563cbd528d70 con 0x563cbd5c3720 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2383550481 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc0b7020 con 0x563cbd5c3720 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2383550481 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd9cc0754b0 con 0x563cbd5c3720 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 10 client.16782 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 10 client.16782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 20 client.16782 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd5dc830 0x563cbd5dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 4 client.16782 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cbd27b1b0 con 0x563cbd5dc830 2024-02-14T15:09:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.413+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd5dc830 0x563cbd5dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.414+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd5dc830 0x563cbd5dec20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7919709885fa5a0b server_cookie=798a3fb36091094d in_seq=0 out_seq=0 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.414+0000 7fd9bb7fe640 10 client.16782 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.414+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2383550481 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cbd27b1b0 con 0x563cbd5dc830 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.414+0000 7fd9bb7fe640 10 client.16782 handle_command_reply: tid=0 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.414+0000 7fd9e51ec740 1 client.16782 shutdown 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c3720 msgr2=0x563cbd5cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c3720 0x563cbd5cc5e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9c0 tx=0x7fd9cc00cf90 comp rx=0 tx=0).stop 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 shutdown_connections 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd5dc830 0x563cbd5dec20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c4040 0x563cbd5c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2383550481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c3720 0x563cbd5cc5e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9bb7fe640 0 client.16782 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 >> 172.21.15.17:0/2383550481 conn(0x563cbd5bde50 msgr2=0x563cbd5bfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 shutdown_connections 2024-02-14T15:09:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2383550481 wait complete. 2024-02-14T15:09:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.415+0000 7fd9e51ec740 20 client.16782 trim_cache size 0 max 16384 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 Processor -- start 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 -- start start 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd5c7ef0 con 0x563cbd5c4040 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd5bd640 con 0x563cbd5ccc20 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd5c5430 con 0x563cbd5c3720 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43352/0 (socket says 172.21.15.17:43352) 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9db7fe640 1 -- 172.21.15.17:0/4074453934 learned_addr learned my addr 172.21.15.17:0/4074453934 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.429+0000 7fd9e0adb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9db7fe640 1 -- 172.21.15.17:0/4074453934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 msgr2=0x563cbd5c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9db7fe640 1 -- 172.21.15.17:0/4074453934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 msgr2=0x563cbd5cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9db7fe640 1 -- 172.21.15.17:0/4074453934 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbd4c5820 con 0x563cbd5c4040 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5cc0a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009900 tx=0x7fd9d00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ea03e069a4d0d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9c1ffb640 1 -- 172.21.15.17:0/4074453934 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0048be0 con 0x563cbd5c4040 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9c1ffb640 1 -- 172.21.15.17:0/4074453934 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9d0048d40 con 0x563cbd5c4040 2024-02-14T15:09:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.430+0000 7fd9c1ffb640 1 -- 172.21.15.17:0/4074453934 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0048600 con 0x563cbd5c4040 2024-02-14T15:09:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 -- 172.21.15.17:0/4074453934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 msgr2=0x563cbd5cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5cc0a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009900 tx=0x7fd9d00058b0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 -- 172.21.15.17:0/4074453934 shutdown_connections 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5cc0a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd5cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/4074453934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 -- 172.21.15.17:0/4074453934 >> 172.21.15.17:0/4074453934 conn(0x563cbd5bde50 msgr2=0x563cbd5c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 -- 172.21.15.17:0/4074453934 shutdown_connections 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 -- 172.21.15.17:0/4074453934 wait complete. 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 -- start start 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.431+0000 7fd9e51ec740 1 Processor -- start 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e51ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd632ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd5d3550 con 0x563cbd5c4040 2024-02-14T15:09:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd5d36c0 con 0x563cbd5ccc20 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e51ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbd6331e0 con 0x563cbd5c3720 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e51ec740 10 client.0 _ll_register_callbacks cb 0x7fd9cc010e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5d2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43362/0 (socket says 172.21.15.17:43362) 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 -- 172.21.15.17:0/2651736042 learned_addr learned my addr 172.21.15.17:0/2651736042 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e0adb640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd632ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 -- 172.21.15.17:0/2651736042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 msgr2=0x563cbd5d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 -- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 msgr2=0x563cbd632ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd632ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 -- 172.21.15.17:0/2651736042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cbd528d70 con 0x563cbd5c4040 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9e0adb640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd632ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9db7fe640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5d2dc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0036040 tx=0x7fd9d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd15f3fce43b719 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.432+0000 7fd9bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2651736042 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d002ccd0 con 0x563cbd5c4040 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2651736042 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd9d0045040 con 0x563cbd5c4040 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9cc0665d0 con 0x563cbd5c4040 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2651736042 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d0004040 con 0x563cbd5c4040 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 10 client.16788 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fd9dc10d7f0 con 0x563cbd5c4040 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2651736042 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc0665d0 con 0x563cbd5c4040 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2651736042 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd9d003c040 con 0x563cbd5c4040 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 10 client.16788 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 20 client.16788 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd6346f0 0x563cbd636b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 4 client.16788 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:09:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563cbd267550 con 0x563cbd6346f0 2024-02-14T15:09:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.433+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd6346f0 0x563cbd636b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.434+0000 7fd9dbfff640 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd6346f0 0x563cbd636b10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778d2ac64bb3b3c1 server_cookie=cfee60cf96759041 in_seq=0 out_seq=0 2024-02-14T15:09:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.434+0000 7fd9bb7fe640 10 client.16788 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.443+0000 7fd9bb7fe640 1 -- 172.21.15.17:0/2651736042 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563cbd267550 con 0x563cbd6346f0 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.443+0000 7fd9bb7fe640 10 client.16788 handle_command_reply: tid=0 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.443+0000 7fd9e51ec740 1 client.16788 shutdown 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 msgr2=0x563cbd5d2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5d2dc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0036040 tx=0x7fd9d0005fd0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 shutdown_connections 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563cbd6346f0 0x563cbd636b10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563cbd5c4040 0x563cbd5d2dc0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563cbd5ccc20 0x563cbd632ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9bb7fe640 0 client.16788 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 --2- 172.21.15.17:0/2651736042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cbd5c3720 0x563cbd5d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 >> 172.21.15.17:0/2651736042 conn(0x563cbd5bde50 msgr2=0x563cbd5bc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 shutdown_connections 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 1 -- 172.21.15.17:0/2651736042 wait complete. 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e51ec740 20 client.16788 trim_cache size 0 max 16384 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:09:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:09:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9c40a2220 msgr2=0x7fd9c40a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9c40a2220 0x7fd9c40a46e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4013570 tx=0x7fd9d4011770 comp rx=0 tx=0).stop 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc100340 msgr2=0x7fd9dc101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc100340 0x7fd9dc101dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9f0 tx=0x7fd9cc00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 shutdown_connections 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9c40a2220 0x7fd9c40a46e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9dc102850 0x7fd9dc1a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9dc105410 0x7fd9dc102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 --2- 172.21.15.17:0/2699689764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9dc100340 0x7fd9dc101dd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.444+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 >> 172.21.15.17:0/2699689764 conn(0x7fd9dc077220 msgr2=0x7fd9dc068dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.445+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 shutdown_connections 2024-02-14T15:09:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:29.445+0000 7fd9e253e640 1 -- 172.21.15.17:0/2699689764 wait complete. 2024-02-14T15:09:29.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 589625 11.78 MB/sec execute 151 sec latency 159.298 ms 2024-02-14T15:09:30.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 590775 11.73 MB/sec execute 152 sec latency 66.261 ms 2024-02-14T15:09:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:30 smithi050 ceph-mon[36512]: pgmap v186: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2024-02-14T15:09:31.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:30 smithi076 ceph-mon[37057]: pgmap v186: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2024-02-14T15:09:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:30 smithi017 ceph-mon[32549]: pgmap v186: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2024-02-14T15:09:31.871 INFO:tasks.workunit.client.0.smithi017.stdout: 1 593442 11.76 MB/sec execute 153 sec latency 12.998 ms 2024-02-14T15:09:32.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 595499 11.73 MB/sec execute 154 sec latency 26.521 ms 2024-02-14T15:09:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:32 smithi050 ceph-mon[36512]: pgmap v187: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 314 op/s 2024-02-14T15:09:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:32 smithi076 ceph-mon[37057]: pgmap v187: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 314 op/s 2024-02-14T15:09:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:32 smithi017 ceph-mon[32549]: pgmap v187: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 314 op/s 2024-02-14T15:09:33.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 597472 11.74 MB/sec execute 155 sec latency 16.678 ms 2024-02-14T15:09:34.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 600062 11.75 MB/sec execute 156 sec latency 64.293 ms 2024-02-14T15:09:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:34 smithi050 ceph-mon[36512]: pgmap v188: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 211 op/s 2024-02-14T15:09:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:34 smithi076 ceph-mon[37057]: pgmap v188: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 211 op/s 2024-02-14T15:09:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:34 smithi017 ceph-mon[32549]: pgmap v188: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 211 op/s 2024-02-14T15:09:35.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 601169 11.71 MB/sec execute 157 sec latency 69.264 ms 2024-02-14T15:09:36.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:35 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:09:36.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:35 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:09:36.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:35 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:09:36.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:35 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:09:36.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:35 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:09:36.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:35 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:09:36.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:35 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:09:36.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:35 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:09:36.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:35 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:09:36.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:35 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:09:36.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:35 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:09:36.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:35 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:09:36.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 603914 11.74 MB/sec execute 158 sec latency 14.845 ms 2024-02-14T15:09:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:36 smithi050 ceph-mon[36512]: pgmap v189: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 211 op/s 2024-02-14T15:09:37.167 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:36 smithi017 ceph-mon[32549]: pgmap v189: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 211 op/s 2024-02-14T15:09:37.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:36 smithi076 ceph-mon[37057]: pgmap v189: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 211 op/s 2024-02-14T15:09:37.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 605896 11.71 MB/sec execute 159 sec latency 14.627 ms 2024-02-14T15:09:38.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 608066 11.73 MB/sec execute 160 sec latency 17.826 ms 2024-02-14T15:09:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:38 smithi050 ceph-mon[36512]: pgmap v190: 129 pgs: 129 active+clean; 393 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2024-02-14T15:09:39.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:38 smithi076 ceph-mon[37057]: pgmap v190: 129 pgs: 129 active+clean; 393 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2024-02-14T15:09:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:38 smithi017 ceph-mon[32549]: pgmap v190: 129 pgs: 129 active+clean; 393 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2024-02-14T15:09:39.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 610914 11.76 MB/sec execute 161 sec latency 13.098 ms 2024-02-14T15:09:40.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 612347 11.73 MB/sec execute 162 sec latency 69.715 ms 2024-02-14T15:09:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:40 smithi050 ceph-mon[36512]: pgmap v191: 129 pgs: 129 active+clean; 393 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:09:41.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:40 smithi076 ceph-mon[37057]: pgmap v191: 129 pgs: 129 active+clean; 393 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:09:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:40 smithi017 ceph-mon[32549]: pgmap v191: 129 pgs: 129 active+clean; 393 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:09:41.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 614581 11.74 MB/sec execute 163 sec latency 16.740 ms 2024-02-14T15:09:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.545+0000 7fc930e15640 1 -- 172.21.15.17:0/2279622130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c072f20 msgr2=0x7fc92c06ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.545+0000 7fc930e15640 1 --2- 172.21.15.17:0/2279622130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c072f20 0x7fc92c06ad00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc924009ca0 tx=0x7fc92402c180 comp rx=0 tx=0).stop 2024-02-14T15:09:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.545+0000 7fc930e15640 1 -- 172.21.15.17:0/2279622130 shutdown_connections 2024-02-14T15:09:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.546+0000 7fc930e15640 1 --2- 172.21.15.17:0/2279622130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c06b240 0x7fc92c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.546+0000 7fc930e15640 1 --2- 172.21.15.17:0/2279622130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc92c113040 0x7fc92c115430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.546+0000 7fc930e15640 1 --2- 172.21.15.17:0/2279622130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c072f20 0x7fc92c06ad00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.546+0000 7fc930e15640 1 -- 172.21.15.17:0/2279622130 >> 172.21.15.17:0/2279622130 conn(0x7fc92c077220 msgr2=0x7fc92c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.546+0000 7fc930e15640 1 -- 172.21.15.17:0/2279622130 shutdown_connections 2024-02-14T15:09:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.546+0000 7fc930e15640 1 -- 172.21.15.17:0/2279622130 wait complete. 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 Processor -- start 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 -- start start 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c06b240 0x7fc92c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c072f20 0x7fc92c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc92c113040 0x7fc92c1a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1a3d90 con 0x7fc92c072f20 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1a3f00 con 0x7fc92c113040 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc930e15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1a40c0 con 0x7fc92c06b240 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc92a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c06b240 0x7fc92c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc929d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c072f20 0x7fc92c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc92a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c06b240 0x7fc92c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:53112/0 (socket says 172.21.15.17:53112) 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc92ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc92c113040 0x7fc92c1a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.547+0000 7fc92a59c640 1 -- 172.21.15.17:0/1912521210 learned_addr learned my addr 172.21.15.17:0/1912521210 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc92a59c640 1 -- 172.21.15.17:0/1912521210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc92c113040 msgr2=0x7fc92c1a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc92a59c640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc92c113040 0x7fc92c1a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc92a59c640 1 -- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c072f20 msgr2=0x7fc92c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc92a59c640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c072f20 0x7fc92c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc92a59c640 1 -- 172.21.15.17:0/1912521210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924009900 con 0x7fc92c06b240 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc92c113040 0x7fc92c1a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc929d9b640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c072f20 0x7fc92c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc92a59c640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c06b240 0x7fc92c19eed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc924005e30 tx=0x7fc924034830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc91b7fe640 1 -- 172.21.15.17:0/1912521210 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924003d80 con 0x7fc92c06b240 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc91b7fe640 1 -- 172.21.15.17:0/1912521210 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc924032c70 con 0x7fc92c06b240 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c1a4340 con 0x7fc92c06b240 2024-02-14T15:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc91b7fe640 1 -- 172.21.15.17:0/1912521210 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924040450 con 0x7fc92c06b240 2024-02-14T15:09:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.548+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1a4610 con 0x7fc92c06b240 2024-02-14T15:09:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.550+0000 7fc91b7fe640 1 -- 172.21.15.17:0/1912521210 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc92400a040 con 0x7fc92c06b240 2024-02-14T15:09:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.551+0000 7fc9339e1740 1 Processor -- start 2024-02-14T15:09:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.551+0000 7fc9339e1740 1 -- start start 2024-02-14T15:09:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.551+0000 7fc91b7fe640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9140a2220 0x7fc9140a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.551+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c6db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.551+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c76c20 0x55ebf0c79010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc9339e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0bdb9e0 con 0x55ebf0c6e040 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc9339e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0bd8f10 con 0x55ebf0c76c20 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc9339e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0bd46f0 con 0x55ebf0c6d720 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9140a2220 0x7fc9140a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53592/0 (socket says 172.21.15.17:53592) 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc92a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c6db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 -- 172.21.15.17:0/2521284187 learned_addr learned my addr 172.21.15.17:0/2521284187 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc91b7fe640 1 -- 172.21.15.17:0/1912521210 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc9240f0ff0 con 0x7fc92c06b240 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c76c20 0x55ebf0c79010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 -- 172.21.15.17:0/2521284187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 msgr2=0x55ebf0c6db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c6db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 -- 172.21.15.17:0/2521284187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c76c20 msgr2=0x55ebf0c79010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c76c20 0x55ebf0c79010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 -- 172.21.15.17:0/2521284187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebf08e5920 con 0x55ebf0c6e040 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9140a2220 0x7fc9140a46e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc91c013840 tx=0x7fc91c00aa90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc929d9b640 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c766e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc920010590 tx=0x7fc920010a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b3ba28203a12a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc918ff9640 1 -- 172.21.15.17:0/2521284187 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920015070 con 0x55ebf0c6e040 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc918ff9640 1 -- 172.21.15.17:0/2521284187 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc920005bf0 con 0x55ebf0c6e040 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.552+0000 7fc918ff9640 1 -- 172.21.15.17:0/2521284187 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920019a40 con 0x55ebf0c6e040 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 -- 172.21.15.17:0/2521284187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 msgr2=0x55ebf0c766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c766e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc920010590 tx=0x7fc920010a50 comp rx=0 tx=0).stop 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 -- 172.21.15.17:0/2521284187 shutdown_connections 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c766e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c76c20 0x55ebf0c79010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 --2- 172.21.15.17:0/2521284187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c6db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 -- 172.21.15.17:0/2521284187 >> 172.21.15.17:0/2521284187 conn(0x55ebf0c67e50 msgr2=0x55ebf0c68230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 -- 172.21.15.17:0/2521284187 shutdown_connections 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 -- 172.21.15.17:0/2521284187 wait complete. 2024-02-14T15:09:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.553+0000 7fc9339e1740 1 -- start start 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 1 Processor -- start 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c73f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0b70fa0 con 0x55ebf0c76c20 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf08a8250 con 0x55ebf0c6e040 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0b5a200 con 0x55ebf0c6d720 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc9339e1740 10 client.0 _ll_register_callbacks cb 0x55ebf0bc9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c73f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c73f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53600/0 (socket says 172.21.15.17:53600) 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1996613201 learned_addr learned my addr 172.21.15.17:0/1996613201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc929d9b640 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1996613201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 msgr2=0x55ebf0c76580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c76580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 msgr2=0x55ebf0c739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1996613201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92000bcd0 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c73f10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0042f0 tx=0x7fc91c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf0e92d24f81c2e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.554+0000 7fc911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc911ffb640 1 -- 172.21.15.17:0/1996613201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c004320 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc911ffb640 1 -- 172.21.15.17:0/1996613201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc91c004480 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebf08d4600 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc911ffb640 1 -- 172.21.15.17:0/1996613201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc91c00f420 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 10 client.16800 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ebf0bd2d70 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc911ffb640 1 -- 172.21.15.17:0/1996613201 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ebf08d4600 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc911ffb640 1 -- 172.21.15.17:0/1996613201 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc91c025020 con 0x55ebf0c76c20 2024-02-14T15:09:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 10 client.16800 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 20 client.16800 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0c869a0 0x55ebf0c88dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 4 client.16800 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.555+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebf09251b0 con 0x55ebf0c869a0 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.556+0000 7fc92a59c640 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0c869a0 0x55ebf0c88dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.556+0000 7fc911ffb640 10 client.16800 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.556+0000 7fc92a59c640 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0c869a0 0x55ebf0c88dc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4074b92c5cbddf1 server_cookie=6f238254cbf6e5d1 in_seq=0 out_seq=0 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.556+0000 7fc911ffb640 1 -- 172.21.15.17:0/1996613201 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ebf09251b0 con 0x55ebf0c869a0 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.556+0000 7fc911ffb640 10 client.16800 handle_command_reply: tid=0 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 client.16800 shutdown 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 msgr2=0x55ebf0c73f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c73f10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0042f0 tx=0x7fc91c004730 comp rx=0 tx=0).stop 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 shutdown_connections 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0c869a0 0x55ebf0c88dc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c73f10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1996613201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c76580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc911ffb640 0 client.16800 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 >> 172.21.15.17:0/1996613201 conn(0x55ebf0c67e50 msgr2=0x55ebf0c69ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 shutdown_connections 2024-02-14T15:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 1 -- 172.21.15.17:0/1996613201 wait complete. 2024-02-14T15:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.557+0000 7fc9339e1740 20 client.16800 trim_cache size 0 max 16384 2024-02-14T15:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 Processor -- start 2024-02-14T15:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 -- start start 2024-02-14T15:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c699b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0c6de80 con 0x55ebf0c76c20 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0c68e50 con 0x55ebf0c6e040 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc9339e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0c68be0 con 0x55ebf0c6d720 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc929d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c76580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53604/0 (socket says 172.21.15.17:53604) 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc929d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c76040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57590/0 (socket says 172.21.15.17:57590) 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1661391437 learned_addr learned my addr 172.21.15.17:0/1661391437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1661391437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 msgr2=0x55ebf0c699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1661391437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 msgr2=0x55ebf0c76040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c76040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 -- 172.21.15.17:0/1661391437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebf0b6f820 con 0x55ebf0c76c20 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92a59c640 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc929d9b640 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.571+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c76580 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc91c011850 tx=0x7fc91c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84d02cfaeb60a5d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc913fff640 1 -- 172.21.15.17:0/1661391437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c038070 con 0x55ebf0c76c20 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc913fff640 1 -- 172.21.15.17:0/1661391437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc91c019070 con 0x55ebf0c76c20 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc913fff640 1 -- 172.21.15.17:0/1661391437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c004320 con 0x55ebf0c76c20 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 -- 172.21.15.17:0/1661391437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 msgr2=0x55ebf0c76580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c76580 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc91c011850 tx=0x7fc91c011df0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 -- 172.21.15.17:0/1661391437 shutdown_connections 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c76c20 0x55ebf0c76580 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6e040 0x55ebf0c76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 --2- 172.21.15.17:0/1661391437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c6d720 0x55ebf0c699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 -- 172.21.15.17:0/1661391437 >> 172.21.15.17:0/1661391437 conn(0x55ebf0c67e50 msgr2=0x55ebf0c692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 -- 172.21.15.17:0/1661391437 shutdown_connections 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.572+0000 7fc9339e1740 1 -- 172.21.15.17:0/1661391437 wait complete. 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 -- start start 2024-02-14T15:09:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 Processor -- start 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6d720 0x55ebf0c7db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c7c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c76c20 0x55ebf0c7c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0c7cb20 con 0x55ebf0c6e040 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0c7cc90 con 0x55ebf0c6d720 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0c7ce50 con 0x55ebf0c76c20 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc9339e1740 10 client.0 _ll_register_callbacks cb 0x7fc92000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c7c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c7c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53616/0 (socket says 172.21.15.17:53616) 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 -- 172.21.15.17:0/528294615 learned_addr learned my addr 172.21.15.17:0/528294615 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc92a59c640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6d720 0x55ebf0c7db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc92ad9d640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c76c20 0x55ebf0c7c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 -- 172.21.15.17:0/528294615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c76c20 msgr2=0x55ebf0c7c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c76c20 0x55ebf0c7c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 -- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6d720 msgr2=0x55ebf0c7db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6d720 0x55ebf0c7db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc929d9b640 1 -- 172.21.15.17:0/528294615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1a4610 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.573+0000 7fc92a59c640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6d720 0x55ebf0c7db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc929d9b640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c7c0a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc920010590 tx=0x7fc9200054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a095d3cc16b34ee1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc911ffb640 1 -- 172.21.15.17:0/528294615 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920015040 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc911ffb640 1 -- 172.21.15.17:0/528294615 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fc920004020 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc920005bf0 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc911ffb640 1 -- 172.21.15.17:0/528294615 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9200041e0 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 10 client.16806 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ebf08e5920 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc911ffb640 1 -- 172.21.15.17:0/528294615 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc920005bf0 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc911ffb640 1 -- 172.21.15.17:0/528294615 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc92000d780 con 0x55ebf0c6e040 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 10 client.16806 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 20 client.16806 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0cde2d0 0x55ebf0ce06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 4 client.16806 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:09:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ebf0911550 con 0x55ebf0cde2d0 2024-02-14T15:09:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.574+0000 7fc92a59c640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0cde2d0 0x55ebf0ce06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.575+0000 7fc92a59c640 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0cde2d0 0x55ebf0ce06f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7c0ba3ffd3bd15 server_cookie=e132e2bf22c10158 in_seq=0 out_seq=0 2024-02-14T15:09:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.575+0000 7fc911ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc911ffb640 1 -- 172.21.15.17:0/528294615 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ebf0911550 con 0x55ebf0cde2d0 2024-02-14T15:09:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc911ffb640 10 client.16806 handle_command_reply: tid=0 2024-02-14T15:09:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 client.16806 shutdown 2024-02-14T15:09:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 msgr2=0x55ebf0c7c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c7c0a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc920010590 tx=0x7fc9200054f0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 shutdown_connections 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55ebf0cde2d0 0x55ebf0ce06f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ebf0c6e040 0x55ebf0c7c0a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebf0c76c20 0x55ebf0c7c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 --2- 172.21.15.17:0/528294615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ebf0c6d720 0x55ebf0c7db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc911ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 >> 172.21.15.17:0/528294615 conn(0x55ebf0c67e50 msgr2=0x55ebf0c66ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 shutdown_connections 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 1 -- 172.21.15.17:0/528294615 wait complete. 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.585+0000 7fc9339e1740 20 client.16806 trim_cache size 0 max 16384 2024-02-14T15:09:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:09:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:09:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:09:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:09:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9140a2220 msgr2=0x7fc9140a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9140a2220 0x7fc9140a46e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc91c013840 tx=0x7fc91c00aa90 comp rx=0 tx=0).stop 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c06b240 msgr2=0x7fc92c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c06b240 0x7fc92c19eed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc924005e30 tx=0x7fc924034830 comp rx=0 tx=0).stop 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 shutdown_connections 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9140a2220 0x7fc9140a46e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c072f20 0x7fc92c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc92c113040 0x7fc92c1a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 --2- 172.21.15.17:0/1912521210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc92c06b240 0x7fc92c19eed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 >> 172.21.15.17:0/1912521210 conn(0x7fc92c077220 msgr2=0x7fc92c070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 shutdown_connections 2024-02-14T15:09:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:42.586+0000 7fc930e15640 1 -- 172.21.15.17:0/1912521210 wait complete. 2024-02-14T15:09:42.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 616283 11.71 MB/sec execute 164 sec latency 17.127 ms 2024-02-14T15:09:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:42 smithi050 ceph-mon[36512]: pgmap v192: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-02-14T15:09:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:42 smithi076 ceph-mon[37057]: pgmap v192: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-02-14T15:09:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:42 smithi017 ceph-mon[32549]: pgmap v192: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-02-14T15:09:43.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 618577 11.72 MB/sec execute 165 sec latency 18.970 ms 2024-02-14T15:09:44.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 621283 11.73 MB/sec execute 166 sec latency 16.242 ms 2024-02-14T15:09:45.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:44 smithi050 ceph-mon[36512]: pgmap v193: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-14T15:09:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:44 smithi076 ceph-mon[37057]: pgmap v193: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-14T15:09:45.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:44 smithi017 ceph-mon[32549]: pgmap v193: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-14T15:09:45.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 622517 11.70 MB/sec execute 167 sec latency 69.982 ms 2024-02-14T15:09:46.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 624419 11.69 MB/sec execute 168 sec latency 244.816 ms 2024-02-14T15:09:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:46 smithi076 ceph-mon[37057]: pgmap v194: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-14T15:09:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:46 smithi017 ceph-mon[32549]: pgmap v194: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-14T15:09:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:46 smithi050 ceph-mon[36512]: pgmap v194: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-14T15:09:47.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 626264 11.68 MB/sec execute 169 sec latency 44.977 ms 2024-02-14T15:09:48.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 628889 11.71 MB/sec execute 170 sec latency 12.561 ms 2024-02-14T15:09:49.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:48 smithi076 ceph-mon[37057]: pgmap v195: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-02-14T15:09:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:48 smithi017 ceph-mon[32549]: pgmap v195: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-02-14T15:09:49.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:48 smithi050 ceph-mon[36512]: pgmap v195: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-02-14T15:09:49.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 630817 11.68 MB/sec execute 171 sec latency 37.079 ms 2024-02-14T15:09:50.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 632700 11.69 MB/sec execute 172 sec latency 65.754 ms 2024-02-14T15:09:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:50 smithi076 ceph-mon[37057]: pgmap v196: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-02-14T15:09:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:50 smithi017 ceph-mon[32549]: pgmap v196: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-02-14T15:09:51.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:50 smithi050 ceph-mon[36512]: pgmap v196: 129 pgs: 129 active+clean; 418 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-02-14T15:09:51.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 634556 11.66 MB/sec execute 173 sec latency 85.102 ms 2024-02-14T15:09:52.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 636522 11.66 MB/sec execute 174 sec latency 19.486 ms 2024-02-14T15:09:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:52 smithi076 ceph-mon[37057]: pgmap v197: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-02-14T15:09:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:52 smithi017 ceph-mon[32549]: pgmap v197: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-02-14T15:09:53.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:52 smithi050 ceph-mon[36512]: pgmap v197: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-02-14T15:09:53.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 639262 11.69 MB/sec execute 175 sec latency 15.392 ms 2024-02-14T15:09:54.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 641032 11.67 MB/sec execute 176 sec latency 15.049 ms 2024-02-14T15:09:55.194 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:54 smithi050 ceph-mon[36512]: pgmap v198: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-02-14T15:09:55.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:54 smithi076 ceph-mon[37057]: pgmap v198: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-02-14T15:09:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:54 smithi017 ceph-mon[32549]: pgmap v198: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-02-14T15:09:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/2933244114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100d40 msgr2=0x7fb8c010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/2933244114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100d40 0x7fb8c010cf00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009cd0 tx=0x7fb8b802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/2933244114 shutdown_connections 2024-02-14T15:09:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/2933244114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100d40 0x7fb8c010cf00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/2933244114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8c0100380 0x7fb8c0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/2933244114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c01062c0 0x7fb8c01066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/2933244114 >> 172.21.15.17:0/2933244114 conn(0x7fb8c0076ef0 msgr2=0x7fb8c0077300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/2933244114 shutdown_connections 2024-02-14T15:09:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.682+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/2933244114 wait complete. 2024-02-14T15:09:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 Processor -- start 2024-02-14T15:09:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 -- start start 2024-02-14T15:09:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100380 0x7fb8c0101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8c0100d40 0x7fb8c0102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c01062c0 0x7fb8c0102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c01a5c20 con 0x7fb8c0100380 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c01a5d90 con 0x7fb8c0100d40 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8c4aeb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c01a5f50 con 0x7fb8c01062c0 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100380 0x7fb8c0101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100380 0x7fb8c0101de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50608/0 (socket says 172.21.15.17:50608) 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8be59c640 1 -- 172.21.15.17:0/3797590558 learned_addr learned my addr 172.21.15.17:0/3797590558 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.683+0000 7fb8bed9d640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c01062c0 0x7fb8c0102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8c0100d40 0x7fb8c0102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8be59c640 1 -- 172.21.15.17:0/3797590558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c01062c0 msgr2=0x7fb8c0102860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c01062c0 0x7fb8c0102860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8be59c640 1 -- 172.21.15.17:0/3797590558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8c0100d40 msgr2=0x7fb8c0102320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8c0100d40 0x7fb8c0102320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8be59c640 1 -- 172.21.15.17:0/3797590558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8009900 con 0x7fb8c0100380 2024-02-14T15:09:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8c0100d40 0x7fb8c0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:09:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100380 0x7fb8c0101de0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000e9f0 tx=0x7fb8b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8b77fe640 1 -- 172.21.15.17:0/3797590558 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0009720 con 0x7fb8c0100380 2024-02-14T15:09:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c01a6230 con 0x7fb8c0100380 2024-02-14T15:09:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8b77fe640 1 -- 172.21.15.17:0/3797590558 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b000cd80 con 0x7fb8c0100380 2024-02-14T15:09:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8b77fe640 1 -- 172.21.15.17:0/3797590558 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0010690 con 0x7fb8c0100380 2024-02-14T15:09:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.684+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c01a66f0 con 0x7fb8c0100380 2024-02-14T15:09:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.686+0000 7fb8b77fe640 1 -- 172.21.15.17:0/3797590558 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb8b00046c0 con 0x7fb8c0100380 2024-02-14T15:09:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 Processor -- start 2024-02-14T15:09:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8b77fe640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb8ac0a2220 0x7fb8ac0a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8b77fe640 1 -- 172.21.15.17:0/3797590558 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb8b0014070 con 0x7fb8c0100380 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8bed9d640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb8ac0a2220 0x7fb8ac0a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 -- start start 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x5598590796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559858fde9e0 con 0x559859070720 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x559858fdbf10 con 0x559859071040 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8c778e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559858fd76f0 con 0x559859079c20 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859070b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50616/0 (socket says 172.21.15.17:50616) 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8be59c640 1 -- 172.21.15.17:0/3707004942 learned_addr learned my addr 172.21.15.17:0/3707004942 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8bed9d640 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.687+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x5598590796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8bed9d640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb8ac0a2220 0x7fb8ac0a46e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8005cc0 tx=0x7fb8b8005c50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8be59c640 1 -- 172.21.15.17:0/3707004942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 msgr2=0x55985907c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8be59c640 1 -- 172.21.15.17:0/3707004942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 msgr2=0x5598590796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x5598590796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8be59c640 1 -- 172.21.15.17:0/3707004942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559858ce8920 con 0x559859070720 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x5598590796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859070b00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00a30e0 tx=0x7fb8b009b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c42c926585e8da server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8b4ff9640 1 -- 172.21.15.17:0/3707004942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b009efc0 con 0x559859070720 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8b4ff9640 1 -- 172.21.15.17:0/3707004942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b00a3610 con 0x559859070720 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.688+0000 7fb8b4ff9640 1 -- 172.21.15.17:0/3707004942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00a3380 con 0x559859070720 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 -- 172.21.15.17:0/3707004942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 msgr2=0x559859070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859070b00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00a30e0 tx=0x7fb8b009b570 comp rx=0 tx=0).stop 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 -- 172.21.15.17:0/3707004942 shutdown_connections 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x5598590796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3707004942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859070b00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 -- 172.21.15.17:0/3707004942 >> 172.21.15.17:0/3707004942 conn(0x55985906ae50 msgr2=0x55985906b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 -- 172.21.15.17:0/3707004942 shutdown_connections 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 -- 172.21.15.17:0/3707004942 wait complete. 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 -- start start 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.689+0000 7fb8c778e740 1 Processor -- start 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x5598590795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x559859076960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x559859076f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559858f73fa0 con 0x559859070720 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x559858cab250 con 0x559859079c20 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559858f5d200 con 0x559859071040 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 10 client.0 _ll_register_callbacks cb 0x559858fccb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x5598590795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x559859076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x5598590795d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50628/0 (socket says 172.21.15.17:50628) 2024-02-14T15:09:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8be59c640 1 -- 172.21.15.17:0/3358357915 learned_addr learned my addr 172.21.15.17:0/3358357915 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8bed9d640 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x559859076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8b7fff640 1 -- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 msgr2=0x559859076f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x559859076f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8b7fff640 1 -- 172.21.15.17:0/3358357915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 msgr2=0x5598590795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x5598590795d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8b7fff640 1 -- 172.21.15.17:0/3358357915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c01a66f0 con 0x559859071040 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x5598590795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x559859076960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5598590701f0 tx=0x7fb8a8012900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=134f0e4f092d270 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/3358357915 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8004220 con 0x559859071040 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/3358357915 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb8a8004380 con 0x559859071040 2024-02-14T15:09:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.690+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559858cd7600 con 0x559859071040 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.693+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/3358357915 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a8010640 con 0x559859071040 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.693+0000 7fb8c778e740 10 client.26341 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.693+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559858fd5d70 con 0x559859071040 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/3358357915 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559858cd7600 con 0x559859071040 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/3358357915 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8a8003b90 con 0x559859071040 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8c778e740 10 client.26341 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8c778e740 10 client.26341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8c778e740 20 client.26341 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559859089ac0 0x55985908bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8c778e740 4 client.26341 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559858d281b0 con 0x559859089ac0 2024-02-14T15:09:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559859089ac0 0x55985908bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8be59c640 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559859089ac0 0x55985908bee0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1250086a3f9bd46 server_cookie=4d88a015d36ade7d in_seq=0 out_seq=0 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.694+0000 7fb8a5ffb640 10 client.26341 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/3358357915 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559858d281b0 con 0x559859089ac0 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8a5ffb640 10 client.26341 handle_command_reply: tid=0 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 client.26341 shutdown 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 msgr2=0x559859076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x559859076960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5598590701f0 tx=0x7fb8a8012900 comp rx=0 tx=0).stop 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 shutdown_connections 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559859089ac0 0x55985908bee0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x559859076f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x559859076960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8a5ffb640 0 client.26341 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 --2- 172.21.15.17:0/3358357915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x5598590795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 >> 172.21.15.17:0/3358357915 conn(0x55985906ae50 msgr2=0x55985906cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 shutdown_connections 2024-02-14T15:09:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 1 -- 172.21.15.17:0/3358357915 wait complete. 2024-02-14T15:09:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.695+0000 7fb8c778e740 20 client.26341 trim_cache size 0 max 16384 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 Processor -- start 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 -- start start 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859079090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x5598590795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x5598590761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559859091110 con 0x559859070720 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x559859072570 con 0x559859079c20 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8c778e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598590692d0 con 0x559859071040 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859079090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859079090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50640/0 (socket says 172.21.15.17:50640) 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8be59c640 1 -- 172.21.15.17:0/1881588836 learned_addr learned my addr 172.21.15.17:0/1881588836 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x5598590795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.710+0000 7fb8bed9d640 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x5598590761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8be59c640 1 -- 172.21.15.17:0/1881588836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 msgr2=0x5598590795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8be59c640 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x5598590795d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8be59c640 1 -- 172.21.15.17:0/1881588836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 msgr2=0x5598590761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8be59c640 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x5598590761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8be59c640 1 -- 172.21.15.17:0/1881588836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559858f72820 con 0x559859070720 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x5598590795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8be59c640 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859079090 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb8b009b720 tx=0x7fb8b000cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f766f1ceb182a2bf server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8a7fff640 1 -- 172.21.15.17:0/1881588836 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00a8070 con 0x559859070720 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8a7fff640 1 -- 172.21.15.17:0/1881588836 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b0099170 con 0x559859070720 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.711+0000 7fb8a7fff640 1 -- 172.21.15.17:0/1881588836 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0097660 con 0x559859070720 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- 172.21.15.17:0/1881588836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 msgr2=0x559859079090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859079090 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb8b009b720 tx=0x7fb8b000cfa0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- 172.21.15.17:0/1881588836 shutdown_connections 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859079c20 0x5598590761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859071040 0x5598590795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 --2- 172.21.15.17:0/1881588836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x559859079090 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- 172.21.15.17:0/1881588836 >> 172.21.15.17:0/1881588836 conn(0x55985906ae50 msgr2=0x559859075230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- 172.21.15.17:0/1881588836 shutdown_connections 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- 172.21.15.17:0/1881588836 wait complete. 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- start start 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 Processor -- start 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x55985907eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x55985907eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598590df180 con 0x559859070720 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598590df2f0 con 0x559859071040 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.712+0000 7fb8c778e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598590df4b0 con 0x559859079c20 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8c778e740 10 client.0 _ll_register_callbacks cb 0x7fb8b000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x55985907eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8bed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x55985907eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50642/0 (socket says 172.21.15.17:50642) 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8bed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:46544/0 (socket says 172.21.15.17:46544) 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 -- 172.21.15.17:0/328880453 learned_addr learned my addr 172.21.15.17:0/328880453 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x55985907eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 -- 172.21.15.17:0/328880453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 msgr2=0x55985907f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 -- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 msgr2=0x55985907eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x55985907eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 -- 172.21.15.17:0/328880453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559858fd5d70 con 0x559859070720 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x55985907eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8be59c640 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x55985907eab0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c6a0 tx=0x7fb8b0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aec54ca97ced8b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/328880453 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00a8070 con 0x559859070720 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/328880453 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fb8b00ce6a0 con 0x559859070720 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8b00a3610 con 0x559859070720 2024-02-14T15:09:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/328880453 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8b00ce430 con 0x559859070720 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8c778e740 10 client.16824 fetch_fsmap learned FSMap version 30 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.713+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb8b00b7040 con 0x559859070720 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/328880453 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b00a3610 con 0x559859070720 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/328880453 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8b001f060 con 0x559859070720 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8c778e740 10 client.16824 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8c778e740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8c778e740 20 client.16824 populate_metadata read hostname 'smithi017' 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8c778e740 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5598590e0800 0x5598590e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8c778e740 4 client.16824 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559858d14550 con 0x5598590e0800 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5598590e0800 0x5598590e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8a5ffb640 10 client.16824 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.714+0000 7fb8b7fff640 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5598590e0800 0x5598590e2bf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38158f5bbe98416c server_cookie=701d8e6f086fb455 in_seq=0 out_seq=0 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.724+0000 7fb8a5ffb640 1 -- 172.21.15.17:0/328880453 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559858d14550 con 0x5598590e0800 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.724+0000 7fb8a5ffb640 10 client.16824 handle_command_reply: tid=0 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.724+0000 7fb8c778e740 1 client.16824 shutdown 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 msgr2=0x55985907eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x55985907eab0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c6a0 tx=0x7fb8b0005760 comp rx=0 tx=0).stop 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 shutdown_connections 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5598590e0800 0x5598590e2bf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559859079c20 0x55985907f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8a5ffb640 0 client.16824 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559859071040 0x55985907eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 --2- 172.21.15.17:0/328880453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559859070720 0x55985907eab0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 >> 172.21.15.17:0/328880453 conn(0x55985906ae50 msgr2=0x5598590761f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 shutdown_connections 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 1 -- 172.21.15.17:0/328880453 wait complete. 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c778e740 20 client.16824 trim_cache size 0 max 16384 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:09:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:09:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb8ac0a2220 msgr2=0x7fb8ac0a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb8ac0a2220 0x7fb8ac0a46e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8005cc0 tx=0x7fb8b8005c50 comp rx=0 tx=0).stop 2024-02-14T15:09:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100380 msgr2=0x7fb8c0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:09:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100380 0x7fb8c0101de0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000e9f0 tx=0x7fb8b000edf0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 shutdown_connections 2024-02-14T15:09:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fb8ac0a2220 0x7fb8ac0a46e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c01062c0 0x7fb8c0102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8c0100d40 0x7fb8c0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 --2- 172.21.15.17:0/3797590558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8c0100380 0x7fb8c0101de0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:09:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.725+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 >> 172.21.15.17:0/3797590558 conn(0x7fb8c0076ef0 msgr2=0x7fb8c00ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:09:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.726+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 shutdown_connections 2024-02-14T15:09:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:09:55.726+0000 7fb8c4aeb640 1 -- 172.21.15.17:0/3797590558 wait complete. 2024-02-14T15:09:55.872 INFO:tasks.workunit.client.0.smithi017.stdout: 1 643245 11.68 MB/sec execute 177 sec latency 77.226 ms 2024-02-14T15:09:56.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 645201 11.65 MB/sec execute 178 sec latency 42.241 ms 2024-02-14T15:09:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:56 smithi076 ceph-mon[37057]: pgmap v199: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-02-14T15:09:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:56 smithi017 ceph-mon[32549]: pgmap v199: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-02-14T15:09:57.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:56 smithi050 ceph-mon[36512]: pgmap v199: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-02-14T15:09:57.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 647066 11.66 MB/sec execute 179 sec latency 42.176 ms 2024-02-14T15:09:58.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 649617 11.67 MB/sec execute 180 sec latency 15.181 ms 2024-02-14T15:09:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:09:58 smithi076 ceph-mon[37057]: pgmap v200: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.7 MiB/s wr, 299 op/s 2024-02-14T15:09:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:09:58 smithi017 ceph-mon[32549]: pgmap v200: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.7 MiB/s wr, 299 op/s 2024-02-14T15:09:59.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:09:58 smithi050 ceph-mon[36512]: pgmap v200: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.7 MiB/s wr, 299 op/s 2024-02-14T15:09:59.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 651397 11.66 MB/sec execute 181 sec latency 16.789 ms 2024-02-14T15:10:00.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32545]: 2024-02-14T15:09:59.999+0000 7ff3e6861700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-14T15:10:00.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32545]: 2024-02-14T15:09:59.999+0000 7ff3e6861700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-14T15:10:00.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32545]: 2024-02-14T15:09:59.999+0000 7ff3e6861700 -1 log_channel(cluster) log [ERR] : mds.c(mds.2): Metadata damage detected 2024-02-14T15:10:00.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 653807 11.67 MB/sec execute 182 sec latency 71.733 ms 2024-02-14T15:10:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:00 smithi076 ceph-mon[37057]: pgmap v201: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 200 op/s 2024-02-14T15:10:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:00 smithi076 ceph-mon[37057]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-14T15:10:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:00 smithi076 ceph-mon[37057]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-14T15:10:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:00 smithi076 ceph-mon[37057]: mds.c(mds.2): Metadata damage detected 2024-02-14T15:10:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:00 smithi017 ceph-mon[32549]: pgmap v201: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 200 op/s 2024-02-14T15:10:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:00 smithi017 ceph-mon[32549]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-14T15:10:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:00 smithi017 ceph-mon[32549]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-14T15:10:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:00 smithi017 ceph-mon[32549]: mds.c(mds.2): Metadata damage detected 2024-02-14T15:10:01.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:00 smithi050 ceph-mon[36512]: pgmap v201: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 200 op/s 2024-02-14T15:10:01.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:00 smithi050 ceph-mon[36512]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-02-14T15:10:01.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:00 smithi050 ceph-mon[36512]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-02-14T15:10:01.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:00 smithi050 ceph-mon[36512]: mds.c(mds.2): Metadata damage detected 2024-02-14T15:10:01.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 655108 11.64 MB/sec execute 183 sec latency 62.867 ms 2024-02-14T15:10:02.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 657448 11.65 MB/sec execute 184 sec latency 25.878 ms 2024-02-14T15:10:03.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:02 smithi076 ceph-mon[37057]: pgmap v202: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 312 op/s 2024-02-14T15:10:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:02 smithi017 ceph-mon[32549]: pgmap v202: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 312 op/s 2024-02-14T15:10:03.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:02 smithi050 ceph-mon[36512]: pgmap v202: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 312 op/s 2024-02-14T15:10:03.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 659902 11.66 MB/sec execute 185 sec latency 24.576 ms 2024-02-14T15:10:04.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 661793 11.64 MB/sec execute 186 sec latency 14.536 ms 2024-02-14T15:10:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:04 smithi076 ceph-mon[37057]: pgmap v203: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-02-14T15:10:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:04 smithi017 ceph-mon[32549]: pgmap v203: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-02-14T15:10:05.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:04 smithi050 ceph-mon[36512]: pgmap v203: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-02-14T15:10:05.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 664329 11.67 MB/sec execute 187 sec latency 13.247 ms 2024-02-14T15:10:06.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 665592 11.63 MB/sec execute 188 sec latency 47.034 ms 2024-02-14T15:10:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:06 smithi076 ceph-mon[37057]: pgmap v204: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-02-14T15:10:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:06 smithi017 ceph-mon[32549]: pgmap v204: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-02-14T15:10:07.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:06 smithi050 ceph-mon[36512]: pgmap v204: 129 pgs: 129 active+clean; 408 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 214 op/s 2024-02-14T15:10:07.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 667981 11.65 MB/sec execute 189 sec latency 13.967 ms 2024-02-14T15:10:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 -- 172.21.15.17:0/2363139568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100d40 msgr2=0x7fd19410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 --2- 172.21.15.17:0/2363139568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100d40 0x7fd19410cf00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009cd0 tx=0x7fd18c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 -- 172.21.15.17:0/2363139568 shutdown_connections 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 --2- 172.21.15.17:0/2363139568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100d40 0x7fd19410cf00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 --2- 172.21.15.17:0/2363139568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd194100380 0x7fd194100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 --2- 172.21.15.17:0/2363139568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1941062c0 0x7fd1941066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 -- 172.21.15.17:0/2363139568 >> 172.21.15.17:0/2363139568 conn(0x7fd194076ef0 msgr2=0x7fd194077300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 -- 172.21.15.17:0/2363139568 shutdown_connections 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.816+0000 7fd19af3e640 1 -- 172.21.15.17:0/2363139568 wait complete. 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 Processor -- start 2024-02-14T15:10:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 -- start start 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100380 0x7fd19419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd194100d40 0x7fd19419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1941062c0 0x7fd194199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd194199ed0 con 0x7fd194100380 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19419a040 con 0x7fd1941062c0 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd19af3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19419a200 con 0x7fd194100d40 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd198cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100380 0x7fd19419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd198cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100380 0x7fd19419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50180/0 (socket says 172.21.15.17:50180) 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd198cda640 1 -- 172.21.15.17:0/293861030 learned_addr learned my addr 172.21.15.17:0/293861030 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd1994db640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1941062c0 0x7fd194199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.817+0000 7fd193fff640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd194100d40 0x7fd19419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd198cda640 1 -- 172.21.15.17:0/293861030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd194100d40 msgr2=0x7fd19419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd198cda640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd194100d40 0x7fd19419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd198cda640 1 -- 172.21.15.17:0/293861030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1941062c0 msgr2=0x7fd194199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd198cda640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1941062c0 0x7fd194199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd198cda640 1 -- 172.21.15.17:0/293861030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c009900 con 0x7fd194100380 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd198cda640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100380 0x7fd19419f6c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9f0 tx=0x7fd18800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd191ffb640 1 -- 172.21.15.17:0/293861030 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188009700 con 0x7fd194100380 2024-02-14T15:10:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd191ffb640 1 -- 172.21.15.17:0/293861030 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd18800cd80 con 0x7fd194100380 2024-02-14T15:10:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19419a4e0 con 0x7fd194100380 2024-02-14T15:10:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd191ffb640 1 -- 172.21.15.17:0/293861030 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188018920 con 0x7fd194100380 2024-02-14T15:10:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.818+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1941a66f0 con 0x7fd194100380 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.820+0000 7fd191ffb640 1 -- 172.21.15.17:0/293861030 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd188018a80 con 0x7fd194100380 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 Processor -- start 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 -- start start 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd191ffb640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd1840a2220 0x7fd1840a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a889cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a880a9e0 con 0x55e0a889c720 2024-02-14T15:10:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a8807f10 con 0x55e0a88a5c20 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a88036f0 con 0x55e0a889d040 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd1994db640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd1840a2220 0x7fd1840a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd191ffb640 1 -- 172.21.15.17:0/293861030 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd188014070 con 0x7fd194100380 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:43114/0 (socket says 172.21.15.17:43114) 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd198cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a889cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 -- 172.21.15.17:0/1389104595 learned_addr learned my addr 172.21.15.17:0/1389104595 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd1994db640 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 -- 172.21.15.17:0/1389104595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 msgr2=0x55e0a88a8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 -- 172.21.15.17:0/1389104595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 msgr2=0x55e0a889cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a889cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd193fff640 1 -- 172.21.15.17:0/1389104595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0a8514920 con 0x55e0a889d040 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.821+0000 7fd198cda640 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a889cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd1994db640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd1840a2220 0x7fd1840a46e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005cc0 tx=0x7fd18c0325d0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd193fff640 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a56e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd180009c60 tx=0x7fd180011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9749f21a927e5c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd17effd640 1 -- 172.21.15.17:0/1389104595 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180004230 con 0x55e0a889d040 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd17effd640 1 -- 172.21.15.17:0/1389104595 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd180004390 con 0x55e0a889d040 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd17effd640 1 -- 172.21.15.17:0/1389104595 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180019780 con 0x55e0a889d040 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1389104595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 msgr2=0x55e0a88a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a56e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd180009c60 tx=0x7fd180011810 comp rx=0 tx=0).stop 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1389104595 shutdown_connections 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a56e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1389104595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a889cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.822+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1389104595 >> 172.21.15.17:0/1389104595 conn(0x55e0a8896e50 msgr2=0x55e0a8897230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1389104595 shutdown_connections 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1389104595 wait complete. 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 -- start start 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 Processor -- start 2024-02-14T15:10:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889d040 0x55e0a88a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a88a5c20 0x55e0a88a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a879ffa0 con 0x55e0a889c720 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a84d7250 con 0x55e0a889d040 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a8789200 con 0x55e0a88a5c20 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd19dbe5740 10 client.0 _ll_register_callbacks cb 0x55e0a87f8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd198cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd198cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50208/0 (socket says 172.21.15.17:50208) 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd198cda640 1 -- 172.21.15.17:0/3722654991 learned_addr learned my addr 172.21.15.17:0/3722654991 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd1994db640 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a88a5c20 0x55e0a88a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.823+0000 7fd193fff640 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889d040 0x55e0a88a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd198cda640 1 -- 172.21.15.17:0/3722654991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a88a5c20 msgr2=0x55e0a88a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd198cda640 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a88a5c20 0x55e0a88a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd198cda640 1 -- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889d040 msgr2=0x55e0a88a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd198cda640 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889d040 0x55e0a88a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd198cda640 1 -- 172.21.15.17:0/3722654991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd180009890 con 0x55e0a889c720 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd198cda640 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88a54e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd188097330 tx=0x7fd188064650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec2dcaefb4687c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd173fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd173fff640 1 -- 172.21.15.17:0/3722654991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18809ef70 con 0x55e0a889c720 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd173fff640 1 -- 172.21.15.17:0/3722654991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd1880a35c0 con 0x55e0a889c720 2024-02-14T15:10:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0a8503600 con 0x55e0a889c720 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd173fff640 1 -- 172.21.15.17:0/3722654991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd188097540 con 0x55e0a889c720 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd19dbe5740 10 client.16836 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e0a8801d70 con 0x55e0a889c720 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd173fff640 1 -- 172.21.15.17:0/3722654991 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e0a8503600 con 0x55e0a889c720 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd173fff640 1 -- 172.21.15.17:0/3722654991 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd188063a10 con 0x55e0a889c720 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd19dbe5740 10 client.16836 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.824+0000 7fd19dbe5740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.825+0000 7fd19dbe5740 20 client.16836 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.825+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a88b59b0 0x55e0a88b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.825+0000 7fd19dbe5740 4 client.16836 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.825+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0a85541b0 con 0x55e0a88b59b0 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.825+0000 7fd193fff640 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a88b59b0 0x55e0a88b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.825+0000 7fd173fff640 10 client.16836 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.825+0000 7fd193fff640 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a88b59b0 0x55e0a88b7dc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac20bf34836d50d server_cookie=f54a37d4e209f050 in_seq=0 out_seq=0 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd173fff640 1 -- 172.21.15.17:0/3722654991 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0a85541b0 con 0x55e0a88b59b0 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd173fff640 10 client.16836 handle_command_reply: tid=0 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 client.16836 shutdown 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 msgr2=0x55e0a88a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88a54e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd188097330 tx=0x7fd188064650 comp rx=0 tx=0).stop 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 shutdown_connections 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a88b59b0 0x55e0a88b7dc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a88a5c20 0x55e0a88a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889d040 0x55e0a88a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd173fff640 0 client.16836 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3722654991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88a54e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 >> 172.21.15.17:0/3722654991 conn(0x55e0a8896e50 msgr2=0x55e0a8898ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 shutdown_connections 2024-02-14T15:10:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3722654991 wait complete. 2024-02-14T15:10:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.826+0000 7fd19dbe5740 20 client.16836 trim_cache size 0 max 16384 2024-02-14T15:10:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 Processor -- start 2024-02-14T15:10:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 -- start start 2024-02-14T15:10:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a88a0e70 con 0x55e0a889c720 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a8896640 con 0x55e0a88a5c20 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a88b1c90 con 0x55e0a889d040 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd198cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd198cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50216/0 (socket says 172.21.15.17:50216) 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd198cda640 1 -- 172.21.15.17:0/1205648194 learned_addr learned my addr 172.21.15.17:0/1205648194 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd1994db640 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.840+0000 7fd193fff640 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd198cda640 1 -- 172.21.15.17:0/1205648194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 msgr2=0x55e0a88a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd198cda640 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd198cda640 1 -- 172.21.15.17:0/1205648194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 msgr2=0x55e0a88a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd198cda640 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd198cda640 1 -- 172.21.15.17:0/1205648194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0a879e820 con 0x55e0a889c720 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd198cda640 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88b7da0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9c0 tx=0x7fd18809b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c209bd6392be00c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd17e7fc640 1 -- 172.21.15.17:0/1205648194 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880a35c0 con 0x55e0a889c720 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd17e7fc640 1 -- 172.21.15.17:0/1205648194 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd18809ef70 con 0x55e0a889c720 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.841+0000 7fd17e7fc640 1 -- 172.21.15.17:0/1205648194 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880ce3f0 con 0x55e0a889c720 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1205648194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 msgr2=0x55e0a88b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88b7da0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9c0 tx=0x7fd18809b050 comp rx=0 tx=0).stop 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1205648194 shutdown_connections 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a88a5c20 0x55e0a88a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/1205648194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a889c720 0x55e0a88b7da0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1205648194 >> 172.21.15.17:0/1205648194 conn(0x55e0a8896e50 msgr2=0x55e0a88b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1205648194 shutdown_connections 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- 172.21.15.17:0/1205648194 wait complete. 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- start start 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 Processor -- start 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889c720 0x55e0a88aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a88a5c20 0x55e0a8904ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a8905430 con 0x55e0a88a5c20 2024-02-14T15:10:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a89055a0 con 0x55e0a889c720 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.842+0000 7fd19dbe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a8905760 con 0x55e0a889d040 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd19dbe5740 10 client.0 _ll_register_callbacks cb 0x7fd180009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd1994db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a88a5c20 0x55e0a8904ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd198cda640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889c720 0x55e0a88aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:43144/0 (socket says 172.21.15.17:43144) 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd1994db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a88a5c20 0x55e0a8904ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50230/0 (socket says 172.21.15.17:50230) 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 -- 172.21.15.17:0/3275292047 learned_addr learned my addr 172.21.15.17:0/3275292047 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 -- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889c720 msgr2=0x55e0a88aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889c720 0x55e0a88aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 -- 172.21.15.17:0/3275292047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a88a5c20 msgr2=0x55e0a8904ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a88a5c20 0x55e0a8904ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 -- 172.21.15.17:0/3275292047 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0a8801d70 con 0x55e0a889d040 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd198cda640 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889c720 0x55e0a88aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd1994db640 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a88a5c20 0x55e0a8904ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd193fff640 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88ab2e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd180005fd0 tx=0x7fd180009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc02b7defce6c97 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd173fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd173fff640 1 -- 172.21.15.17:0/3275292047 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180005250 con 0x55e0a889d040 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd173fff640 1 -- 172.21.15.17:0/3275292047 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fd1800053b0 con 0x55e0a889d040 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.843+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd180004390 con 0x55e0a889d040 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd173fff640 1 -- 172.21.15.17:0/3275292047 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd180005570 con 0x55e0a889d040 2024-02-14T15:10:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 10 client.26365 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e0a87c4720 con 0x55e0a889d040 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd173fff640 1 -- 172.21.15.17:0/3275292047 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd180004390 con 0x55e0a889d040 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd173fff640 1 -- 172.21.15.17:0/3275292047 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd180003950 con 0x55e0a889d040 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 10 client.26365 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 10 client.26365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 20 client.26365 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a890cf40 0x55e0a890f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 4 client.26365 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0a8540550 con 0x55e0a890cf40 2024-02-14T15:10:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd198cda640 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a890cf40 0x55e0a890f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd173fff640 10 client.26365 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.844+0000 7fd198cda640 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a890cf40 0x55e0a890f360 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd67c286d8c6141 server_cookie=db1ec22d94451c8d in_seq=0 out_seq=0 2024-02-14T15:10:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd173fff640 1 -- 172.21.15.17:0/3275292047 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0a8540550 con 0x55e0a890cf40 2024-02-14T15:10:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd173fff640 10 client.26365 handle_command_reply: tid=0 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 client.26365 shutdown 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 msgr2=0x55e0a88ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88ab2e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd180005fd0 tx=0x7fd180009330 comp rx=0 tx=0).stop 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 shutdown_connections 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e0a890cf40 0x55e0a890f360 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e0a88a5c20 0x55e0a8904ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e0a889d040 0x55e0a88ab2e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd173fff640 0 client.26365 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 --2- 172.21.15.17:0/3275292047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0a889c720 0x55e0a88aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 >> 172.21.15.17:0/3275292047 conn(0x55e0a8896e50 msgr2=0x55e0a88a1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 shutdown_connections 2024-02-14T15:10:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 1 -- 172.21.15.17:0/3275292047 wait complete. 2024-02-14T15:10:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.854+0000 7fd19dbe5740 20 client.26365 trim_cache size 0 max 16384 2024-02-14T15:10:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:10:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:10:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:10:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:10:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd1840a2220 msgr2=0x7fd1840a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd1840a2220 0x7fd1840a46e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005cc0 tx=0x7fd18c0325d0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100380 msgr2=0x7fd19419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100380 0x7fd19419f6c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9f0 tx=0x7fd18800eeb0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 shutdown_connections 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd1840a2220 0x7fd1840a46e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1941062c0 0x7fd194199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd194100d40 0x7fd19419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 --2- 172.21.15.17:0/293861030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd194100380 0x7fd19419f6c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 >> 172.21.15.17:0/293861030 conn(0x7fd194076ef0 msgr2=0x7fd194069840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 shutdown_connections 2024-02-14T15:10:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:08.855+0000 7fd19af3e640 1 -- 172.21.15.17:0/293861030 wait complete. 2024-02-14T15:10:08.873 INFO:tasks.workunit.client.0.smithi017.stdout: 1 669271 11.61 MB/sec execute 190 sec latency 286.221 ms 2024-02-14T15:10:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:08 smithi076 ceph-mon[37057]: pgmap v205: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 307 op/s 2024-02-14T15:10:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:08 smithi017 ceph-mon[32549]: pgmap v205: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 307 op/s 2024-02-14T15:10:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:08 smithi050 ceph-mon[36512]: pgmap v205: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 307 op/s 2024-02-14T15:10:09.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 671443 11.62 MB/sec execute 191 sec latency 314.687 ms 2024-02-14T15:10:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:09 smithi076 ceph-mon[37057]: osdmap e96: 12 total, 12 up, 12 in 2024-02-14T15:10:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:09 smithi017 ceph-mon[32549]: osdmap e96: 12 total, 12 up, 12 in 2024-02-14T15:10:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:09 smithi050 ceph-mon[36512]: osdmap e96: 12 total, 12 up, 12 in 2024-02-14T15:10:10.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 673653 11.60 MB/sec execute 192 sec latency 32.255 ms 2024-02-14T15:10:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:10 smithi076 ceph-mon[37057]: pgmap v207: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-02-14T15:10:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:10 smithi017 ceph-mon[32549]: pgmap v207: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-02-14T15:10:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:10 smithi050 ceph-mon[36512]: pgmap v207: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-02-14T15:10:11.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 675327 11.60 MB/sec execute 193 sec latency 45.976 ms 2024-02-14T15:10:12.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 677940 11.61 MB/sec execute 194 sec latency 15.042 ms 2024-02-14T15:10:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:12 smithi076 ceph-mon[37057]: pgmap v208: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:12 smithi017 ceph-mon[32549]: pgmap v208: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:13.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:12 smithi050 ceph-mon[36512]: pgmap v208: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:13.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 679368 11.59 MB/sec execute 195 sec latency 64.278 ms 2024-02-14T15:10:14.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:13 smithi076 ceph-mon[37057]: pgmap v209: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:14.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:13 smithi017 ceph-mon[32549]: pgmap v209: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:14.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:13 smithi050 ceph-mon[36512]: pgmap v209: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:14.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 681962 11.61 MB/sec execute 196 sec latency 38.219 ms 2024-02-14T15:10:15.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 683561 11.59 MB/sec execute 197 sec latency 45.346 ms 2024-02-14T15:10:16.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 685648 11.60 MB/sec execute 198 sec latency 98.738 ms 2024-02-14T15:10:16.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:16 smithi076 ceph-mon[37057]: pgmap v210: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:16.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:16 smithi076 ceph-mon[37057]: osdmap e97: 12 total, 12 up, 12 in 2024-02-14T15:10:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:16 smithi017 ceph-mon[32549]: pgmap v210: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:16 smithi017 ceph-mon[32549]: osdmap e97: 12 total, 12 up, 12 in 2024-02-14T15:10:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:16 smithi050 ceph-mon[36512]: pgmap v210: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-02-14T15:10:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:16 smithi050 ceph-mon[36512]: osdmap e97: 12 total, 12 up, 12 in 2024-02-14T15:10:17.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 687649 11.57 MB/sec execute 199 sec latency 32.361 ms 2024-02-14T15:10:18.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 689761 11.58 MB/sec execute 200 sec latency 19.920 ms 2024-02-14T15:10:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:18 smithi076 ceph-mon[37057]: pgmap v212: 129 pgs: 129 active+clean; 430 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 295 op/s 2024-02-14T15:10:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:18 smithi017 ceph-mon[32549]: pgmap v212: 129 pgs: 129 active+clean; 430 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 295 op/s 2024-02-14T15:10:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:18 smithi050 ceph-mon[36512]: pgmap v212: 129 pgs: 129 active+clean; 430 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 295 op/s 2024-02-14T15:10:19.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 692576 11.61 MB/sec execute 201 sec latency 19.953 ms 2024-02-14T15:10:20.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 694195 11.59 MB/sec execute 202 sec latency 15.100 ms 2024-02-14T15:10:20.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:20 smithi076 ceph-mon[37057]: pgmap v213: 129 pgs: 129 active+clean; 430 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 240 op/s 2024-02-14T15:10:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:20 smithi017 ceph-mon[32549]: pgmap v213: 129 pgs: 129 active+clean; 430 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 240 op/s 2024-02-14T15:10:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:20 smithi050 ceph-mon[36512]: pgmap v213: 129 pgs: 129 active+clean; 430 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 240 op/s 2024-02-14T15:10:21.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 696408 11.60 MB/sec execute 203 sec latency 59.709 ms 2024-02-14T15:10:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.948+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/636709966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0101e10 msgr2=0x7f5aa0105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.948+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/636709966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0101e10 0x7f5aa0105e50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009930 tx=0x7f5a8c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:10:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.948+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/636709966 shutdown_connections 2024-02-14T15:10:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.948+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/636709966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0101e10 0x7f5aa0105e50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.948+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/636709966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa0106530 0x7f5aa010e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.948+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/636709966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0101460 0x7f5aa0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.948+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/636709966 >> 172.21.15.17:0/636709966 conn(0x7f5aa00fccc0 msgr2=0x7f5aa00ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/636709966 shutdown_connections 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/636709966 wait complete. 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 Processor -- start 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 -- start start 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa0101460 0x7f5aa019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0106530 0x7f5aa019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0197750 0x7f5aa0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0198220 con 0x7f5aa0106530 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0198390 con 0x7f5aa0101460 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5aa6c2e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0198550 con 0x7f5aa0197750 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.949+0000 7f5a9ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0106530 0x7f5aa019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0106530 0x7f5aa019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35788/0 (socket says 172.21.15.17:35788) 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 -- 172.21.15.17:0/813528014 learned_addr learned my addr 172.21.15.17:0/813528014 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5aa51cb640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0197750 0x7f5aa0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa0101460 0x7f5aa019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 -- 172.21.15.17:0/813528014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0197750 msgr2=0x7f5aa0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0197750 0x7f5aa0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 -- 172.21.15.17:0/813528014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa0101460 msgr2=0x7f5aa019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa0101460 0x7f5aa019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 -- 172.21.15.17:0/813528014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c009560 con 0x7f5aa0106530 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0106530 0x7f5aa019db80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009230 tx=0x7f5a8c004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9dffb640 1 -- 172.21.15.17:0/813528014 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0043d0 con 0x7f5aa0106530 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9dffb640 1 -- 172.21.15.17:0/813528014 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c0052d0 con 0x7f5aa0106530 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa01a4440 con 0x7f5aa0106530 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5a9dffb640 1 -- 172.21.15.17:0/813528014 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c003d80 con 0x7f5aa0106530 2024-02-14T15:10:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.950+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa01a4920 con 0x7f5aa0106530 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.952+0000 7f5a9dffb640 1 -- 172.21.15.17:0/813528014 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5a8c005440 con 0x7f5aa0106530 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 Processor -- start 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 -- start start 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5a9dffb640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a900a2220 0x7f5a900a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319753b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x56531975c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x56531975f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653196c19e0 con 0x565319753720 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653196bef10 con 0x56531975cc20 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa9800740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653196ba6f0 con 0x565319754040 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5a9dffb640 1 -- 172.21.15.17:0/813528014 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f5a8c0f04c0 con 0x7f5aa0106530 2024-02-14T15:10:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5aa51cb640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a900a2220 0x7f5a900a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.953+0000 7f5a9ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x56531975c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x56531975c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:39692/0 (socket says 172.21.15.17:39692) 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 -- 172.21.15.17:0/1419253716 learned_addr learned my addr 172.21.15.17:0/1419253716 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319753b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 -- 172.21.15.17:0/1419253716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 msgr2=0x56531975f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x56531975f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 -- 172.21.15.17:0/1419253716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 msgr2=0x565319753b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319753b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 -- 172.21.15.17:0/1419253716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653193cb920 con 0x565319754040 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319753b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5aa51cb640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a900a2220 0x7f5a900a46e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a98005fd0 tx=0x7f5a980107a0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x56531975c6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0b9670 tx=0x7f5a8c0bc7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=402bbd266e706767 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a8affd640 1 -- 172.21.15.17:0/1419253716 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0bcca0 con 0x565319754040 2024-02-14T15:10:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a8affd640 1 -- 172.21.15.17:0/1419253716 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c0c0df0 con 0x565319754040 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.954+0000 7f5a8affd640 1 -- 172.21.15.17:0/1419253716 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c004f80 con 0x565319754040 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 -- 172.21.15.17:0/1419253716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 msgr2=0x56531975c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x56531975c6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0b9670 tx=0x7f5a8c0bc7e0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 -- 172.21.15.17:0/1419253716 shutdown_connections 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x56531975f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x56531975c6e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1419253716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319753b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 -- 172.21.15.17:0/1419253716 >> 172.21.15.17:0/1419253716 conn(0x56531974de50 msgr2=0x56531974e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 -- 172.21.15.17:0/1419253716 shutdown_connections 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 -- 172.21.15.17:0/1419253716 wait complete. 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 -- start start 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.955+0000 7f5aa9800740 1 Processor -- start 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319759fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa9800740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x565319656fa0 con 0x565319753720 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa9800740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531938e250 con 0x56531975cc20 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa9800740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x565319640200 con 0x565319754040 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa9800740 10 client.0 _ll_register_callbacks cb 0x5653196afb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35810/0 (socket says 172.21.15.17:35810) 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 -- 172.21.15.17:0/4101172200 learned_addr learned my addr 172.21.15.17:0/4101172200 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa51cb640 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319759fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 -- 172.21.15.17:0/4101172200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 msgr2=0x5653197599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 -- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 msgr2=0x565319759fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319759fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 -- 172.21.15.17:0/4101172200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa01a4920 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c5e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5a94013790 tx=0x7f5a94013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21b1642783bce51 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.956+0000 7f5a7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5a7ffff640 1 -- 172.21.15.17:0/4101172200 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940097e0 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5a7ffff640 1 -- 172.21.15.17:0/4101172200 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f5a94013e40 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653193ba600 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5a7ffff640 1 -- 172.21.15.17:0/4101172200 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a94011e10 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 10 client.16854 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5653196b8d70 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5a7ffff640 1 -- 172.21.15.17:0/4101172200 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5653193ba600 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5a7ffff640 1 -- 172.21.15.17:0/4101172200 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a9400e740 con 0x565319753720 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 20 client.16854 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56531976c750 0x56531976eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 4 client.16854 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56531940b1b0 con 0x56531976c750 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.957+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56531976c750 0x56531976eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5a7ffff640 10 client.16854 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56531976c750 0x56531976eb70 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f90046c51e6357c server_cookie=bf9360bb4b1cba66 in_seq=0 out_seq=0 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5a7ffff640 1 -- 172.21.15.17:0/4101172200 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56531940b1b0 con 0x56531976c750 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5a7ffff640 10 client.16854 handle_command_reply: tid=0 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 client.16854 shutdown 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 msgr2=0x56531975c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c5e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5a94013790 tx=0x7f5a94013c50 comp rx=0 tx=0).stop 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 shutdown_connections 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56531976c750 0x56531976eb70 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319759fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 --2- 172.21.15.17:0/4101172200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c5e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5a7ffff640 0 client.16854 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.958+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 >> 172.21.15.17:0/4101172200 conn(0x56531974de50 msgr2=0x56531974fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.959+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 shutdown_connections 2024-02-14T15:10:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.959+0000 7f5aa9800740 1 -- 172.21.15.17:0/4101172200 wait complete. 2024-02-14T15:10:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.959+0000 7f5aa9800740 20 client.16854 trim_cache size 0 max 16384 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 Processor -- start 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 -- start start 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565319754040 0x56531975c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56531975cc20 0x565319756c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531974ee80 con 0x565319753720 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531975acf0 con 0x565319754040 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa9800740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x565319749080 con 0x56531975cc20 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa49ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa49ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35818/0 (socket says 172.21.15.17:35818) 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa49ca640 1 -- 172.21.15.17:0/1150819236 learned_addr learned my addr 172.21.15.17:0/1150819236 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa51cb640 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56531975cc20 0x565319756c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565319754040 0x56531975c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa49ca640 1 -- 172.21.15.17:0/1150819236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56531975cc20 msgr2=0x565319756c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56531975cc20 0x565319756c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa49ca640 1 -- 172.21.15.17:0/1150819236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565319754040 msgr2=0x56531975c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.973+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565319754040 0x56531975c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.974+0000 7f5aa49ca640 1 -- 172.21.15.17:0/1150819236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565319655820 con 0x565319753720 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.974+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565319754040 0x56531975c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.974+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c0a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5a940117b0 tx=0x7f5a940117e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784f9cc3043cdd05 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.974+0000 7f5a8a7fc640 1 -- 172.21.15.17:0/1150819236 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400eec0 con 0x565319753720 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.974+0000 7f5a8a7fc640 1 -- 172.21.15.17:0/1150819236 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a9400e890 con 0x565319753720 2024-02-14T15:10:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.974+0000 7f5a8a7fc640 1 -- 172.21.15.17:0/1150819236 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94005720 con 0x565319753720 2024-02-14T15:10:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 -- 172.21.15.17:0/1150819236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 msgr2=0x56531975c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c0a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5a940117b0 tx=0x7f5a940117e0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 -- 172.21.15.17:0/1150819236 shutdown_connections 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56531975cc20 0x565319756c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565319754040 0x56531975c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 --2- 172.21.15.17:0/1150819236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x56531975c0a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 -- 172.21.15.17:0/1150819236 >> 172.21.15.17:0/1150819236 conn(0x56531974de50 msgr2=0x5653197582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 -- 172.21.15.17:0/1150819236 shutdown_connections 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 -- 172.21.15.17:0/1150819236 wait complete. 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 -- start start 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.975+0000 7f5aa9800740 1 Processor -- start 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319762490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa9800740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319762f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa9800740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653197c2d20 con 0x565319753720 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa9800740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653197c2e60 con 0x56531975cc20 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa9800740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653197c3020 con 0x565319754040 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa9800740 10 client.0 _ll_register_callbacks cb 0x7f5a8c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319762490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5a9ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319762490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35820/0 (socket says 172.21.15.17:35820) 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 -- 172.21.15.17:0/2206641400 learned_addr learned my addr 172.21.15.17:0/2206641400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa51cb640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319762f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 -- 172.21.15.17:0/2206641400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 msgr2=0x5653197629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 -- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 msgr2=0x565319762f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319762f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 -- 172.21.15.17:0/2206641400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653196b8d70 con 0x565319753720 2024-02-14T15:10:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa51cb640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319762f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5aa49ca640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319762490 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5a9402e040 tx=0x7f5a94011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d8c2985e3eabc1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5a7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.976+0000 7f5a7ffff640 1 -- 172.21.15.17:0/2206641400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400e5b0 con 0x565319753720 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a8c0c0df0 con 0x565319753720 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5a7ffff640 1 -- 172.21.15.17:0/2206641400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f5a94000e40 con 0x565319753720 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5a7ffff640 1 -- 172.21.15.17:0/2206641400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a940050d0 con 0x565319753720 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 10 client.16860 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56531967b720 con 0x565319753720 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5a7ffff640 1 -- 172.21.15.17:0/2206641400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c0c0df0 con 0x565319753720 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5a7ffff640 1 -- 172.21.15.17:0/2206641400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a94032030 con 0x565319753720 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 10 client.16860 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 20 client.16860 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5653197c4460 0x5653197c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 4 client.16860 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:10:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653193f7550 con 0x5653197c4460 2024-02-14T15:10:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.977+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5653197c4460 0x5653197c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.978+0000 7f5a9ffff640 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5653197c4460 0x5653197c6850 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cd2e5ea40c23ea server_cookie=f2939c4427a5c29d in_seq=0 out_seq=0 2024-02-14T15:10:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.978+0000 7f5a7ffff640 10 client.16860 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5a7ffff640 1 -- 172.21.15.17:0/2206641400 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653193f7550 con 0x5653197c4460 2024-02-14T15:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5a7ffff640 10 client.16860 handle_command_reply: tid=0 2024-02-14T15:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 client.16860 shutdown 2024-02-14T15:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 msgr2=0x565319762490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319762490 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5a9402e040 tx=0x7f5a94011e80 comp rx=0 tx=0).stop 2024-02-14T15:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 shutdown_connections 2024-02-14T15:10:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5653197c4460 0x5653197c6850 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531975cc20 0x565319762f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5a7ffff640 0 client.16860 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565319754040 0x5653197629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 --2- 172.21.15.17:0/2206641400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x565319753720 0x565319762490 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 >> 172.21.15.17:0/2206641400 conn(0x56531974de50 msgr2=0x56531974c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 shutdown_connections 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 1 -- 172.21.15.17:0/2206641400 wait complete. 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.989+0000 7f5aa9800740 20 client.16860 trim_cache size 0 max 16384 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:10:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:10:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:10:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a900a2220 msgr2=0x7f5a900a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a900a2220 0x7f5a900a46e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a98005fd0 tx=0x7f5a980107a0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0106530 msgr2=0x7f5aa019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0106530 0x7f5aa019db80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009230 tx=0x7f5a8c004c00 comp rx=0 tx=0).stop 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 shutdown_connections 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a900a2220 0x7f5a900a46e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa0106530 0x7f5aa019db80 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0197750 0x7f5aa0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 --2- 172.21.15.17:0/813528014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa0101460 0x7f5aa019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 >> 172.21.15.17:0/813528014 conn(0x7f5aa00fccc0 msgr2=0x7f5aa00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 shutdown_connections 2024-02-14T15:10:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:21.990+0000 7f5aa6c2e640 1 -- 172.21.15.17:0/813528014 wait complete. 2024-02-14T15:10:22.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 698538 11.58 MB/sec execute 204 sec latency 17.441 ms 2024-02-14T15:10:22.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:22 smithi076 ceph-mon[37057]: pgmap v214: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:22 smithi017 ceph-mon[32549]: pgmap v214: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:22 smithi050 ceph-mon[36512]: pgmap v214: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:23.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 700526 11.58 MB/sec execute 205 sec latency 21.329 ms 2024-02-14T15:10:24.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 703287 11.61 MB/sec execute 206 sec latency 15.293 ms 2024-02-14T15:10:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:24 smithi076 ceph-mon[37057]: pgmap v215: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:24 smithi017 ceph-mon[32549]: pgmap v215: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:24 smithi050 ceph-mon[36512]: pgmap v215: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:25.874 INFO:tasks.workunit.client.0.smithi017.stdout: 1 704540 11.58 MB/sec execute 207 sec latency 177.070 ms 2024-02-14T15:10:26.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 706783 11.59 MB/sec execute 208 sec latency 61.278 ms 2024-02-14T15:10:26.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:26 smithi076 ceph-mon[37057]: pgmap v216: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:26 smithi017 ceph-mon[32549]: pgmap v216: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:26 smithi050 ceph-mon[36512]: pgmap v216: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-02-14T15:10:27.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 708365 11.56 MB/sec execute 209 sec latency 29.823 ms 2024-02-14T15:10:28.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 710722 11.57 MB/sec execute 210 sec latency 13.093 ms 2024-02-14T15:10:28.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:28 smithi076 ceph-mon[37057]: pgmap v217: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 323 op/s 2024-02-14T15:10:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:28 smithi017 ceph-mon[32549]: pgmap v217: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 323 op/s 2024-02-14T15:10:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:28 smithi050 ceph-mon[36512]: pgmap v217: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 323 op/s 2024-02-14T15:10:29.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 713198 11.58 MB/sec execute 211 sec latency 19.257 ms 2024-02-14T15:10:30.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 714900 11.56 MB/sec execute 212 sec latency 66.345 ms 2024-02-14T15:10:30.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:30 smithi076 ceph-mon[37057]: pgmap v218: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 202 op/s 2024-02-14T15:10:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:30 smithi017 ceph-mon[32549]: pgmap v218: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 202 op/s 2024-02-14T15:10:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:30 smithi050 ceph-mon[36512]: pgmap v218: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 202 op/s 2024-02-14T15:10:31.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 717321 11.58 MB/sec execute 213 sec latency 82.314 ms 2024-02-14T15:10:32.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 718832 11.56 MB/sec execute 214 sec latency 28.522 ms 2024-02-14T15:10:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:32 smithi076 ceph-mon[37057]: pgmap v219: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 301 op/s 2024-02-14T15:10:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:32 smithi017 ceph-mon[32549]: pgmap v219: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 301 op/s 2024-02-14T15:10:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:32 smithi050 ceph-mon[36512]: pgmap v219: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 301 op/s 2024-02-14T15:10:33.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 721227 11.57 MB/sec execute 215 sec latency 16.310 ms 2024-02-14T15:10:34.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 723418 11.55 MB/sec execute 216 sec latency 15.713 ms 2024-02-14T15:10:34.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:34 smithi076 ceph-mon[37057]: pgmap v220: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-02-14T15:10:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:34 smithi017 ceph-mon[32549]: pgmap v220: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-02-14T15:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 -- 172.21.15.17:0/1281753039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d240696f0 msgr2=0x7f8d24105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 --2- 172.21.15.17:0/1281753039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d240696f0 0x7f8d24105e30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009930 tx=0x7f8d0c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 -- 172.21.15.17:0/1281753039 shutdown_connections 2024-02-14T15:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 --2- 172.21.15.17:0/1281753039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d240696f0 0x7f8d24105e30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 --2- 172.21.15.17:0/1281753039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d24106510 0x7f8d2410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 --2- 172.21.15.17:0/1281753039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d24068d40 0x7f8d24069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 -- 172.21.15.17:0/1281753039 >> 172.21.15.17:0/1281753039 conn(0x7f8d24077220 msgr2=0x7f8d24077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 -- 172.21.15.17:0/1281753039 shutdown_connections 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 -- 172.21.15.17:0/1281753039 wait complete. 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 Processor -- start 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.081+0000 7f8d2912c640 1 -- start start 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d24068d40 0x7f8d241000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d24106510 0x7f8d24100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d24100b70 0x7f8d241a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a6050 con 0x7f8d24106510 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a61c0 con 0x7f8d24100b70 2024-02-14T15:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a6380 con 0x7f8d24068d40 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d24106510 0x7f8d24100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d24106510 0x7f8d24100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60914/0 (socket says 172.21.15.17:60914) 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 -- 172.21.15.17:0/970672854 learned_addr learned my addr 172.21.15.17:0/970672854 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d24068d40 0x7f8d241000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2359e640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d24100b70 0x7f8d241a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 -- 172.21.15.17:0/970672854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d24068d40 msgr2=0x7f8d241000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d24068d40 0x7f8d241000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 -- 172.21.15.17:0/970672854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d24100b70 msgr2=0x7f8d241a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d24100b70 0x7f8d241a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 -- 172.21.15.17:0/970672854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c009560 con 0x7f8d24106510 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2359e640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d24100b70 0x7f8d241a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d24106510 0x7f8d24100630 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009230 tx=0x7f8d0c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d20d99640 1 -- 172.21.15.17:0/970672854 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c004350 con 0x7f8d24106510 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d241a6600 con 0x7f8d24106510 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d20d99640 1 -- 172.21.15.17:0/970672854 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c0044b0 con 0x7f8d24106510 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d20d99640 1 -- 172.21.15.17:0/970672854 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c003da0 con 0x7f8d24106510 2024-02-14T15:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.082+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241a6b60 con 0x7f8d24106510 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.085+0000 7f8d20d99640 1 -- 172.21.15.17:0/970672854 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8d0c004df0 con 0x7f8d24106510 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.085+0000 7f8d2bdda740 1 Processor -- start 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.085+0000 7f8d2bdda740 1 -- start start 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e722b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e72e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e6909e0 con 0x56292e722720 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e68df10 con 0x56292e723040 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e6896f0 con 0x56292e72bc20 2024-02-14T15:10:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d20d99640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d100a2250 0x7f8d100a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e722b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:45660/0 (socket says 172.21.15.17:45660) 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 -- 172.21.15.17:0/3375707923 learned_addr learned my addr 172.21.15.17:0/3375707923 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e72e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2359e640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d100a2250 0x7f8d100a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d20d99640 1 -- 172.21.15.17:0/970672854 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f8d0c0f0380 con 0x7f8d24106510 2024-02-14T15:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 -- 172.21.15.17:0/3375707923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 msgr2=0x56292e72e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e72e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 -- 172.21.15.17:0/3375707923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 msgr2=0x56292e722b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e722b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 -- 172.21.15.17:0/3375707923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56292e39a920 con 0x56292e723040 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2359e640 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e722b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b6e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009cf0 tx=0x7f8d14011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637deb4ab1ced155 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.086+0000 7f8d2359e640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d100a2250 0x7f8d100a4710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0099e0 tx=0x7f8d1c009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d197fa640 1 -- 172.21.15.17:0/3375707923 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14009ea0 con 0x56292e723040 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d197fa640 1 -- 172.21.15.17:0/3375707923 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d14004230 con 0x56292e723040 2024-02-14T15:10:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d197fa640 1 -- 172.21.15.17:0/3375707923 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14019800 con 0x56292e723040 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3375707923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 msgr2=0x56292e72b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b6e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009cf0 tx=0x7f8d14011810 comp rx=0 tx=0).stop 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3375707923 shutdown_connections 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e72e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b6e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3375707923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e722b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3375707923 >> 172.21.15.17:0/3375707923 conn(0x56292e71ce50 msgr2=0x56292e71d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3375707923 shutdown_connections 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.087+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3375707923 wait complete. 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 -- start start 2024-02-14T15:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 Processor -- start 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e72b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e728900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e728e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e625fa0 con 0x56292e723040 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e35d250 con 0x56292e722720 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e60f200 con 0x56292e72bc20 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2bdda740 10 client.0 _ll_register_callbacks cb 0x56292e67eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e728900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e728e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d22d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e72b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e728900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60928/0 (socket says 172.21.15.17:60928) 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d1bfff640 1 -- 172.21.15.17:0/1085561920 learned_addr learned my addr 172.21.15.17:0/1085561920 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.088+0000 7f8d2359e640 1 -- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 msgr2=0x56292e72b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2359e640 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e72b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2359e640 1 -- 172.21.15.17:0/1085561920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 msgr2=0x56292e728900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2359e640 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e728900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2359e640 1 -- 172.21.15.17:0/1085561920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d14009950 con 0x56292e72bc20 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e72b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e728900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2359e640 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e728e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005c40 tx=0x7f8d1c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ccb225045720da server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/1085561920 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c024070 con 0x56292e72bc20 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56292e389600 con 0x56292e72bc20 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/1085561920 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f8d1c00fe40 con 0x56292e72bc20 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/1085561920 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d1c01fdc0 con 0x56292e72bc20 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2bdda740 10 client.26395 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56292e687d70 con 0x56292e72bc20 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.089+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/1085561920 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56292e389600 con 0x56292e72bc20 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/1085561920 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d1c01dcd0 con 0x56292e72bc20 2024-02-14T15:10:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d2bdda740 10 client.26395 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d2bdda740 10 client.26395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d2bdda740 20 client.26395 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e73b9f0 0x56292e73dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d2bdda740 4 client.26395 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56292e3da1b0 con 0x56292e73b9f0 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e73b9f0 0x56292e73dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d0a7fc640 10 client.26395 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.090+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e73b9f0 0x56292e73dde0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89df87943d0045bf server_cookie=cbf6281a6bb088dd in_seq=0 out_seq=0 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/1085561920 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56292e3da1b0 con 0x56292e73b9f0 2024-02-14T15:10:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d0a7fc640 10 client.26395 handle_command_reply: tid=0 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 client.26395 shutdown 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 msgr2=0x56292e728e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e728e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005c40 tx=0x7f8d1c004a40 comp rx=0 tx=0).stop 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 shutdown_connections 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e73b9f0 0x56292e73dde0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e728900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e728e60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d0a7fc640 0 client.26395 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1085561920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e72b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 >> 172.21.15.17:0/1085561920 conn(0x56292e71ce50 msgr2=0x56292e71eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 shutdown_connections 2024-02-14T15:10:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1085561920 wait complete. 2024-02-14T15:10:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.091+0000 7f8d2bdda740 20 client.26395 trim_cache size 0 max 16384 2024-02-14T15:10:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 Processor -- start 2024-02-14T15:10:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 -- start start 2024-02-14T15:10:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e72b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e727230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e738470 con 0x56292e722720 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e718090 con 0x56292e723040 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e722380 con 0x56292e72bc20 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e72b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d2359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e727230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e72b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60936/0 (socket says 172.21.15.17:60936) 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 -- 172.21.15.17:0/1221174227 learned_addr learned my addr 172.21.15.17:0/1221174227 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 -- 172.21.15.17:0/1221174227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 msgr2=0x56292e727230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e727230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 -- 172.21.15.17:0/1221174227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 msgr2=0x56292e72b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d22d9d640 1 -- 172.21.15.17:0/1221174227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56292e624820 con 0x56292e722720 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.105+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.106+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e72b030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8d14014040 tx=0x7f8d14011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd639265f2beaf2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.106+0000 7f8d18ff9640 1 -- 172.21.15.17:0/1221174227 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1402f070 con 0x56292e722720 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.106+0000 7f8d18ff9640 1 -- 172.21.15.17:0/1221174227 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d14003900 con 0x56292e722720 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.106+0000 7f8d18ff9640 1 -- 172.21.15.17:0/1221174227 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14005460 con 0x56292e722720 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.106+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1221174227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 msgr2=0x56292e72b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.106+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e72b030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8d14014040 tx=0x7f8d14011810 comp rx=0 tx=0).stop 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1221174227 shutdown_connections 2024-02-14T15:10:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e727230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e723040 0x56292e72b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/1221174227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e722720 0x56292e72b030 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1221174227 >> 172.21.15.17:0/1221174227 conn(0x56292e71ce50 msgr2=0x56292e742760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1221174227 shutdown_connections 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- 172.21.15.17:0/1221174227 wait complete. 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- start start 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 Processor -- start 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e730ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e730ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e790ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e731790 con 0x56292e723040 2024-02-14T15:10:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e731900 con 0x56292e722720 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292e791420 con 0x56292e72bc20 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2bdda740 10 client.0 _ll_register_callbacks cb 0x7f8d140097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e730ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e730ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60952/0 (socket says 172.21.15.17:60952) 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3206193791 learned_addr learned my addr 172.21.15.17:0/3206193791 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.107+0000 7f8d2359e640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e790ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e730ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3206193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 msgr2=0x56292e790ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e790ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 msgr2=0x56292e730ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e730ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3206193791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241a6b60 con 0x56292e723040 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e730ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d2359e640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e790ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e730ff0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005ec0 tx=0x7f8d0c08bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b0470ac751ec9c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/3206193791 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0bb080 con 0x56292e723040 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/3206193791 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f8d0c0bd1c0 con 0x56292e723040 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d14004230 con 0x56292e723040 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/3206193791 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d0c0740b0 con 0x56292e723040 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d2bdda740 10 client.16878 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56292e39a920 con 0x56292e723040 2024-02-14T15:10:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.108+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/3206193791 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d14004230 con 0x56292e723040 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/3206193791 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d0c03c070 con 0x56292e723040 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d2bdda740 10 client.16878 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d2bdda740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d2bdda740 20 client.16878 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e792ac0 0x56292e794ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d2bdda740 4 client.16878 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56292e3c6550 con 0x56292e792ac0 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e792ac0 0x56292e794ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d0a7fc640 10 client.16878 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.109+0000 7f8d22d9d640 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e792ac0 0x56292e794ee0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309f5ea39b7fbaef server_cookie=9f3ef821412f4d13 in_seq=0 out_seq=0 2024-02-14T15:10:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.119+0000 7f8d0a7fc640 1 -- 172.21.15.17:0/3206193791 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56292e3c6550 con 0x56292e792ac0 2024-02-14T15:10:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.119+0000 7f8d0a7fc640 10 client.16878 handle_command_reply: tid=0 2024-02-14T15:10:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.119+0000 7f8d2bdda740 1 client.16878 shutdown 2024-02-14T15:10:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 msgr2=0x56292e730ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e730ff0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005ec0 tx=0x7f8d0c08bc60 comp rx=0 tx=0).stop 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 shutdown_connections 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56292e792ac0 0x56292e794ee0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56292e723040 0x56292e730ff0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56292e72bc20 0x56292e790ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d0a7fc640 0 client.16878 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 --2- 172.21.15.17:0/3206193791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56292e722720 0x56292e730ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 >> 172.21.15.17:0/3206193791 conn(0x56292e71ce50 msgr2=0x56292e7271f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 shutdown_connections 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 1 -- 172.21.15.17:0/3206193791 wait complete. 2024-02-14T15:10:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2bdda740 20 client.16878 trim_cache size 0 max 16384 2024-02-14T15:10:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:10:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:10:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:10:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:10:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d100a2250 msgr2=0x7f8d100a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d100a2250 0x7f8d100a4710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0099e0 tx=0x7f8d1c009500 comp rx=0 tx=0).stop 2024-02-14T15:10:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d24106510 msgr2=0x7f8d24100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d24106510 0x7f8d24100630 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009230 tx=0x7f8d0c02c680 comp rx=0 tx=0).stop 2024-02-14T15:10:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 shutdown_connections 2024-02-14T15:10:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d100a2250 0x7f8d100a4710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d24106510 0x7f8d24100630 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d24100b70 0x7f8d241a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 --2- 172.21.15.17:0/970672854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d24068d40 0x7f8d241000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.120+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 >> 172.21.15.17:0/970672854 conn(0x7f8d24077220 msgr2=0x7f8d240fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.121+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 shutdown_connections 2024-02-14T15:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:35.121+0000 7f8d2912c640 1 -- 172.21.15.17:0/970672854 wait complete. 2024-02-14T15:10:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:34 smithi050 ceph-mon[36512]: pgmap v220: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-02-14T15:10:35.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 725276 11.56 MB/sec execute 217 sec latency 94.528 ms 2024-02-14T15:10:36.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 727592 11.56 MB/sec execute 218 sec latency 63.682 ms 2024-02-14T15:10:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:36 smithi076 ceph-mon[37057]: pgmap v221: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-02-14T15:10:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:36 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:10:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:36 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:10:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:36 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:10:36.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:36 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:10:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:36 smithi017 ceph-mon[32549]: pgmap v221: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-02-14T15:10:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:36 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:10:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:36 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:10:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:36 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:10:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:36 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:10:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:36 smithi050 ceph-mon[36512]: pgmap v221: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 202 op/s 2024-02-14T15:10:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:36 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:10:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:36 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:10:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:36 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:10:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:36 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:10:37.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 729101 11.54 MB/sec execute 219 sec latency 56.447 ms 2024-02-14T15:10:38.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 731715 11.57 MB/sec execute 220 sec latency 22.180 ms 2024-02-14T15:10:38.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:38 smithi076 ceph-mon[37057]: pgmap v222: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 300 op/s 2024-02-14T15:10:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:38 smithi017 ceph-mon[32549]: pgmap v222: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 300 op/s 2024-02-14T15:10:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:38 smithi050 ceph-mon[36512]: pgmap v222: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 300 op/s 2024-02-14T15:10:39.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 733536 11.54 MB/sec execute 221 sec latency 24.810 ms 2024-02-14T15:10:40.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 735754 11.55 MB/sec execute 222 sec latency 18.158 ms 2024-02-14T15:10:40.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:40 smithi076 ceph-mon[37057]: pgmap v223: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-02-14T15:10:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:40 smithi017 ceph-mon[32549]: pgmap v223: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-02-14T15:10:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:40 smithi050 ceph-mon[36512]: pgmap v223: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-02-14T15:10:41.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 738414 11.56 MB/sec execute 223 sec latency 16.922 ms 2024-02-14T15:10:42.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 739579 11.54 MB/sec execute 224 sec latency 48.751 ms 2024-02-14T15:10:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:42 smithi076 ceph-mon[37057]: pgmap v224: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-02-14T15:10:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:42 smithi017 ceph-mon[32549]: pgmap v224: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-02-14T15:10:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:42 smithi050 ceph-mon[36512]: pgmap v224: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.8 MiB/s wr, 298 op/s 2024-02-14T15:10:43.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 742237 11.56 MB/sec execute 225 sec latency 18.120 ms 2024-02-14T15:10:44.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 743820 11.54 MB/sec execute 226 sec latency 19.148 ms 2024-02-14T15:10:44.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:44 smithi050 ceph-mon[36512]: pgmap v225: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:10:44.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:44 smithi076 ceph-mon[37057]: pgmap v225: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:10:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:44 smithi017 ceph-mon[32549]: pgmap v225: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:10:45.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 746080 11.55 MB/sec execute 227 sec latency 18.727 ms 2024-02-14T15:10:46.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 748258 11.53 MB/sec execute 228 sec latency 16.722 ms 2024-02-14T15:10:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:46 smithi017 ceph-mon[32549]: pgmap v226: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:10:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:46 smithi050 ceph-mon[36512]: pgmap v226: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:10:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:46 smithi076 ceph-mon[37057]: pgmap v226: 129 pgs: 129 active+clean; 440 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 199 op/s 2024-02-14T15:10:47.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 749590 11.53 MB/sec execute 229 sec latency 191.115 ms 2024-02-14T15:10:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 -- 172.21.15.17:0/1917399420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc100340 msgr2=0x7fc9dc100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/1917399420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc100340 0x7fc9dc100720 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009cd0 tx=0x7fc9d402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 -- 172.21.15.17:0/1917399420 shutdown_connections 2024-02-14T15:10:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/1917399420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc105410 0x7fc9dc10cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/1917399420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9dc100cf0 0x7fc9dc104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/1917399420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc100340 0x7fc9dc100720 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 -- 172.21.15.17:0/1917399420 >> 172.21.15.17:0/1917399420 conn(0x7fc9dc077220 msgr2=0x7fc9dc077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 -- 172.21.15.17:0/1917399420 shutdown_connections 2024-02-14T15:10:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.213+0000 7fc9e0f93640 1 -- 172.21.15.17:0/1917399420 wait complete. 2024-02-14T15:10:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 Processor -- start 2024-02-14T15:10:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 -- start start 2024-02-14T15:10:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc100340 0x7fc9dc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc105410 0x7fc9dc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9dc197700 0x7fc9dc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc1981d0 con 0x7fc9dc105410 2024-02-14T15:10:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc198340 con 0x7fc9dc100340 2024-02-14T15:10:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9e0f93640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc198500 con 0x7fc9dc197700 2024-02-14T15:10:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc105410 0x7fc9dc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc105410 0x7fc9dc19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57380/0 (socket says 172.21.15.17:57380) 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/2034086484 learned_addr learned my addr 172.21.15.17:0/2034086484 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9da59c640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc100340 0x7fc9dc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9dad9d640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9dc197700 0x7fc9dc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9da59c640 1 -- 172.21.15.17:0/2034086484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9dc197700 msgr2=0x7fc9dc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9da59c640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9dc197700 0x7fc9dc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9da59c640 1 -- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc105410 msgr2=0x7fc9dc19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9da59c640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc105410 0x7fc9dc19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9da59c640 1 -- 172.21.15.17:0/2034086484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4009900 con 0x7fc9dc100340 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.214+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc105410 0x7fc9dc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.215+0000 7fc9da59c640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc100340 0x7fc9dc19d640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9d402c700 tx=0x7fc9d40325d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.215+0000 7fc9d37fe640 1 -- 172.21.15.17:0/2034086484 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40328b0 con 0x7fc9dc100340 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.215+0000 7fc9d37fe640 1 -- 172.21.15.17:0/2034086484 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9d4003e50 con 0x7fc9dc100340 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.215+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc1a4440 con 0x7fc9dc100340 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.215+0000 7fc9d37fe640 1 -- 172.21.15.17:0/2034086484 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4004860 con 0x7fc9dc100340 2024-02-14T15:10:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.215+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc1a48c0 con 0x7fc9dc100340 2024-02-14T15:10:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.217+0000 7fc9d37fe640 1 -- 172.21.15.17:0/2034086484 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc9d400a040 con 0x7fc9dc100340 2024-02-14T15:10:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.218+0000 7fc9d37fe640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9c80a2220 0x7fc9c80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.218+0000 7fc9e3b67740 1 Processor -- start 2024-02-14T15:10:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.218+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9c80a2220 0x7fc9c80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.218+0000 7fc9d37fe640 1 -- 172.21.15.17:0/2034086484 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fc9d40f0ef0 con 0x7fc9dc100340 2024-02-14T15:10:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.218+0000 7fc9e3b67740 1 -- start start 2024-02-14T15:10:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b64010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8ac69e0 con 0x5631e8b59040 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8ac3f10 con 0x5631e8b61c20 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8abf6f0 con 0x5631e8b58720 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9c80a2220 0x7fc9c80a46e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc198f50 tx=0x7fc9c4005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57392/0 (socket says 172.21.15.17:57392) 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b58b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:54884/0 (socket says 172.21.15.17:54884) 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9dad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b64010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/2193762441 learned_addr learned my addr 172.21.15.17:0/2193762441 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 -- 172.21.15.17:0/2193762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 msgr2=0x5631e8b64010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b64010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 -- 172.21.15.17:0/2193762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 msgr2=0x5631e8b616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 -- 172.21.15.17:0/2193762441 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e87d0920 con 0x5631e8b58720 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9da59c640 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b58b00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40ee330 tx=0x7fc9d40b99c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d07d5e08eb5f2ff3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.219+0000 7fc9d0ff9640 1 -- 172.21.15.17:0/2193762441 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40bdd60 con 0x5631e8b58720 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9d0ff9640 1 -- 172.21.15.17:0/2193762441 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9d40bd660 con 0x5631e8b58720 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9d0ff9640 1 -- 172.21.15.17:0/2193762441 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40040f0 con 0x5631e8b58720 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 -- 172.21.15.17:0/2193762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 msgr2=0x5631e8b58b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b58b00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40ee330 tx=0x7fc9d40b99c0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 -- 172.21.15.17:0/2193762441 shutdown_connections 2024-02-14T15:10:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b64010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/2193762441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b58b00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 -- 172.21.15.17:0/2193762441 >> 172.21.15.17:0/2193762441 conn(0x5631e8b52e50 msgr2=0x5631e8b53230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 -- 172.21.15.17:0/2193762441 shutdown_connections 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.220+0000 7fc9e3b67740 1 -- 172.21.15.17:0/2193762441 wait complete. 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 -- start start 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 Processor -- start 2024-02-14T15:10:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b58720 0x5631e8b61530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b59040 0x5631e8b5e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b5ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8a5bfa0 con 0x5631e8b58720 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8793250 con 0x5631e8b61c20 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8a45200 con 0x5631e8b59040 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9e3b67740 10 client.0 _ll_register_callbacks cb 0x5631e8ab4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9da59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b58720 0x5631e8b61530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9da59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b58720 0x5631e8b61530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57394/0 (socket says 172.21.15.17:57394) 2024-02-14T15:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9da59c640 1 -- 172.21.15.17:0/3307032318 learned_addr learned my addr 172.21.15.17:0/3307032318 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9dad9d640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b5ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.221+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b59040 0x5631e8b5e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9da59c640 1 -- 172.21.15.17:0/3307032318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b59040 msgr2=0x5631e8b5e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9da59c640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b59040 0x5631e8b5e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9da59c640 1 -- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 msgr2=0x5631e8b5ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9da59c640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b5ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9da59c640 1 -- 172.21.15.17:0/3307032318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc1a48c0 con 0x5631e8b58720 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9dad9d640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b5ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9da59c640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b58720 0x5631e8b61530 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40bd5f0 tx=0x7fc9d4004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e03fdba980024a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/3307032318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40bdd60 con 0x5631e8b58720 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/3307032318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fc9d40c17b0 con 0x5631e8b58720 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631e87bf600 con 0x5631e8b58720 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/3307032318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9d40dd0f0 con 0x5631e8b58720 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9e3b67740 10 client.16890 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.222+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5631e8abdd70 con 0x5631e8b58720 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/3307032318 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5631e87bf600 con 0x5631e8b58720 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/3307032318 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc9d40c58e0 con 0x5631e8b58720 2024-02-14T15:10:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9e3b67740 10 client.16890 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9e3b67740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9e3b67740 20 client.16890 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8b712a0 0x5631e8b73690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9e3b67740 4 client.16890 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:10:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631e88101b0 con 0x5631e8b712a0 2024-02-14T15:10:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8b712a0 0x5631e8b73690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8b712a0 0x5631e8b73690 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64378efbe63bb63d server_cookie=5dff08acad2af36 in_seq=0 out_seq=0 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.223+0000 7fc9c1ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/3307032318 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631e88101b0 con 0x5631e8b712a0 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9c1ffb640 10 client.16890 handle_command_reply: tid=0 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 client.16890 shutdown 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b58720 msgr2=0x5631e8b61530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b58720 0x5631e8b61530 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40bd5f0 tx=0x7fc9d4004300 comp rx=0 tx=0).stop 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 shutdown_connections 2024-02-14T15:10:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8b712a0 0x5631e8b73690 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b5ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b59040 0x5631e8b5e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9c1ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/3307032318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b58720 0x5631e8b61530 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 >> 172.21.15.17:0/3307032318 conn(0x5631e8b52e50 msgr2=0x5631e8b54ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 shutdown_connections 2024-02-14T15:10:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 1 -- 172.21.15.17:0/3307032318 wait complete. 2024-02-14T15:10:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.224+0000 7fc9e3b67740 20 client.16890 trim_cache size 0 max 16384 2024-02-14T15:10:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 Processor -- start 2024-02-14T15:10:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 -- start start 2024-02-14T15:10:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b58720 0x5631e8b59c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b60ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b61c20 0x5631e8b61530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b6e1e0 con 0x5631e8b59040 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b53e50 con 0x5631e8b58720 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b5f160 con 0x5631e8b61c20 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b60ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9da59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b58720 0x5631e8b59c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b60ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57404/0 (socket says 172.21.15.17:57404) 2024-02-14T15:10:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/60318149 learned_addr learned my addr 172.21.15.17:0/60318149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9dad9d640 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b61c20 0x5631e8b61530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/60318149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b61c20 msgr2=0x5631e8b61530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b61c20 0x5631e8b61530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/60318149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b58720 msgr2=0x5631e8b59c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b58720 0x5631e8b59c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/60318149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e8a5a820 con 0x5631e8b59040 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.239+0000 7fc9da59c640 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b58720 0x5631e8b59c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.240+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b60ff0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4005d00 tx=0x7fc9c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3118796fb76dfa4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.240+0000 7fc9c3fff640 1 -- 172.21.15.17:0/60318149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4027040 con 0x5631e8b59040 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.240+0000 7fc9c3fff640 1 -- 172.21.15.17:0/60318149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9c400fed0 con 0x5631e8b59040 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.240+0000 7fc9c3fff640 1 -- 172.21.15.17:0/60318149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4023ea0 con 0x5631e8b59040 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.240+0000 7fc9e3b67740 1 -- 172.21.15.17:0/60318149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 msgr2=0x5631e8b60ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.240+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b60ff0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4005d00 tx=0x7fc9c4014000 comp rx=0 tx=0).stop 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- 172.21.15.17:0/60318149 shutdown_connections 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b60ff0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b61c20 0x5631e8b61530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/60318149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b58720 0x5631e8b59c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- 172.21.15.17:0/60318149 >> 172.21.15.17:0/60318149 conn(0x5631e8b52e50 msgr2=0x5631e8b59560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- 172.21.15.17:0/60318149 shutdown_connections 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- 172.21.15.17:0/60318149 wait complete. 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- start start 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 Processor -- start 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b67290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b67d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8b69110 con 0x5631e8b59040 2024-02-14T15:10:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8bc7780 con 0x5631e8b61c20 2024-02-14T15:10:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e8bc78c0 con 0x5631e8b58720 2024-02-14T15:10:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9e3b67740 10 client.0 _ll_register_callbacks cb 0x7fc9c4009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:10:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b67290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9d9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b677d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57418/0 (socket says 172.21.15.17:57418) 2024-02-14T15:10:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/884639095 learned_addr learned my addr 172.21.15.17:0/884639095 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:10:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.241+0000 7fc9dad9d640 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b67d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/884639095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 msgr2=0x5631e8b67290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b67290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 msgr2=0x5631e8b67d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b67d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9d9d9b640 1 -- 172.21.15.17:0/884639095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631e8abdd70 con 0x5631e8b59040 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9d9d9b640 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b677d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4031040 tx=0x7fc9c400f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0bed1ed3ca2bee server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:10:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/884639095 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4027040 con 0x5631e8b59040 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/884639095 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fc9c4023830 con 0x5631e8b59040 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9d40bd660 con 0x5631e8b59040 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/884639095 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9c40239f0 con 0x5631e8b59040 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 10 client.16896 fetch_fsmap learned FSMap version 30 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fc9c400fa80 con 0x5631e8b59040 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/884639095 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9d40bd660 con 0x5631e8b59040 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/884639095 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc9c4004c90 con 0x5631e8b59040 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 10 client.16896 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:10:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 20 client.16896 populate_metadata read hostname 'smithi017' 2024-02-14T15:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8bc9030 0x5631e8bcb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 4 client.16896 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.242+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631e87fc550 con 0x5631e8bc9030 2024-02-14T15:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.243+0000 7fc9da59c640 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8bc9030 0x5631e8bcb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.243+0000 7fc9da59c640 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8bc9030 0x5631e8bcb450 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d211e939e713c3 server_cookie=83dac537102d4be in_seq=0 out_seq=0 2024-02-14T15:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.243+0000 7fc9c1ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9c1ffb640 1 -- 172.21.15.17:0/884639095 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631e87fc550 con 0x5631e8bc9030 2024-02-14T15:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9c1ffb640 10 client.16896 handle_command_reply: tid=0 2024-02-14T15:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 client.16896 shutdown 2024-02-14T15:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 msgr2=0x5631e8b677d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b677d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4031040 tx=0x7fc9c400f650 comp rx=0 tx=0).stop 2024-02-14T15:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 shutdown_connections 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5631e8bc9030 0x5631e8bcb450 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5631e8b59040 0x5631e8b677d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5631e8b61c20 0x5631e8b67d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 --2- 172.21.15.17:0/884639095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5631e8b58720 0x5631e8b67290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 >> 172.21.15.17:0/884639095 conn(0x5631e8b52e50 msgr2=0x5631e8b548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9c1ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 shutdown_connections 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 1 -- 172.21.15.17:0/884639095 wait complete. 2024-02-14T15:10:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.253+0000 7fc9e3b67740 20 client.16896 trim_cache size 0 max 16384 2024-02-14T15:10:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:10:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:10:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:10:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9c80a2220 msgr2=0x7fc9c80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9c80a2220 0x7fc9c80a46e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc198f50 tx=0x7fc9c4005f90 comp rx=0 tx=0).stop 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc100340 msgr2=0x7fc9dc19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc100340 0x7fc9dc19d640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9d402c700 tx=0x7fc9d40325d0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 shutdown_connections 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc9c80a2220 0x7fc9c80a46e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9dc105410 0x7fc9dc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9dc197700 0x7fc9dc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 --2- 172.21.15.17:0/2034086484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc100340 0x7fc9dc19d640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 >> 172.21.15.17:0/2034086484 conn(0x7fc9dc077220 msgr2=0x7fc9dc068dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 shutdown_connections 2024-02-14T15:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:10:48.254+0000 7fc9e0f93640 1 -- 172.21.15.17:0/2034086484 wait complete. 2024-02-14T15:10:48.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 751614 11.51 MB/sec execute 230 sec latency 17.809 ms 2024-02-14T15:10:49.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:48 smithi017 ceph-mon[32549]: pgmap v227: 129 pgs: 129 active+clean; 448 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 295 op/s 2024-02-14T15:10:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:48 smithi050 ceph-mon[36512]: pgmap v227: 129 pgs: 129 active+clean; 448 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 295 op/s 2024-02-14T15:10:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:48 smithi076 ceph-mon[37057]: pgmap v227: 129 pgs: 129 active+clean; 448 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 295 op/s 2024-02-14T15:10:49.875 INFO:tasks.workunit.client.0.smithi017.stdout: 1 753642 11.52 MB/sec execute 231 sec latency 17.619 ms 2024-02-14T15:10:50.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 756436 11.54 MB/sec execute 232 sec latency 13.191 ms 2024-02-14T15:10:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:50 smithi017 ceph-mon[32549]: pgmap v228: 129 pgs: 129 active+clean; 448 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-02-14T15:10:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:50 smithi050 ceph-mon[36512]: pgmap v228: 129 pgs: 129 active+clean; 448 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-02-14T15:10:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:50 smithi076 ceph-mon[37057]: pgmap v228: 129 pgs: 129 active+clean; 448 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 196 op/s 2024-02-14T15:10:51.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 758188 11.52 MB/sec execute 233 sec latency 30.172 ms 2024-02-14T15:10:52.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 760158 11.53 MB/sec execute 234 sec latency 74.073 ms 2024-02-14T15:10:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:52 smithi017 ceph-mon[32549]: pgmap v229: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 295 op/s 2024-02-14T15:10:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:52 smithi050 ceph-mon[36512]: pgmap v229: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 295 op/s 2024-02-14T15:10:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:52 smithi076 ceph-mon[37057]: pgmap v229: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 295 op/s 2024-02-14T15:10:53.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 762307 11.51 MB/sec execute 235 sec latency 16.290 ms 2024-02-14T15:10:54.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 764293 11.52 MB/sec execute 236 sec latency 23.634 ms 2024-02-14T15:10:55.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:54 smithi017 ceph-mon[32549]: pgmap v230: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-02-14T15:10:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:54 smithi050 ceph-mon[36512]: pgmap v230: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-02-14T15:10:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:54 smithi076 ceph-mon[37057]: pgmap v230: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-02-14T15:10:55.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 767086 11.54 MB/sec execute 237 sec latency 13.681 ms 2024-02-14T15:10:56.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 768783 11.52 MB/sec execute 238 sec latency 30.426 ms 2024-02-14T15:10:57.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:56 smithi017 ceph-mon[32549]: pgmap v231: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-02-14T15:10:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:56 smithi050 ceph-mon[36512]: pgmap v231: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-02-14T15:10:57.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:56 smithi076 ceph-mon[37057]: pgmap v231: 129 pgs: 129 active+clean; 457 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 194 op/s 2024-02-14T15:10:57.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 770740 11.53 MB/sec execute 239 sec latency 115.274 ms 2024-02-14T15:10:58.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 772203 11.50 MB/sec execute 240 sec latency 92.886 ms 2024-02-14T15:10:59.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:10:58 smithi017 ceph-mon[32549]: pgmap v232: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-02-14T15:10:59.035 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:10:58 smithi076 ceph-mon[37057]: pgmap v232: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-02-14T15:10:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:10:58 smithi050 ceph-mon[36512]: pgmap v232: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.0 MiB/s wr, 294 op/s 2024-02-14T15:10:59.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 774642 11.51 MB/sec execute 241 sec latency 17.651 ms 2024-02-14T15:11:00.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 777014 11.52 MB/sec execute 242 sec latency 19.286 ms 2024-02-14T15:11:01.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:00 smithi017 ceph-mon[32549]: pgmap v233: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:00 smithi050 ceph-mon[36512]: pgmap v233: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:00 smithi076 ceph-mon[37057]: pgmap v233: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.355+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2375306103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 msgr2=0x7f6b6810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.355+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2375306103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b6810cff0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009cd0 tx=0x7f6b6002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.355+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2375306103 shutdown_connections 2024-02-14T15:11:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.355+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2375306103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b6810cff0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.355+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2375306103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b68100d00 0x7f6b68104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.355+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2375306103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b68100350 0x7f6b68100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.355+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2375306103 >> 172.21.15.17:0/2375306103 conn(0x7f6b680fbbd0 msgr2=0x7f6b680fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.356+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2375306103 shutdown_connections 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.356+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2375306103 wait complete. 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.356+0000 7f6b6f2b5640 1 Processor -- start 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.356+0000 7f6b6f2b5640 1 -- start start 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6f2b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b68100350 0x7f6b68101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6f2b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b68102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6f2b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b68102860 0x7f6b681a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6f2b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681a6070 con 0x7f6b68105420 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6f2b5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681a61e0 con 0x7f6b68102860 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6f2b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681a63a0 con 0x7f6b68100350 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b68102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6d852640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b68102860 0x7f6b681a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b68102320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43222/0 (socket says 172.21.15.17:43222) 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 -- 172.21.15.17:0/2056183894 learned_addr learned my addr 172.21.15.17:0/2056183894 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b68100350 0x7f6b68101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 -- 172.21.15.17:0/2056183894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b68100350 msgr2=0x7f6b68101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b68100350 0x7f6b68101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 -- 172.21.15.17:0/2056183894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b68102860 msgr2=0x7f6b681a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b68102860 0x7f6b681a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 -- 172.21.15.17:0/2056183894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60009900 con 0x7f6b68105420 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.357+0000 7f6b6c850640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b68102320 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800e9c0 tx=0x7f6b5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.358+0000 7f6b5e7fc640 1 -- 172.21.15.17:0/2056183894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58009700 con 0x7f6b68105420 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.358+0000 7f6b5e7fc640 1 -- 172.21.15.17:0/2056183894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b5800cd80 con 0x7f6b68105420 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.358+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b681a6680 con 0x7f6b68105420 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.358+0000 7f6b5e7fc640 1 -- 172.21.15.17:0/2056183894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58010430 con 0x7f6b68105420 2024-02-14T15:11:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.358+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b681a6b90 con 0x7f6b68105420 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.360+0000 7f6b5e7fc640 1 -- 172.21.15.17:0/2056183894 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6b58005000 con 0x7f6b68105420 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 Processor -- start 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 -- start start 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b5e7fc640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6b540a2250 0x7f6b540a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5b720 0x561d12f5bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f64c20 0x561d12f67010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12ec99e0 con 0x561d12f5b720 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12ec6f10 con 0x561d12f64c20 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b71f67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12ec26f0 con 0x561d12f5c040 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b5e7fc640 1 -- 172.21.15.17:0/2056183894 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f6b58014070 con 0x7f6b68105420 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5b720 0x561d12f5bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.361+0000 7f6b6d852640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6b540a2250 0x7f6b540a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5b720 0x561d12f5bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43234/0 (socket says 172.21.15.17:43234) 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 -- 172.21.15.17:0/2307524969 learned_addr learned my addr 172.21.15.17:0/2307524969 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d852640 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f64c20 0x561d12f67010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 -- 172.21.15.17:0/2307524969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 msgr2=0x561d12f646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 -- 172.21.15.17:0/2307524969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f64c20 msgr2=0x561d12f67010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f64c20 0x561d12f67010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 -- 172.21.15.17:0/2307524969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d12bd3920 con 0x561d12f5b720 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d852640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6b540a2250 0x7f6b540a4710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b60005cc0 tx=0x7f6b60032f20 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5b720 0x561d12f5bb00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009cf0 tx=0x7f6b6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ba6a61e2554a6e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b537fe640 1 -- 172.21.15.17:0/2307524969 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64009de0 con 0x561d12f5b720 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b537fe640 1 -- 172.21.15.17:0/2307524969 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b640035f0 con 0x561d12f5b720 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.362+0000 7f6b537fe640 1 -- 172.21.15.17:0/2307524969 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64021690 con 0x561d12f5b720 2024-02-14T15:11:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 -- 172.21.15.17:0/2307524969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5b720 msgr2=0x561d12f5bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5b720 0x561d12f5bb00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009cf0 tx=0x7f6b6400f790 comp rx=0 tx=0).stop 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 -- 172.21.15.17:0/2307524969 shutdown_connections 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f64c20 0x561d12f67010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2307524969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5b720 0x561d12f5bb00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 -- 172.21.15.17:0/2307524969 >> 172.21.15.17:0/2307524969 conn(0x561d12f55e50 msgr2=0x561d12f56230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 -- 172.21.15.17:0/2307524969 shutdown_connections 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.363+0000 7f6b71f67740 1 -- 172.21.15.17:0/2307524969 wait complete. 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.364+0000 7f6b71f67740 1 -- start start 2024-02-14T15:11:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.364+0000 7f6b71f67740 1 Processor -- start 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.364+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5b720 0x561d12f64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.364+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5c040 0x561d12f619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.364+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f61fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b71f67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12e5efa0 con 0x561d12f64c20 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b71f67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12b96250 con 0x561d12f5c040 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b71f67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12e48200 con 0x561d12f5b720 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b71f67740 10 client.0 _ll_register_callbacks cb 0x561d12eb7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6c850640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5c040 0x561d12f619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f61fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f61fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43240/0 (socket says 172.21.15.17:43240) 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 -- 172.21.15.17:0/2353329726 learned_addr learned my addr 172.21.15.17:0/2353329726 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6c850640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5c040 0x561d12f619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:35158/0 (socket says 172.21.15.17:35158) 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5b720 0x561d12f64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 -- 172.21.15.17:0/2353329726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5b720 msgr2=0x561d12f64630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5b720 0x561d12f64630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 -- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5c040 msgr2=0x561d12f619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5c040 0x561d12f619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 -- 172.21.15.17:0/2353329726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b64009950 con 0x561d12f64c20 2024-02-14T15:11:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5b720 0x561d12f64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6c850640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5c040 0x561d12f619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b6d852640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f61fd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6b60004af0 tx=0x7f6b60005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec68d11e7952d2f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b50ff9640 1 -- 172.21.15.17:0/2353329726 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60048070 con 0x561d12f64c20 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b50ff9640 1 -- 172.21.15.17:0/2353329726 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f6b60034c10 con 0x561d12f64c20 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.365+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d12bc2600 con 0x561d12f64c20 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.366+0000 7f6b50ff9640 1 -- 172.21.15.17:0/2353329726 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b60034e10 con 0x561d12f64c20 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.366+0000 7f6b71f67740 10 client.16908 fetch_fsmap learned FSMap version 30 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.366+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561d12ec0d70 con 0x561d12f64c20 2024-02-14T15:11:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.366+0000 7f6b50ff9640 1 -- 172.21.15.17:0/2353329726 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561d12bc2600 con 0x561d12f64c20 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b50ff9640 1 -- 172.21.15.17:0/2353329726 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b60004040 con 0x561d12f64c20 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b71f67740 10 client.16908 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b71f67740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b71f67740 20 client.16908 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12f74a30 0x561d12f76e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b71f67740 4 client.16908 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d12c131b0 con 0x561d12f74a30 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.367+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12f74a30 0x561d12f76e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.368+0000 7f6b6d051640 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12f74a30 0x561d12f76e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92f4c7e1bd457bf server_cookie=225597bf396540ba in_seq=0 out_seq=0 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.368+0000 7f6b50ff9640 10 client.16908 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b50ff9640 1 -- 172.21.15.17:0/2353329726 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d12c131b0 con 0x561d12f74a30 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b50ff9640 10 client.16908 handle_command_reply: tid=0 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 client.16908 shutdown 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 msgr2=0x561d12f61fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f61fd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6b60004af0 tx=0x7f6b60005c50 comp rx=0 tx=0).stop 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 shutdown_connections 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12f74a30 0x561d12f76e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f61fd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5c040 0x561d12f619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 --2- 172.21.15.17:0/2353329726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5b720 0x561d12f64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 >> 172.21.15.17:0/2353329726 conn(0x561d12f55e50 msgr2=0x561d12f57ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b50ff9640 0 client.16908 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 shutdown_connections 2024-02-14T15:11:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 1 -- 172.21.15.17:0/2353329726 wait complete. 2024-02-14T15:11:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.369+0000 7f6b71f67740 20 client.16908 trim_cache size 0 max 16384 2024-02-14T15:11:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.389+0000 7f6b71f67740 1 Processor -- start 2024-02-14T15:11:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b71f67740 1 -- start start 2024-02-14T15:11:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5c040 0x561d12f64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f64c20 0x561d12f54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b71f67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12f71530 con 0x561d12f5c040 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b71f67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12f62960 con 0x561d12f5b720 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b71f67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12f74830 con 0x561d12f64c20 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5c040 0x561d12f64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5c040 0x561d12f64630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43250/0 (socket says 172.21.15.17:43250) 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 -- 172.21.15.17:0/521098575 learned_addr learned my addr 172.21.15.17:0/521098575 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6d051640 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6d852640 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f64c20 0x561d12f54d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 -- 172.21.15.17:0/521098575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f64c20 msgr2=0x561d12f54d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f64c20 0x561d12f54d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 -- 172.21.15.17:0/521098575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 msgr2=0x561d12f640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f640f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6c850640 1 -- 172.21.15.17:0/521098575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d12e5d820 con 0x561d12f5c040 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.390+0000 7f6b6d051640 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.391+0000 7f6b6c850640 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5c040 0x561d12f64630 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6b5809aaf0 tx=0x7f6b5809a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4097b9724fc239ce server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.391+0000 7f6b52ffd640 1 -- 172.21.15.17:0/521098575 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5809ef40 con 0x561d12f5c040 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.391+0000 7f6b52ffd640 1 -- 172.21.15.17:0/521098575 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b580a3590 con 0x561d12f5c040 2024-02-14T15:11:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.391+0000 7f6b52ffd640 1 -- 172.21.15.17:0/521098575 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b580a3100 con 0x561d12f5c040 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 -- 172.21.15.17:0/521098575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5c040 msgr2=0x561d12f64630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5c040 0x561d12f64630 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6b5809aaf0 tx=0x7f6b5809a9f0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 -- 172.21.15.17:0/521098575 shutdown_connections 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f5c040 0x561d12f64630 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f64c20 0x561d12f54d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 --2- 172.21.15.17:0/521098575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 -- 172.21.15.17:0/521098575 >> 172.21.15.17:0/521098575 conn(0x561d12f55e50 msgr2=0x561d12f54630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 -- 172.21.15.17:0/521098575 shutdown_connections 2024-02-14T15:11:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 -- 172.21.15.17:0/521098575 wait complete. 2024-02-14T15:11:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 -- start start 2024-02-14T15:11:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.392+0000 7f6b71f67740 1 Processor -- start 2024-02-14T15:11:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f6a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b71f67740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f6a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b71f67740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12f6ac40 con 0x561d12f64c20 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b71f67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12f6adb0 con 0x561d12f5b720 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b71f67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d12f6af70 con 0x561d12f5c040 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b71f67740 10 client.0 _ll_register_callbacks cb 0x7f6b640097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b6d051640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b6d852640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f6a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b6d051640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f6bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:35176/0 (socket says 172.21.15.17:35176) 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b6d051640 1 -- 172.21.15.17:0/1920914882 learned_addr learned my addr 172.21.15.17:0/1920914882 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.393+0000 7f6b6c850640 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f6a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b6d051640 1 -- 172.21.15.17:0/1920914882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 msgr2=0x561d12f6a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b6d051640 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f6a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b6d051640 1 -- 172.21.15.17:0/1920914882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 msgr2=0x561d12f6a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b6d051640 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f6a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b6d051640 1 -- 172.21.15.17:0/1920914882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d12bd3920 con 0x561d12f5b720 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b6c850640 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f6a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b6d051640 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f6bcd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009c40 tx=0x7f6b64009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a67837b3a3cd62 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b50ff9640 1 -- 172.21.15.17:0/1920914882 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64033070 con 0x561d12f5b720 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b50ff9640 1 -- 172.21.15.17:0/1920914882 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f6b64005250 con 0x561d12f5b720 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b640035f0 con 0x561d12f5b720 2024-02-14T15:11:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b50ff9640 1 -- 172.21.15.17:0/1920914882 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b64005410 con 0x561d12f5b720 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b71f67740 10 client.26273 fetch_fsmap learned FSMap version 30 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.394+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f6b681a6b90 con 0x561d12f5b720 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b50ff9640 1 -- 172.21.15.17:0/1920914882 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b640035f0 con 0x561d12f5b720 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b50ff9640 1 -- 172.21.15.17:0/1920914882 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b64004750 con 0x561d12f5b720 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b71f67740 10 client.26273 fetch_fsmap finished waiting for FSMap version 30 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b71f67740 10 client.26273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b71f67740 20 client.26273 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b71f67740 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12fcc490 0x561d12fce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b71f67740 4 client.26273 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:11:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d12bff550 con 0x561d12fcc490 2024-02-14T15:11:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.395+0000 7f6b6c850640 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12fcc490 0x561d12fce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.396+0000 7f6b50ff9640 10 client.26273 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:01.396+0000 7f6b6c850640 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12fcc490 0x561d12fce8b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330bc93dc96420f server_cookie=70b418493c7e1bf3 in_seq=0 out_seq=0 2024-02-14T15:11:01.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 777789 11.50 MB/sec execute 243 sec latency 73.731 ms 2024-02-14T15:11:02.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 779000 11.47 MB/sec execute 244 sec latency 57.380 ms 2024-02-14T15:11:03.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:02 smithi017 ceph-mon[32549]: pgmap v234: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 298 op/s 2024-02-14T15:11:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:02 smithi050 ceph-mon[36512]: pgmap v234: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 298 op/s 2024-02-14T15:11:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b50ff9640 1 -- 172.21.15.17:0/1920914882 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d12bff550 con 0x561d12fcc490 2024-02-14T15:11:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b50ff9640 10 client.26273 handle_command_reply: tid=0 2024-02-14T15:11:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 client.26273 shutdown 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 msgr2=0x561d12f6bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f6bcd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009c40 tx=0x7f6b64009310 comp rx=0 tx=0).stop 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 shutdown_connections 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561d12fcc490 0x561d12fce8b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561d12f64c20 0x561d12f6a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d12f5c040 0x561d12f6a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b50ff9640 0 client.26273 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 --2- 172.21.15.17:0/1920914882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d12f5b720 0x561d12f6bcd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 >> 172.21.15.17:0/1920914882 conn(0x561d12f55e50 msgr2=0x561d12f54ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 shutdown_connections 2024-02-14T15:11:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 1 -- 172.21.15.17:0/1920914882 wait complete. 2024-02-14T15:11:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.176+0000 7f6b71f67740 20 client.26273 trim_cache size 0 max 16384 2024-02-14T15:11:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:11:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:11:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:11:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:11:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6b540a2250 msgr2=0x7f6b540a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6b540a2250 0x7f6b540a4710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b60005cc0 tx=0x7f6b60032f20 comp rx=0 tx=0).stop 2024-02-14T15:11:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 msgr2=0x7f6b68102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b68102320 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800e9c0 tx=0x7f6b5800ee80 comp rx=0 tx=0).stop 2024-02-14T15:11:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 shutdown_connections 2024-02-14T15:11:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f6b540a2250 0x7f6b540a4710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b68105420 0x7f6b68102320 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b68102860 0x7f6b681a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 --2- 172.21.15.17:0/2056183894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b68100350 0x7f6b68101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 >> 172.21.15.17:0/2056183894 conn(0x7f6b680fbbd0 msgr2=0x7f6b680fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 shutdown_connections 2024-02-14T15:11:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:03.177+0000 7f6b6f2b5640 1 -- 172.21.15.17:0/2056183894 wait complete. 2024-02-14T15:11:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:02 smithi076 ceph-mon[37057]: pgmap v234: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 298 op/s 2024-02-14T15:11:03.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 781244 11.48 MB/sec execute 245 sec latency 23.631 ms 2024-02-14T15:11:04.142 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: *** Caught signal (Segmentation fault) ** 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: in thread 7f1e6c958700 thread_name:md_log_replay 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1e7dd19d20] 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5560eb0e3d5f] 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5560eb0e5f93] 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 4: (MDLog::_replay_thread()+0x647) [0x5560eb1427b7] 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5560eb14c651] 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1e7dd0f1ca] 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 7: clone() 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2024-02-14T15:11:03.855+0000 7f1e6c958700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: in thread 7f1e6c958700 thread_name:md_log_replay 2024-02-14T15:11:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1e7dd19d20] 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5560eb0e3d5f] 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5560eb0e5f93] 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 4: (MDLog::_replay_thread()+0x647) [0x5560eb1427b7] 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5560eb14c651] 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1e7dd0f1ca] 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 7: clone() 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 0> 2024-02-14T15:11:03.855+0000 7f1e6c958700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:11:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: in thread 7f1e6c958700 thread_name:md_log_replay 2024-02-14T15:11:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2024-02-14T15:11:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:11:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1e7dd19d20] 2024-02-14T15:11:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5560eb0e3d5f] 2024-02-14T15:11:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5560eb0e5f93] 2024-02-14T15:11:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 4: (MDLog::_replay_thread()+0x647) [0x5560eb1427b7] 2024-02-14T15:11:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5560eb14c651] 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1e7dd0f1ca] 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 7: clone() 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 0> 2024-02-14T15:11:03.855+0000 7f1e6c958700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: in thread 7f1e6c958700 thread_name:md_log_replay 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1e7dd19d20] 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5560eb0e3d5f] 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5560eb0e5f93] 2024-02-14T15:11:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 4: (MDLog::_replay_thread()+0x647) [0x5560eb1427b7] 2024-02-14T15:11:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5560eb14c651] 2024-02-14T15:11:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1e7dd0f1ca] 2024-02-14T15:11:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 7: clone() 2024-02-14T15:11:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:11:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71256]: 2024-02-14T15:11:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:03 smithi050 podman[71698]: 2024-02-14 15:11:03.915781855 +0000 UTC m=+0.020304990 container died 17713ed48b61dfb22bb8baba90322cb37101ba3e9a21cce306a28b0793a7c9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-14T15:11:04.876 INFO:tasks.workunit.client.0.smithi017.stdout: 1 782658 11.46 MB/sec execute 246 sec latency 68.686 ms 2024-02-14T15:11:04.892 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:04 smithi050 podman[71698]: 2024-02-14 15:11:04.548283164 +0000 UTC m=+0.652806296 container remove 17713ed48b61dfb22bb8baba90322cb37101ba3e9a21cce306a28b0793a7c9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-14T15:11:04.892 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:04 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-02-14T15:11:04.892 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:04 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-02-14T15:11:04.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:04 smithi050 ceph-mon[36512]: pgmap v235: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2024-02-14T15:11:04.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:04 smithi050 ceph-mon[36512]: osdmap e98: 12 total, 12 up, 12 in 2024-02-14T15:11:05.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:04 smithi017 ceph-mon[32549]: pgmap v235: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2024-02-14T15:11:05.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:04 smithi017 ceph-mon[32549]: osdmap e98: 12 total, 12 up, 12 in 2024-02-14T15:11:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:04 smithi076 ceph-mon[37057]: pgmap v235: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2024-02-14T15:11:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:04 smithi076 ceph-mon[37057]: osdmap e98: 12 total, 12 up, 12 in 2024-02-14T15:11:05.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 785091 11.47 MB/sec execute 247 sec latency 15.012 ms 2024-02-14T15:11:06.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 787278 11.45 MB/sec execute 248 sec latency 16.518 ms 2024-02-14T15:11:07.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:06 smithi017 ceph-mon[32549]: pgmap v237: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-02-14T15:11:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:06 smithi050 ceph-mon[36512]: pgmap v237: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-02-14T15:11:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:06 smithi076 ceph-mon[37057]: pgmap v237: 129 pgs: 129 active+clean; 475 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-02-14T15:11:07.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 788951 11.45 MB/sec execute 249 sec latency 42.986 ms 2024-02-14T15:11:08.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 791436 11.46 MB/sec execute 250 sec latency 15.146 ms 2024-02-14T15:11:09.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:08 smithi017 ceph-mon[32549]: pgmap v238: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-02-14T15:11:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:08 smithi050 ceph-mon[36512]: pgmap v238: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-02-14T15:11:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:08 smithi076 ceph-mon[37057]: pgmap v238: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-02-14T15:11:09.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 792955 11.44 MB/sec execute 251 sec latency 101.226 ms 2024-02-14T15:11:10.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 795480 11.46 MB/sec execute 252 sec latency 30.780 ms 2024-02-14T15:11:11.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:10 smithi017 ceph-mon[32549]: pgmap v239: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-02-14T15:11:11.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:10 smithi017 ceph-mon[32549]: osdmap e99: 12 total, 12 up, 12 in 2024-02-14T15:11:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:10 smithi050 ceph-mon[36512]: pgmap v239: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-02-14T15:11:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:10 smithi050 ceph-mon[36512]: osdmap e99: 12 total, 12 up, 12 in 2024-02-14T15:11:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:10 smithi076 ceph-mon[37057]: pgmap v239: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-02-14T15:11:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:10 smithi076 ceph-mon[37057]: osdmap e99: 12 total, 12 up, 12 in 2024-02-14T15:11:11.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 797026 11.44 MB/sec execute 253 sec latency 30.540 ms 2024-02-14T15:11:12.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 799233 11.45 MB/sec execute 254 sec latency 64.714 ms 2024-02-14T15:11:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:12 smithi050 ceph-mon[36512]: pgmap v241: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-02-14T15:11:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:12 smithi076 ceph-mon[37057]: pgmap v241: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-02-14T15:11:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:12 smithi017 ceph-mon[32549]: pgmap v241: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-02-14T15:11:13.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 801122 11.43 MB/sec execute 255 sec latency 35.375 ms 2024-02-14T15:11:14.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 803095 11.44 MB/sec execute 256 sec latency 35.808 ms 2024-02-14T15:11:15.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:14 smithi050 ceph-mon[36512]: pgmap v242: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 283 op/s 2024-02-14T15:11:15.127 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:14 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 2. 2024-02-14T15:11:15.127 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:14 smithi050 systemd[1]: Stopped Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:11:15.127 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:14 smithi050 systemd[1]: Starting Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:11:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:14 smithi076 ceph-mon[37057]: pgmap v242: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 283 op/s 2024-02-14T15:11:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:14 smithi017 ceph-mon[32549]: pgmap v242: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 283 op/s 2024-02-14T15:11:15.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 podman[71800]: 2024-02-14 15:11:15.029831734 +0000 UTC m=+0.019109408 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:11:15.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 podman[71800]: 2024-02-14 15:11:15.141433592 +0000 UTC m=+0.130711254 container create 843e3bb22bc41ce9b425ceae67f2ef39fd0eb15c461c35a4c4e46c9060f313d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-14T15:11:15.774 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 podman[71800]: 2024-02-14 15:11:15.39507956 +0000 UTC m=+0.384357242 container init 843e3bb22bc41ce9b425ceae67f2ef39fd0eb15c461c35a4c4e46c9060f313d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-14T15:11:15.774 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 podman[71800]: 2024-02-14 15:11:15.398120694 +0000 UTC m=+0.387398358 container start 843e3bb22bc41ce9b425ceae67f2ef39fd0eb15c461c35a4c4e46c9060f313d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_CLEAN=True) 2024-02-14T15:11:15.774 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 ceph-mds[71820]: -- 172.21.15.50:0/701289661 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646943f3520 con 0x5646943c9000 2024-02-14T15:11:15.774 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: starting mds.h at 2024-02-14T15:11:15.774 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 bash[71800]: 843e3bb22bc41ce9b425ceae67f2ef39fd0eb15c461c35a4c4e46c9060f313d5 2024-02-14T15:11:15.774 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:11:15 smithi050 systemd[1]: Started Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:11:15.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 805585 11.44 MB/sec execute 257 sec latency 22.052 ms 2024-02-14T15:11:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:15 smithi050 ceph-mon[36512]: daemon mds.h restarted 2024-02-14T15:11:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:15 smithi050 ceph-mon[36512]: osdmap e100: 12 total, 12 up, 12 in 2024-02-14T15:11:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:15 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6836/3593762597,v1:172.21.15.50:6837/3593762597] up:boot 2024-02-14T15:11:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:15 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:11:16.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:15 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:11:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:15 smithi076 ceph-mon[37057]: daemon mds.h restarted 2024-02-14T15:11:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:15 smithi076 ceph-mon[37057]: osdmap e100: 12 total, 12 up, 12 in 2024-02-14T15:11:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:15 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6836/3593762597,v1:172.21.15.50:6837/3593762597] up:boot 2024-02-14T15:11:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:15 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:11:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:15 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:11:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 -- 172.21.15.17:0/3441679309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 msgr2=0x7f206810e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 --2- 172.21.15.17:0/3441679309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206810e0e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2060009cd0 tx=0x7f206002c0f0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:15 smithi017 ceph-mon[32549]: daemon mds.h restarted 2024-02-14T15:11:16.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:15 smithi017 ceph-mon[32549]: osdmap e100: 12 total, 12 up, 12 in 2024-02-14T15:11:16.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:15 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6836/3593762597,v1:172.21.15.50:6837/3593762597] up:boot 2024-02-14T15:11:16.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:15 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:11:16.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:15 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:11:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 -- 172.21.15.17:0/3441679309 shutdown_connections 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 --2- 172.21.15.17:0/3441679309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206810e0e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 --2- 172.21.15.17:0/3441679309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20680696f0 0x7f2068105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 --2- 172.21.15.17:0/3441679309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2068068d40 0x7f2068069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 -- 172.21.15.17:0/3441679309 >> 172.21.15.17:0/3441679309 conn(0x7f2068077220 msgr2=0x7f2068077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 -- 172.21.15.17:0/3441679309 shutdown_connections 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.265+0000 7f206dbc7640 1 -- 172.21.15.17:0/3441679309 wait complete. 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 Processor -- start 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 -- start start 2024-02-14T15:11:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2068068d40 0x7f206819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20681976a0 0x7f2068197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2068198170 con 0x7f2068106510 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20681982e0 con 0x7f20681976a0 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f206dbc7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20681984a0 con 0x7f2068068d40 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206819db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56888/0 (socket says 172.21.15.17:56888) 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f2066ffd640 1 -- 172.21.15.17:0/170164817 learned_addr learned my addr 172.21.15.17:0/170164817 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.266+0000 7f20677fe640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2068068d40 0x7f206819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2067fff640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20681976a0 0x7f2068197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2066ffd640 1 -- 172.21.15.17:0/170164817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2068068d40 msgr2=0x7f206819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2066ffd640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2068068d40 0x7f206819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2066ffd640 1 -- 172.21.15.17:0/170164817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20681976a0 msgr2=0x7f2068197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2066ffd640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20681976a0 0x7f2068197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2066ffd640 1 -- 172.21.15.17:0/170164817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2060009900 con 0x7f2068106510 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2066ffd640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206819db20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f205c00e9f0 tx=0x7f205c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2064ff9640 1 -- 172.21.15.17:0/170164817 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0096c0 con 0x7f2068106510 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2064ff9640 1 -- 172.21.15.17:0/170164817 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f205c005500 con 0x7f2068106510 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20681a4440 con 0x7f2068106510 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f2064ff9640 1 -- 172.21.15.17:0/170164817 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c010430 con 0x7f2068106510 2024-02-14T15:11:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.267+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20681a4920 con 0x7f2068106510 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.269+0000 7f2064ff9640 1 -- 172.21.15.17:0/170164817 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f205c005020 con 0x7f2068106510 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 Processor -- start 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 -- start start 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2064ff9640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20500a22a0 0x7f20500a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c361b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c36a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c36ac20 0x556f6c36d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c2cf9e0 con 0x556f6c36ac20 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2067fff640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20500a22a0 0x7f20500a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c2ccf10 con 0x556f6c362040 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2070793740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c2c86f0 con 0x556f6c361720 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2064ff9640 1 -- 172.21.15.17:0/170164817 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f205c014070 con 0x7f2068106510 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c361b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2067fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c36ac20 0x556f6c36d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c361b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:35566/0 (socket says 172.21.15.17:35566) 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 -- 172.21.15.17:0/1942520337 learned_addr learned my addr 172.21.15.17:0/1942520337 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2066ffd640 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c36a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 -- 172.21.15.17:0/1942520337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 msgr2=0x556f6c36a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c36a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 -- 172.21.15.17:0/1942520337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c36ac20 msgr2=0x556f6c36d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c36ac20 0x556f6c36d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f20677fe640 1 -- 172.21.15.17:0/1942520337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6bfd9920 con 0x556f6c361720 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.270+0000 7f2066ffd640 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c36a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.271+0000 7f2067fff640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20500a22a0 0x7f20500a4760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2060006010 tx=0x7f2060032e80 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.271+0000 7f20677fe640 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c361b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2058009cf0 tx=0x7f2058011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e89a76a0c75e2c9f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.271+0000 7f204dffb640 1 -- 172.21.15.17:0/1942520337 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058009de0 con 0x556f6c361720 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.271+0000 7f204dffb640 1 -- 172.21.15.17:0/1942520337 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20580035f0 con 0x556f6c361720 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.271+0000 7f204dffb640 1 -- 172.21.15.17:0/1942520337 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058019650 con 0x556f6c361720 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.271+0000 7f2070793740 1 -- 172.21.15.17:0/1942520337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 msgr2=0x556f6c361b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.271+0000 7f2070793740 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c361b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2058009cf0 tx=0x7f2058011810 comp rx=0 tx=0).stop 2024-02-14T15:11:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- 172.21.15.17:0/1942520337 shutdown_connections 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c36ac20 0x556f6c36d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c36a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 --2- 172.21.15.17:0/1942520337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c361b00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- 172.21.15.17:0/1942520337 >> 172.21.15.17:0/1942520337 conn(0x556f6c35be50 msgr2=0x556f6c35c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- 172.21.15.17:0/1942520337 shutdown_connections 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- 172.21.15.17:0/1942520337 wait complete. 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- start start 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 Processor -- start 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c361720 0x556f6c36a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c367930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c367eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c264fa0 con 0x556f6c362040 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6bf9c250 con 0x556f6c361720 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c24e200 con 0x556f6c36ac20 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2070793740 10 client.0 _ll_register_callbacks cb 0x556f6c2bdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.272+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c367930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c367930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56902/0 (socket says 172.21.15.17:56902) 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 -- 172.21.15.17:0/3335549567 learned_addr learned my addr 172.21.15.17:0/3335549567 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2067fff640 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c367eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f20677fe640 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c361720 0x556f6c36a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 -- 172.21.15.17:0/3335549567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 msgr2=0x556f6c367eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c367eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 -- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c361720 msgr2=0x556f6c36a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c361720 0x556f6c36a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 -- 172.21.15.17:0/3335549567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2058009950 con 0x556f6c362040 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2066ffd640 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c367930 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f205c09ad40 tx=0x7f205c09b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f9bf1a5536a88 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2046ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2046ffd640 1 -- 172.21.15.17:0/3335549567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c09ef00 con 0x556f6c362040 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2046ffd640 1 -- 172.21.15.17:0/3335549567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f205c0a3550 con 0x556f6c362040 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f6bfc8600 con 0x556f6c362040 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2046ffd640 1 -- 172.21.15.17:0/3335549567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f205c00ccc0 con 0x556f6c362040 2024-02-14T15:11:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2070793740 10 client.16932 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.273+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556f6c2c6d70 con 0x556f6c362040 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2046ffd640 1 -- 172.21.15.17:0/3335549567 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556f6bfc8600 con 0x556f6c362040 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2046ffd640 1 -- 172.21.15.17:0/3335549567 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f205c01f060 con 0x556f6c362040 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2070793740 10 client.16932 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2070793740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2070793740 20 client.16932 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2070793740 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c37a9f0 0x556f6c37cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2070793740 4 client.16932 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f6c0191b0 con 0x556f6c37a9f0 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f20677fe640 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c37a9f0 0x556f6c37cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f2046ffd640 10 client.16932 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.274+0000 7f20677fe640 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c37a9f0 0x556f6c37cde0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64884c71ce85dcac server_cookie=e951423d86665fbd in_seq=0 out_seq=0 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2046ffd640 1 -- 172.21.15.17:0/3335549567 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f6c0191b0 con 0x556f6c37a9f0 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2046ffd640 10 client.16932 handle_command_reply: tid=0 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 client.16932 shutdown 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 msgr2=0x556f6c367930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c367930 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f205c09ad40 tx=0x7f205c09b570 comp rx=0 tx=0).stop 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 shutdown_connections 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c37a9f0 0x556f6c37cde0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c367930 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c367eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2046ffd640 0 client.16932 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 --2- 172.21.15.17:0/3335549567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c361720 0x556f6c36a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 >> 172.21.15.17:0/3335549567 conn(0x556f6c35be50 msgr2=0x556f6c35dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 shutdown_connections 2024-02-14T15:11:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.275+0000 7f2070793740 1 -- 172.21.15.17:0/3335549567 wait complete. 2024-02-14T15:11:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.276+0000 7f2070793740 20 client.16932 trim_cache size 0 max 16384 2024-02-14T15:11:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 Processor -- start 2024-02-14T15:11:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 -- start start 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c36a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c36a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c36ac20 0x556f6c364bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c376e30 con 0x556f6c362040 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c37a7f0 con 0x556f6c36ac20 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2070793740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c35a2d0 con 0x556f6c361720 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c36a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c36a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56918/0 (socket says 172.21.15.17:56918) 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2067fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c36ac20 0x556f6c364bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f20677fe640 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c36a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 -- 172.21.15.17:0/2632162116 learned_addr learned my addr 172.21.15.17:0/2632162116 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 -- 172.21.15.17:0/2632162116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 msgr2=0x556f6c36a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c36a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 -- 172.21.15.17:0/2632162116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c36ac20 msgr2=0x556f6c364bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c36ac20 0x556f6c364bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 -- 172.21.15.17:0/2632162116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6c263820 con 0x556f6c362040 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2067fff640 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c36ac20 0x556f6c364bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.289+0000 7f2066ffd640 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c36a5a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f205c099060 tx=0x7f205c09af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d0bd0e7515bac1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f204d7fa640 1 -- 172.21.15.17:0/2632162116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c005670 con 0x556f6c362040 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f204d7fa640 1 -- 172.21.15.17:0/2632162116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f205c0a3550 con 0x556f6c362040 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f204d7fa640 1 -- 172.21.15.17:0/2632162116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c099140 con 0x556f6c362040 2024-02-14T15:11:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 -- 172.21.15.17:0/2632162116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 msgr2=0x556f6c36a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c36a5a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f205c099060 tx=0x7f205c09af30 comp rx=0 tx=0).stop 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 -- 172.21.15.17:0/2632162116 shutdown_connections 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c362040 0x556f6c36a5a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c36ac20 0x556f6c364bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 --2- 172.21.15.17:0/2632162116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c361720 0x556f6c36a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 -- 172.21.15.17:0/2632162116 >> 172.21.15.17:0/2632162116 conn(0x556f6c35be50 msgr2=0x556f6c366200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 -- 172.21.15.17:0/2632162116 shutdown_connections 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.290+0000 7f2070793740 1 -- 172.21.15.17:0/2632162116 wait complete. 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 -- start start 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 Processor -- start 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c361720 0x556f6c371d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c3701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c370730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c370c70 con 0x556f6c361720 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c370de0 con 0x556f6c362040 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6c370fa0 con 0x556f6c36ac20 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2070793740 10 client.0 _ll_register_callbacks cb 0x7f20580097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c361720 0x556f6c371d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c361720 0x556f6c371d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56926/0 (socket says 172.21.15.17:56926) 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2067fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c370730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 -- 172.21.15.17:0/849674659 learned_addr learned my addr 172.21.15.17:0/849674659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f2066ffd640 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c3701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 -- 172.21.15.17:0/849674659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 msgr2=0x556f6c370730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c370730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 -- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 msgr2=0x556f6c3701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c3701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.291+0000 7f20677fe640 1 -- 172.21.15.17:0/849674659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2060049040 con 0x556f6c361720 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f20677fe640 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c361720 0x556f6c371d00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2058009c40 tx=0x7f20580053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad83166c3c81517 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2046ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2046ffd640 1 -- 172.21.15.17:0/849674659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058009310 con 0x556f6c361720 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2046ffd640 1 -- 172.21.15.17:0/849674659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f2058003760 con 0x556f6c361720 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20580035f0 con 0x556f6c361720 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2046ffd640 1 -- 172.21.15.17:0/849674659 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2058003920 con 0x556f6c361720 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 10 client.16938 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556f6c2c6d70 con 0x556f6c361720 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2046ffd640 1 -- 172.21.15.17:0/849674659 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20580035f0 con 0x556f6c361720 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2046ffd640 1 -- 172.21.15.17:0/849674659 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2058025c30 con 0x556f6c361720 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 10 client.16938 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 20 client.16938 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c3d2450 0x556f6c3d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 4 client.16938 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f6c005550 con 0x556f6c3d2450 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.292+0000 7f2066ffd640 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c3d2450 0x556f6c3d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.293+0000 7f2066ffd640 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c3d2450 0x556f6c3d4870 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9cc03347ad080f server_cookie=c96edbb5f47f944 in_seq=0 out_seq=0 2024-02-14T15:11:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.293+0000 7f2046ffd640 10 client.16938 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.303+0000 7f2046ffd640 1 -- 172.21.15.17:0/849674659 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f6c005550 con 0x556f6c3d2450 2024-02-14T15:11:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.303+0000 7f2046ffd640 10 client.16938 handle_command_reply: tid=0 2024-02-14T15:11:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.303+0000 7f2070793740 1 client.16938 shutdown 2024-02-14T15:11:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c361720 msgr2=0x556f6c371d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c361720 0x556f6c371d00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2058009c40 tx=0x7f20580053b0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 shutdown_connections 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556f6c3d2450 0x556f6c3d4870 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f6c36ac20 0x556f6c370730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f6c362040 0x556f6c3701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 --2- 172.21.15.17:0/849674659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556f6c361720 0x556f6c371d00 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2046ffd640 0 client.16938 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 >> 172.21.15.17:0/849674659 conn(0x556f6c35be50 msgr2=0x556f6c35d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 shutdown_connections 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 1 -- 172.21.15.17:0/849674659 wait complete. 2024-02-14T15:11:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f2070793740 20 client.16938 trim_cache size 0 max 16384 2024-02-14T15:11:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:11:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:11:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:11:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:11:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20500a22a0 msgr2=0x7f20500a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20500a22a0 0x7f20500a4760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2060006010 tx=0x7f2060032e80 comp rx=0 tx=0).stop 2024-02-14T15:11:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 msgr2=0x7f206819db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206819db20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f205c00e9f0 tx=0x7f205c00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 shutdown_connections 2024-02-14T15:11:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20500a22a0 0x7f20500a4760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2068106510 0x7f206819db20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20681976a0 0x7f2068197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 --2- 172.21.15.17:0/170164817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2068068d40 0x7f206819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.304+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 >> 172.21.15.17:0/170164817 conn(0x7f2068077220 msgr2=0x7f20681076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.305+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 shutdown_connections 2024-02-14T15:11:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:16.305+0000 7f206dbc7640 1 -- 172.21.15.17:0/170164817 wait complete. 2024-02-14T15:11:16.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 807407 11.44 MB/sec execute 258 sec latency 28.423 ms 2024-02-14T15:11:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:16 smithi050 ceph-mon[36512]: pgmap v243: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-02-14T15:11:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:16 smithi076 ceph-mon[37057]: pgmap v243: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-02-14T15:11:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:16 smithi017 ceph-mon[32549]: pgmap v243: 129 pgs: 129 active+clean; 457 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-02-14T15:11:17.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 809715 11.45 MB/sec execute 259 sec latency 77.271 ms 2024-02-14T15:11:18.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 811197 11.43 MB/sec execute 260 sec latency 45.449 ms 2024-02-14T15:11:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:18 smithi050 ceph-mon[36512]: pgmap v245: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 295 op/s 2024-02-14T15:11:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:18 smithi076 ceph-mon[37057]: pgmap v245: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 295 op/s 2024-02-14T15:11:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:18 smithi017 ceph-mon[32549]: pgmap v245: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.6 MiB/s wr, 295 op/s 2024-02-14T15:11:19.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 813332 11.44 MB/sec execute 261 sec latency 79.101 ms 2024-02-14T15:11:20.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 815184 11.42 MB/sec execute 262 sec latency 41.759 ms 2024-02-14T15:11:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:20 smithi050 ceph-mon[36512]: pgmap v246: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 283 op/s 2024-02-14T15:11:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:20 smithi076 ceph-mon[37057]: pgmap v246: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 283 op/s 2024-02-14T15:11:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:20 smithi017 ceph-mon[32549]: pgmap v246: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 283 op/s 2024-02-14T15:11:21.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 817282 11.42 MB/sec execute 263 sec latency 16.824 ms 2024-02-14T15:11:22.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 819787 11.43 MB/sec execute 264 sec latency 31.644 ms 2024-02-14T15:11:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:22 smithi050 ceph-mon[36512]: pgmap v247: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:22 smithi076 ceph-mon[37057]: pgmap v247: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:22 smithi017 ceph-mon[32549]: pgmap v247: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:23.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 821314 11.42 MB/sec execute 265 sec latency 104.233 ms 2024-02-14T15:11:24.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 823840 11.43 MB/sec execute 266 sec latency 20.551 ms 2024-02-14T15:11:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:24 smithi050 ceph-mon[36512]: pgmap v248: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:24 smithi076 ceph-mon[37057]: pgmap v248: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:24 smithi017 ceph-mon[32549]: pgmap v248: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:25.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 825599 11.42 MB/sec execute 267 sec latency 25.418 ms 2024-02-14T15:11:26.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 827894 11.42 MB/sec execute 268 sec latency 19.663 ms 2024-02-14T15:11:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:26 smithi050 ceph-mon[36512]: pgmap v249: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:26 smithi076 ceph-mon[37057]: pgmap v249: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:26 smithi017 ceph-mon[32549]: pgmap v249: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-02-14T15:11:27.877 INFO:tasks.workunit.client.0.smithi017.stdout: 1 830214 11.43 MB/sec execute 269 sec latency 23.989 ms 2024-02-14T15:11:28.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 831565 11.41 MB/sec execute 270 sec latency 123.972 ms 2024-02-14T15:11:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:28 smithi050 ceph-mon[36512]: pgmap v250: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 317 op/s 2024-02-14T15:11:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:28 smithi076 ceph-mon[37057]: pgmap v250: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 317 op/s 2024-02-14T15:11:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:28 smithi017 ceph-mon[32549]: pgmap v250: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 317 op/s 2024-02-14T15:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 -- 172.21.15.17:0/291458726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 msgr2=0x7fa680100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 --2- 172.21.15.17:0/291458726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa680100730 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009930 tx=0x7fa67c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 -- 172.21.15.17:0/291458726 shutdown_connections 2024-02-14T15:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 --2- 172.21.15.17:0/291458726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680105420 0x7fa68010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 --2- 172.21.15.17:0/291458726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa680100d00 0x7fa680104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 --2- 172.21.15.17:0/291458726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa680100730 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 -- 172.21.15.17:0/291458726 >> 172.21.15.17:0/291458726 conn(0x7fa6800fbbd0 msgr2=0x7fa6800fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 -- 172.21.15.17:0/291458726 shutdown_connections 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.396+0000 7fa68798c640 1 -- 172.21.15.17:0/291458726 wait complete. 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 Processor -- start 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 -- start start 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa6801986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680105420 0x7fa680198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6801991b0 0x7fa6801a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801a3e00 con 0x7fa680100350 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801a3f70 con 0x7fa680105420 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa68798c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6801a4130 con 0x7fa6801991b0 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa6801986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa6801986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42236/0 (socket says 172.21.15.17:42236) 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 -- 172.21.15.17:0/2120230800 learned_addr learned my addr 172.21.15.17:0/2120230800 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685f29640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6801991b0 0x7fa6801a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa684f27640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680105420 0x7fa680198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 -- 172.21.15.17:0/2120230800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6801991b0 msgr2=0x7fa6801a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6801991b0 0x7fa6801a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 -- 172.21.15.17:0/2120230800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680105420 msgr2=0x7fa680198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680105420 0x7fa680198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.397+0000 7fa685728640 1 -- 172.21.15.17:0/2120230800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c009560 con 0x7fa680100350 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.398+0000 7fa685728640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa6801986e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009230 tx=0x7fa67c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.398+0000 7fa6767fc640 1 -- 172.21.15.17:0/2120230800 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c004350 con 0x7fa680100350 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.398+0000 7fa6767fc640 1 -- 172.21.15.17:0/2120230800 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67c0044b0 con 0x7fa680100350 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.398+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6801a43b0 con 0x7fa680100350 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.398+0000 7fa6767fc640 1 -- 172.21.15.17:0/2120230800 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c003d00 con 0x7fa680100350 2024-02-14T15:11:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.398+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6801a4910 con 0x7fa680100350 2024-02-14T15:11:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.400+0000 7fa6767fc640 1 -- 172.21.15.17:0/2120230800 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa67c004df0 con 0x7fa680100350 2024-02-14T15:11:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 Processor -- start 2024-02-14T15:11:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 -- start start 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa6767fc640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa66c0a22a0 0x7fa66c0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b07b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b08040 0x562c58b106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b13010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58a759e0 con 0x562c58b07720 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58a72f10 con 0x562c58b08040 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58a6e6f0 con 0x562c58b10c20 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685f29640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa66c0a22a0 0x7fa66c0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa6767fc640 1 -- 172.21.15.17:0/2120230800 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa67c0f0290 con 0x7fa680100350 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b07b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685f29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b13010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa684f27640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b08040 0x562c58b106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b07b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42242/0 (socket says 172.21.15.17:42242) 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685f29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b13010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:36392/0 (socket says 172.21.15.17:36392) 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 -- 172.21.15.17:0/3157533192 learned_addr learned my addr 172.21.15.17:0/3157533192 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 -- 172.21.15.17:0/3157533192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 msgr2=0x562c58b13010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b13010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 -- 172.21.15.17:0/3157533192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b08040 msgr2=0x562c58b106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b08040 0x562c58b106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 -- 172.21.15.17:0/3157533192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c5877f920 con 0x562c58b07720 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa684f27640 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b08040 0x562c58b106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685f29640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa66c0a22a0 0x7fa66c0a4760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa678009fd0 tx=0x7fa678009d80 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.401+0000 7fa685728640 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b07b00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0bc9c0 tx=0x7fa67c0bc450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5463190d1a7841ea server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa66b7fe640 1 -- 172.21.15.17:0/3157533192 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0bca30 con 0x562c58b07720 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa66b7fe640 1 -- 172.21.15.17:0/3157533192 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67c0c0b80 con 0x562c58b07720 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa66b7fe640 1 -- 172.21.15.17:0/3157533192 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0056a0 con 0x562c58b07720 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa68a1fd740 1 -- 172.21.15.17:0/3157533192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 msgr2=0x562c58b07b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b07b00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0bc9c0 tx=0x7fa67c0bc450 comp rx=0 tx=0).stop 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa68a1fd740 1 -- 172.21.15.17:0/3157533192 shutdown_connections 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b13010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b08040 0x562c58b106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/3157533192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b07b00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.402+0000 7fa68a1fd740 1 -- 172.21.15.17:0/3157533192 >> 172.21.15.17:0/3157533192 conn(0x562c58b01e50 msgr2=0x562c58b02230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 -- 172.21.15.17:0/3157533192 shutdown_connections 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 -- 172.21.15.17:0/3157533192 wait complete. 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 -- start start 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 Processor -- start 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b0d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58a0afa0 con 0x562c58b08040 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58742250 con 0x562c58b07720 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c589f4200 con 0x562c58b10c20 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa68a1fd740 10 client.0 _ll_register_callbacks cb 0x562c58a63b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.403+0000 7fa684f27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b0d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa685f29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b0d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42252/0 (socket says 172.21.15.17:42252) 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 -- 172.21.15.17:0/1178092097 learned_addr learned my addr 172.21.15.17:0/1178092097 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa685728640 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 -- 172.21.15.17:0/1178092097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 msgr2=0x562c58b0def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 -- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 msgr2=0x562c58b105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 -- 172.21.15.17:0/1178092097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6801a4910 con 0x562c58b08040 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa685f29640 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa684f27640 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b0d950 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa670012700 tx=0x7fa670012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d22aa2329399f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa668ff9640 1 -- 172.21.15.17:0/1178092097 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670017070 con 0x562c58b08040 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa668ff9640 1 -- 172.21.15.17:0/1178092097 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa670004050 con 0x562c58b08040 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c5876e600 con 0x562c58b08040 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa668ff9640 1 -- 172.21.15.17:0/1178092097 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa670004210 con 0x562c58b08040 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa68a1fd740 10 client.16950 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.404+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562c58a6cd70 con 0x562c58b08040 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa668ff9640 1 -- 172.21.15.17:0/1178092097 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562c5876e600 con 0x562c58b08040 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa668ff9640 1 -- 172.21.15.17:0/1178092097 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa67000f6b0 con 0x562c58b08040 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa68a1fd740 10 client.16950 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa68a1fd740 10 client.16950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa68a1fd740 20 client.16950 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b22c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa68a1fd740 4 client.16950 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c587bf1b0 con 0x562c58b20860 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa685728640 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b22c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa685728640 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b22c80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d968c1017335b76 server_cookie=e91177074efa89d in_seq=0 out_seq=0 2024-02-14T15:11:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.405+0000 7fa668ff9640 10 client.16950 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa668ff9640 1 -- 172.21.15.17:0/1178092097 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c587bf1b0 con 0x562c58b20860 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa668ff9640 10 client.16950 handle_command_reply: tid=0 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 client.16950 shutdown 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 msgr2=0x562c58b0d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b0d950 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa670012700 tx=0x7fa670012b00 comp rx=0 tx=0).stop 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 shutdown_connections 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b22c80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b0d950 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa668ff9640 0 client.16950 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/1178092097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 >> 172.21.15.17:0/1178092097 conn(0x562c58b01e50 msgr2=0x562c58b03ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 shutdown_connections 2024-02-14T15:11:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 1 -- 172.21.15.17:0/1178092097 wait complete. 2024-02-14T15:11:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.406+0000 7fa68a1fd740 20 client.16950 trim_cache size 0 max 16384 2024-02-14T15:11:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 Processor -- start 2024-02-14T15:11:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 -- start start 2024-02-14T15:11:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b10010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b10550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58b06ec0 con 0x562c58b08040 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58b07380 con 0x562c58b07720 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58b27ee0 con 0x562c58b10c20 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b10550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b10550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42264/0 (socket says 172.21.15.17:42264) 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 -- 172.21.15.17:0/515117085 learned_addr learned my addr 172.21.15.17:0/515117085 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa685f29640 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa685728640 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b10010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 -- 172.21.15.17:0/515117085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 msgr2=0x562c58b0d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 -- 172.21.15.17:0/515117085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 msgr2=0x562c58b10010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b10010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa684f27640 1 -- 172.21.15.17:0/515117085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c58a09820 con 0x562c58b08040 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.420+0000 7fa685728640 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b10010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa684f27640 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b10550 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa67000fa90 tx=0x7fa670009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40fc3ba1db987a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa66affd640 1 -- 172.21.15.17:0/515117085 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670017040 con 0x562c58b08040 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa66affd640 1 -- 172.21.15.17:0/515117085 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6700055b0 con 0x562c58b08040 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa66affd640 1 -- 172.21.15.17:0/515117085 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670003b20 con 0x562c58b08040 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa68a1fd740 1 -- 172.21.15.17:0/515117085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 msgr2=0x562c58b10550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b10550 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa67000fa90 tx=0x7fa670009620 comp rx=0 tx=0).stop 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa68a1fd740 1 -- 172.21.15.17:0/515117085 shutdown_connections 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b08040 0x562c58b10550 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b10c20 0x562c58b0d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/515117085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b07720 0x562c58b10010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.421+0000 7fa68a1fd740 1 -- 172.21.15.17:0/515117085 >> 172.21.15.17:0/515117085 conn(0x562c58b01e50 msgr2=0x562c58b0c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 -- 172.21.15.17:0/515117085 shutdown_connections 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 -- 172.21.15.17:0/515117085 wait complete. 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 -- start start 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 Processor -- start 2024-02-14T15:11:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b15e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b08040 0x562c58b163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b10c20 0x562c58b6ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58b6f280 con 0x562c58b07720 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58b6f3f0 con 0x562c58b10c20 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c58b6f5b0 con 0x562c58b08040 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa68a1fd740 10 client.0 _ll_register_callbacks cb 0x7fa67c04afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa685728640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b15e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa685728640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b15e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42266/0 (socket says 172.21.15.17:42266) 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa685728640 1 -- 172.21.15.17:0/2410885691 learned_addr learned my addr 172.21.15.17:0/2410885691 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa684f27640 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b08040 0x562c58b163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.422+0000 7fa685f29640 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b10c20 0x562c58b6ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa685728640 1 -- 172.21.15.17:0/2410885691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b08040 msgr2=0x562c58b163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa685728640 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b08040 0x562c58b163a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa685728640 1 -- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b10c20 msgr2=0x562c58b6ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa685728640 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b10c20 0x562c58b6ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa685728640 1 -- 172.21.15.17:0/2410885691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c58a6cd70 con 0x562c58b07720 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa685728640 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b15e60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0b9400 tx=0x7fa67c0b8e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdfd6aab98f03ae server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa668ff9640 1 -- 172.21.15.17:0/2410885691 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0bab50 con 0x562c58b07720 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa668ff9640 1 -- 172.21.15.17:0/2410885691 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fa67c0bca30 con 0x562c58b07720 2024-02-14T15:11:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa67c0c0b80 con 0x562c58b07720 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa668ff9640 1 -- 172.21.15.17:0/2410885691 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa67c0055d0 con 0x562c58b07720 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 10 client.16956 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562c58a2f720 con 0x562c58b07720 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa668ff9640 1 -- 172.21.15.17:0/2410885691 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67c0c0b80 con 0x562c58b07720 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa668ff9640 1 -- 172.21.15.17:0/2410885691 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa67c03c070 con 0x562c58b07720 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 10 client.16956 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 20 client.16956 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b79c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 4 client.16956 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:11:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.423+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c587ab550 con 0x562c58b20860 2024-02-14T15:11:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.424+0000 7fa684f27640 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b79c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.424+0000 7fa668ff9640 10 client.16956 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.424+0000 7fa684f27640 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b79c60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee1cfc27a96f70d server_cookie=1858378c2deed8b in_seq=0 out_seq=0 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa668ff9640 1 -- 172.21.15.17:0/2410885691 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c587ab550 con 0x562c58b20860 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa668ff9640 10 client.16956 handle_command_reply: tid=0 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 client.16956 shutdown 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 msgr2=0x562c58b15e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b15e60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0b9400 tx=0x7fa67c0b8e00 comp rx=0 tx=0).stop 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 shutdown_connections 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x562c58b20860 0x562c58b79c60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c58b10c20 0x562c58b6ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c58b08040 0x562c58b163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa668ff9640 0 client.16956 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 --2- 172.21.15.17:0/2410885691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562c58b07720 0x562c58b15e60 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 >> 172.21.15.17:0/2410885691 conn(0x562c58b01e50 msgr2=0x562c58b0d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 shutdown_connections 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 1 -- 172.21.15.17:0/2410885691 wait complete. 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.435+0000 7fa68a1fd740 20 client.16956 trim_cache size 0 max 16384 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:11:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:11:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa66c0a22a0 msgr2=0x7fa66c0a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa66c0a22a0 0x7fa66c0a4760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa678009fd0 tx=0x7fa678009d80 comp rx=0 tx=0).stop 2024-02-14T15:11:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 msgr2=0x7fa6801986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa6801986e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009230 tx=0x7fa67c02c650 comp rx=0 tx=0).stop 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 shutdown_connections 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fa66c0a22a0 0x7fa66c0a4760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6801991b0 0x7fa6801a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa680105420 0x7fa680198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 --2- 172.21.15.17:0/2120230800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa680100350 0x7fa6801986e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 >> 172.21.15.17:0/2120230800 conn(0x7fa6800fbbd0 msgr2=0x7fa6800fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 shutdown_connections 2024-02-14T15:11:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:29.436+0000 7fa68798c640 1 -- 172.21.15.17:0/2120230800 wait complete. 2024-02-14T15:11:29.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 833785 11.41 MB/sec execute 271 sec latency 23.251 ms 2024-02-14T15:11:30.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 835502 11.40 MB/sec execute 272 sec latency 39.120 ms 2024-02-14T15:11:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:30 smithi050 ceph-mon[36512]: pgmap v251: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-02-14T15:11:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:30 smithi076 ceph-mon[37057]: pgmap v251: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-02-14T15:11:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:30 smithi017 ceph-mon[32549]: pgmap v251: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-02-14T15:11:31.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 837564 11.40 MB/sec execute 273 sec latency 172.979 ms 2024-02-14T15:11:32.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 839342 11.40 MB/sec execute 274 sec latency 17.547 ms 2024-02-14T15:11:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:32 smithi050 ceph-mon[36512]: pgmap v252: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 299 op/s 2024-02-14T15:11:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:32 smithi076 ceph-mon[37057]: pgmap v252: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 299 op/s 2024-02-14T15:11:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:32 smithi017 ceph-mon[32549]: pgmap v252: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 299 op/s 2024-02-14T15:11:33.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 841684 11.41 MB/sec execute 275 sec latency 96.039 ms 2024-02-14T15:11:34.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 843309 11.39 MB/sec execute 276 sec latency 20.432 ms 2024-02-14T15:11:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:34 smithi050 ceph-mon[36512]: pgmap v253: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:34 smithi076 ceph-mon[37057]: pgmap v253: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:34 smithi017 ceph-mon[32549]: pgmap v253: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:35.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 845554 11.40 MB/sec execute 277 sec latency 24.756 ms 2024-02-14T15:11:36.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 847488 11.38 MB/sec execute 278 sec latency 89.145 ms 2024-02-14T15:11:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:36 smithi050 ceph-mon[36512]: pgmap v254: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:36 smithi076 ceph-mon[37057]: pgmap v254: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:36 smithi017 ceph-mon[32549]: pgmap v254: 129 pgs: 129 active+clean; 498 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-02-14T15:11:37.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 849048 11.38 MB/sec execute 279 sec latency 14.133 ms 2024-02-14T15:11:38.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:37 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:11:38.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:37 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:11:38.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:37 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:11:38.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:37 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:11:38.167 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:37 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:11:38.167 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:37 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:11:38.167 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:37 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:11:38.167 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:37 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:11:38.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:37 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:11:38.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:37 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:11:38.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:37 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:11:38.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:37 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:11:38.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 850999 11.37 MB/sec execute 280 sec latency 80.406 ms 2024-02-14T15:11:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:38 smithi050 ceph-mon[36512]: pgmap v255: 129 pgs: 129 active+clean; 508 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-02-14T15:11:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:38 smithi076 ceph-mon[37057]: pgmap v255: 129 pgs: 129 active+clean; 508 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-02-14T15:11:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:38 smithi017 ceph-mon[32549]: pgmap v255: 129 pgs: 129 active+clean; 508 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 291 op/s 2024-02-14T15:11:39.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 852557 11.37 MB/sec execute 281 sec latency 20.415 ms 2024-02-14T15:11:40.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 854871 11.37 MB/sec execute 282 sec latency 28.682 ms 2024-02-14T15:11:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:40 smithi050 ceph-mon[36512]: pgmap v256: 129 pgs: 129 active+clean; 508 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-02-14T15:11:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:40 smithi076 ceph-mon[37057]: pgmap v256: 129 pgs: 129 active+clean; 508 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-02-14T15:11:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:40 smithi017 ceph-mon[32549]: pgmap v256: 129 pgs: 129 active+clean; 508 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-02-14T15:11:41.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 856320 11.36 MB/sec execute 283 sec latency 53.070 ms 2024-02-14T15:11:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 -- 172.21.15.17:0/472734851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60106540 msgr2=0x7f8e6010e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 --2- 172.21.15.17:0/472734851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60106540 0x7f8e6010e110 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009cd0 tx=0x7f8e5802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 -- 172.21.15.17:0/472734851 shutdown_connections 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 --2- 172.21.15.17:0/472734851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60106540 0x7f8e6010e110 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 --2- 172.21.15.17:0/472734851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e60101e20 0x7f8e60105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 --2- 172.21.15.17:0/472734851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e60101470 0x7f8e60101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 -- 172.21.15.17:0/472734851 >> 172.21.15.17:0/472734851 conn(0x7f8e600fcd50 msgr2=0x7f8e600ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 -- 172.21.15.17:0/472734851 shutdown_connections 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.526+0000 7f8e68481640 1 -- 172.21.15.17:0/472734851 wait complete. 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 Processor -- start 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 -- start start 2024-02-14T15:11:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60101470 0x7f8e60073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e60106540 0x7f8e60073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e60073fb0 0x7f8e601a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e601a6030 con 0x7f8e60101470 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e601a61a0 con 0x7f8e60073fb0 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e68481640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e601a6360 con 0x7f8e60106540 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60101470 0x7f8e60073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60101470 0x7f8e60073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44698/0 (socket says 172.21.15.17:44698) 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 -- 172.21.15.17:0/3965592280 learned_addr learned my addr 172.21.15.17:0/3965592280 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e60106540 0x7f8e60073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e66a1e640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e60073fb0 0x7f8e601a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 -- 172.21.15.17:0/3965592280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e60106540 msgr2=0x7f8e60073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e60106540 0x7f8e60073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 -- 172.21.15.17:0/3965592280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e60073fb0 msgr2=0x7f8e601a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e60073fb0 0x7f8e601a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 -- 172.21.15.17:0/3965592280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e50009490 con 0x7f8e60101470 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.527+0000 7f8e6621d640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60101470 0x7f8e60073530 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e9f0 tx=0x7f8e5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.528+0000 7f8e577fe640 1 -- 172.21.15.17:0/3965592280 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c009720 con 0x7f8e60101470 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.528+0000 7f8e577fe640 1 -- 172.21.15.17:0/3965592280 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e5c00cd80 con 0x7f8e60101470 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.528+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e58009900 con 0x7f8e60101470 2024-02-14T15:11:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.528+0000 7f8e577fe640 1 -- 172.21.15.17:0/3965592280 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c010690 con 0x7f8e60101470 2024-02-14T15:11:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.528+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e601a69c0 con 0x7f8e60101470 2024-02-14T15:11:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.530+0000 7f8e577fe640 1 -- 172.21.15.17:0/3965592280 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8e5c0046c0 con 0x7f8e60101470 2024-02-14T15:11:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.530+0000 7f8e6b04d740 1 Processor -- start 2024-02-14T15:11:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6b04d740 1 -- start start 2024-02-14T15:11:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e577fe640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e4c0a22d0 0x7f8e4c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd87ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd87b040 0x5639fd8836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e66a1e640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e4c0a22d0 0x7f8e4c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd883c20 0x5639fd886010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd7e89e0 con 0x5639fd883c20 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e577fe640 1 -- 172.21.15.17:0/3965592280 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8e5c014070 con 0x7f8e60101470 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd7e5f10 con 0x5639fd87a720 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd7e16f0 con 0x5639fd87b040 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd87b040 0x5639fd8836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6621d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd87ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e66a1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd883c20 0x5639fd886010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd87b040 0x5639fd8836e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:32818/0 (socket says 172.21.15.17:32818) 2024-02-14T15:11:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4140796583 learned_addr learned my addr 172.21.15.17:0/4140796583 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4140796583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 msgr2=0x5639fd87ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd87ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4140796583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd883c20 msgr2=0x5639fd886010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd883c20 0x5639fd886010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4140796583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639fd4f2920 con 0x5639fd87b040 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e66a1e640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e4c0a22d0 0x7f8e4c0a4790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005cc0 tx=0x7f8e580325d0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.531+0000 7f8e6621d640 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd87ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd87b040 0x5639fd8836e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009db0 tx=0x7f8e50011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ea30dd1ab68d41 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e54bfe640 1 -- 172.21.15.17:0/4140796583 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50019410 con 0x5639fd87b040 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e54bfe640 1 -- 172.21.15.17:0/4140796583 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e50019a50 con 0x5639fd87b040 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e54bfe640 1 -- 172.21.15.17:0/4140796583 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50018a20 con 0x5639fd87b040 2024-02-14T15:11:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4140796583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd87b040 msgr2=0x5639fd8836e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd87b040 0x5639fd8836e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009db0 tx=0x7f8e50011810 comp rx=0 tx=0).stop 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4140796583 shutdown_connections 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.532+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd883c20 0x5639fd886010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd87b040 0x5639fd8836e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4140796583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd87ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4140796583 >> 172.21.15.17:0/4140796583 conn(0x5639fd874e50 msgr2=0x5639fd875230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4140796583 shutdown_connections 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4140796583 wait complete. 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 -- start start 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 Processor -- start 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd8835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd880990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd880f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd77dfa0 con 0x5639fd87a720 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd4b5250 con 0x5639fd87b040 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd767200 con 0x5639fd883c20 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6b04d740 10 client.0 _ll_register_callbacks cb 0x5639fd7d6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6621d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd8835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6621d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd8835b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44714/0 (socket says 172.21.15.17:44714) 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e6621d640 1 -- 172.21.15.17:0/2522313564 learned_addr learned my addr 172.21.15.17:0/2522313564 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd880990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.533+0000 7f8e66a1e640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd880f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6621d640 1 -- 172.21.15.17:0/2522313564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 msgr2=0x5639fd880f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6621d640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd880f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6621d640 1 -- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 msgr2=0x5639fd880990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6621d640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd880990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6621d640 1 -- 172.21.15.17:0/2522313564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e500099e0 con 0x5639fd87a720 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd880990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6621d640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd8835b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c09aa80 tx=0x7f8e5c09ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8b5713e1d831d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e49ffb640 1 -- 172.21.15.17:0/2522313564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0a3290 con 0x5639fd87a720 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e49ffb640 1 -- 172.21.15.17:0/2522313564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8e5c0ce2b0 con 0x5639fd87a720 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639fd4e1600 con 0x5639fd87a720 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e49ffb640 1 -- 172.21.15.17:0/2522313564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e5c0b92a0 con 0x5639fd87a720 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6b04d740 10 client.16968 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5639fd7dfd70 con 0x5639fd87a720 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e49ffb640 1 -- 172.21.15.17:0/2522313564 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5639fd4e1600 con 0x5639fd87a720 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e49ffb640 1 -- 172.21.15.17:0/2522313564 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e5c01f060 con 0x5639fd87a720 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6b04d740 10 client.16968 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.534+0000 7f8e6b04d740 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.535+0000 7f8e6b04d740 20 client.16968 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.535+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd893990 0x5639fd895d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.535+0000 7f8e6b04d740 4 client.16968 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.535+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639fd5321b0 con 0x5639fd893990 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.535+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd893990 0x5639fd895d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.535+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd893990 0x5639fd895d80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2033f13be71b01 server_cookie=9b270f16d6058c57 in_seq=0 out_seq=0 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.535+0000 7f8e49ffb640 10 client.16968 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e49ffb640 1 -- 172.21.15.17:0/2522313564 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639fd5321b0 con 0x5639fd893990 2024-02-14T15:11:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e49ffb640 10 client.16968 handle_command_reply: tid=0 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 client.16968 shutdown 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 msgr2=0x5639fd8835b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd8835b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c09aa80 tx=0x7f8e5c09ad80 comp rx=0 tx=0).stop 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 shutdown_connections 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd893990 0x5639fd895d80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd880f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd880990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e49ffb640 0 client.16968 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/2522313564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd8835b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 >> 172.21.15.17:0/2522313564 conn(0x5639fd874e50 msgr2=0x5639fd876ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 shutdown_connections 2024-02-14T15:11:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 1 -- 172.21.15.17:0/2522313564 wait complete. 2024-02-14T15:11:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.536+0000 7f8e6b04d740 20 client.16968 trim_cache size 0 max 16384 2024-02-14T15:11:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 Processor -- start 2024-02-14T15:11:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 -- start start 2024-02-14T15:11:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd880600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87b040 0x5639fd883070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd8835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd89af30 con 0x5639fd87b040 2024-02-14T15:11:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd890460 con 0x5639fd87a720 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd8732d0 con 0x5639fd883c20 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87b040 0x5639fd883070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87b040 0x5639fd883070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44722/0 (socket says 172.21.15.17:44722) 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4158186765 learned_addr learned my addr 172.21.15.17:0/4158186765 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e66a1e640 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd8835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6621d640 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd880600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4158186765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 msgr2=0x5639fd8835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd8835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4158186765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 msgr2=0x5639fd880600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd880600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 -- 172.21.15.17:0/4158186765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639fd77c820 con 0x5639fd87b040 2024-02-14T15:11:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e6621d640 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd880600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.550+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87b040 0x5639fd883070 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8e50000c00 tx=0x7f8e500185c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e3320a4048e433 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e4bfff640 1 -- 172.21.15.17:0/4158186765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50029ea0 con 0x5639fd87b040 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e4bfff640 1 -- 172.21.15.17:0/4158186765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e50005c50 con 0x5639fd87b040 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e4bfff640 1 -- 172.21.15.17:0/4158186765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e500189f0 con 0x5639fd87b040 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4158186765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87b040 msgr2=0x5639fd883070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87b040 0x5639fd883070 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8e50000c00 tx=0x7f8e500185c0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4158186765 shutdown_connections 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87b040 0x5639fd883070 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd8835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4158186765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87a720 0x5639fd880600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4158186765 >> 172.21.15.17:0/4158186765 conn(0x5639fd874e50 msgr2=0x5639fd87fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4158186765 shutdown_connections 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.551+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4158186765 wait complete. 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 -- start start 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 Processor -- start 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd88aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd88b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd889080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd889750 con 0x5639fd87a720 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd8898c0 con 0x5639fd87b040 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639fd889a80 con 0x5639fd883c20 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6b04d740 10 client.0 _ll_register_callbacks cb 0x7f8e50009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd88aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd88aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44732/0 (socket says 172.21.15.17:44732) 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 -- 172.21.15.17:0/4253881826 learned_addr learned my addr 172.21.15.17:0/4253881826 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e66a1e640 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd889080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd88b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 -- 172.21.15.17:0/4253881826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 msgr2=0x5639fd889080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd889080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 -- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 msgr2=0x5639fd88b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd88b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 -- 172.21.15.17:0/4253881826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639fd4f2920 con 0x5639fd87a720 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e6621d640 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd88aae0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0cc0e0 tx=0x7f8e5c097600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4058c07f0429ea50 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.552+0000 7f8e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e49ffb640 1 -- 172.21.15.17:0/4253881826 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c09a6e0 con 0x5639fd87a720 2024-02-14T15:11:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e49ffb640 1 -- 172.21.15.17:0/4253881826 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8e5c0b92a0 con 0x5639fd87a720 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e50019a50 con 0x5639fd87a720 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e49ffb640 1 -- 172.21.15.17:0/4253881826 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e5c0a32a0 con 0x5639fd87a720 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 10 client.16974 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5639fd7a2720 con 0x5639fd87a720 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e49ffb640 1 -- 172.21.15.17:0/4253881826 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e50019a50 con 0x5639fd87a720 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e49ffb640 1 -- 172.21.15.17:0/4253881826 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e5c01f070 con 0x5639fd87a720 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 10 client.16974 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 20 client.16974 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd8f3490 0x5639fd8f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 4 client.16974 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639fd51e550 con 0x5639fd8f3490 2024-02-14T15:11:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.553+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd8f3490 0x5639fd8f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.554+0000 7f8e49ffb640 10 client.16974 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.554+0000 7f8e65a1c640 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd8f3490 0x5639fd8f5880 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ead01f9e61f864 server_cookie=336f0a25304a95c0 in_seq=0 out_seq=0 2024-02-14T15:11:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e49ffb640 1 -- 172.21.15.17:0/4253881826 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639fd51e550 con 0x5639fd8f3490 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e49ffb640 10 client.16974 handle_command_reply: tid=0 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 client.16974 shutdown 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 msgr2=0x5639fd88aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd88aae0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0cc0e0 tx=0x7f8e5c097600 comp rx=0 tx=0).stop 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 shutdown_connections 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5639fd8f3490 0x5639fd8f5880 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639fd883c20 0x5639fd889080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e49ffb640 0 client.16974 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639fd87b040 0x5639fd88b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 --2- 172.21.15.17:0/4253881826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5639fd87a720 0x5639fd88aae0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 >> 172.21.15.17:0/4253881826 conn(0x5639fd874e50 msgr2=0x5639fd880780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 shutdown_connections 2024-02-14T15:11:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 1 -- 172.21.15.17:0/4253881826 wait complete. 2024-02-14T15:11:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.564+0000 7f8e6b04d740 20 client.16974 trim_cache size 0 max 16384 2024-02-14T15:11:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:11:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:11:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:11:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:11:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e4c0a22d0 msgr2=0x7f8e4c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e4c0a22d0 0x7f8e4c0a4790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005cc0 tx=0x7f8e580325d0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60101470 msgr2=0x7f8e60073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60101470 0x7f8e60073530 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e9f0 tx=0x7f8e5c00edf0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 shutdown_connections 2024-02-14T15:11:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8e4c0a22d0 0x7f8e4c0a4790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e60073fb0 0x7f8e601a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e60106540 0x7f8e60073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 --2- 172.21.15.17:0/3965592280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e60101470 0x7f8e60073530 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 >> 172.21.15.17:0/3965592280 conn(0x7f8e600fcd50 msgr2=0x7f8e600ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 shutdown_connections 2024-02-14T15:11:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:42.565+0000 7f8e68481640 1 -- 172.21.15.17:0/3965592280 wait complete. 2024-02-14T15:11:42.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 858675 11.36 MB/sec execute 284 sec latency 24.427 ms 2024-02-14T15:11:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:42 smithi050 ceph-mon[36512]: pgmap v257: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-02-14T15:11:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:42 smithi076 ceph-mon[37057]: pgmap v257: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-02-14T15:11:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:42 smithi017 ceph-mon[32549]: pgmap v257: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-02-14T15:11:43.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 859862 11.35 MB/sec execute 285 sec latency 125.079 ms 2024-02-14T15:11:44.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 862117 11.35 MB/sec execute 286 sec latency 17.830 ms 2024-02-14T15:11:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:44 smithi050 ceph-mon[36512]: pgmap v258: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-02-14T15:11:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:44 smithi076 ceph-mon[37057]: pgmap v258: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-02-14T15:11:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:44 smithi017 ceph-mon[32549]: pgmap v258: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-02-14T15:11:45.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 863461 11.33 MB/sec execute 287 sec latency 21.931 ms 2024-02-14T15:11:46.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 865757 11.34 MB/sec execute 288 sec latency 19.547 ms 2024-02-14T15:11:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:46 smithi050 ceph-mon[36512]: pgmap v259: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-02-14T15:11:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:46 smithi076 ceph-mon[37057]: pgmap v259: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-02-14T15:11:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:46 smithi017 ceph-mon[32549]: pgmap v259: 129 pgs: 129 active+clean; 516 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 181 op/s 2024-02-14T15:11:47.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 866969 11.32 MB/sec execute 289 sec latency 99.619 ms 2024-02-14T15:11:48.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 868688 11.31 MB/sec execute 290 sec latency 119.136 ms 2024-02-14T15:11:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:48 smithi050 ceph-mon[36512]: pgmap v260: 129 pgs: 129 active+clean; 505 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 273 op/s 2024-02-14T15:11:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:48 smithi076 ceph-mon[37057]: pgmap v260: 129 pgs: 129 active+clean; 505 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 273 op/s 2024-02-14T15:11:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:48 smithi017 ceph-mon[32549]: pgmap v260: 129 pgs: 129 active+clean; 505 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 273 op/s 2024-02-14T15:11:49.878 INFO:tasks.workunit.client.0.smithi017.stdout: 1 870217 11.31 MB/sec execute 291 sec latency 28.759 ms 2024-02-14T15:11:50.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 872305 11.30 MB/sec execute 292 sec latency 18.826 ms 2024-02-14T15:11:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:50 smithi050 ceph-mon[36512]: pgmap v261: 129 pgs: 129 active+clean; 505 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-02-14T15:11:51.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:50 smithi076 ceph-mon[37057]: pgmap v261: 129 pgs: 129 active+clean; 505 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-02-14T15:11:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:50 smithi017 ceph-mon[32549]: pgmap v261: 129 pgs: 129 active+clean; 505 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 182 op/s 2024-02-14T15:11:51.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 873930 11.30 MB/sec execute 293 sec latency 19.115 ms 2024-02-14T15:11:52.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 876304 11.30 MB/sec execute 294 sec latency 17.795 ms 2024-02-14T15:11:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:52 smithi050 ceph-mon[36512]: pgmap v262: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-02-14T15:11:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:52 smithi076 ceph-mon[37057]: pgmap v262: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-02-14T15:11:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:52 smithi017 ceph-mon[32549]: pgmap v262: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-02-14T15:11:53.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 877330 11.29 MB/sec execute 295 sec latency 95.170 ms 2024-02-14T15:11:54.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 879350 11.27 MB/sec execute 296 sec latency 32.828 ms 2024-02-14T15:11:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:54 smithi050 ceph-mon[36512]: pgmap v263: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-14T15:11:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:54 smithi076 ceph-mon[37057]: pgmap v263: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-14T15:11:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:54 smithi017 ceph-mon[32549]: pgmap v263: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-14T15:11:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2763829542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b900696f0 msgr2=0x7f4b90105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2763829542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b900696f0 0x7f4b90105e30 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009930 tx=0x7f4b7802bc30 comp rx=0 tx=0).stop 2024-02-14T15:11:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2763829542 shutdown_connections 2024-02-14T15:11:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2763829542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b900696f0 0x7f4b90105e30 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2763829542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b90106510 0x7f4b9010e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2763829542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b90068d40 0x7f4b90069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2763829542 >> 172.21.15.17:0/2763829542 conn(0x7f4b90077220 msgr2=0x7f4b90077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2763829542 shutdown_connections 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2763829542 wait complete. 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.656+0000 7f4b94ad4640 1 Processor -- start 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b94ad4640 1 -- start start 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b94ad4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b90068d40 0x7f4b9019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b94ad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b90106510 0x7f4b9019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b94ad4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b901976b0 0x7f4b90197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b94ad4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90198180 con 0x7f4b90068d40 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b94ad4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b901982f0 con 0x7f4b901976b0 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b94ad4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b901984b0 con 0x7f4b90106510 2024-02-14T15:11:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b90068d40 0x7f4b9019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b90068d40 0x7f4b9019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35108/0 (socket says 172.21.15.17:35108) 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 -- 172.21.15.17:0/2713136377 learned_addr learned my addr 172.21.15.17:0/2713136377 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b87fff640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b90106510 0x7f4b9019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b901976b0 0x7f4b90197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 -- 172.21.15.17:0/2713136377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b90106510 msgr2=0x7f4b9019db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b90106510 0x7f4b9019db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 -- 172.21.15.17:0/2713136377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b901976b0 msgr2=0x7f4b90197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b901976b0 0x7f4b90197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 -- 172.21.15.17:0/2713136377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78009560 con 0x7f4b90068d40 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b901976b0 0x7f4b90197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b90068d40 0x7f4b9019d5f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000e9f0 tx=0x7f4b8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.657+0000 7f4b877fe640 1 -- 172.21.15.17:0/2713136377 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80009720 con 0x7f4b90068d40 2024-02-14T15:11:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.658+0000 7f4b877fe640 1 -- 172.21.15.17:0/2713136377 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b8000cd80 con 0x7f4b90068d40 2024-02-14T15:11:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.658+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b901a4450 con 0x7f4b90068d40 2024-02-14T15:11:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.658+0000 7f4b877fe640 1 -- 172.21.15.17:0/2713136377 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80010690 con 0x7f4b90068d40 2024-02-14T15:11:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.658+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b901a4930 con 0x7f4b90068d40 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.660+0000 7f4b877fe640 1 -- 172.21.15.17:0/2713136377 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4b800046c0 con 0x7f4b90068d40 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.660+0000 7f4b97779740 1 Processor -- start 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b97779740 1 -- start start 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b877fe640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4b7c0a22a0 0x7f4b7c0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b97779740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b97779740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b97779740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b97779740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc6269e0 con 0x5633fc6b8720 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b97779740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc623f10 con 0x5633fc6c1c20 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b97779740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc61f6f0 con 0x5633fc6b9040 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4b7c0a22a0 0x7f4b7c0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b877fe640 1 -- 172.21.15.17:0/2713136377 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f4b80014070 con 0x7f4b90068d40 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:39326/0 (socket says 172.21.15.17:39326) 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35114/0 (socket says 172.21.15.17:35114) 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 -- 172.21.15.17:0/841048846 learned_addr learned my addr 172.21.15.17:0/841048846 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 -- 172.21.15.17:0/841048846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 msgr2=0x5633fc6c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 -- 172.21.15.17:0/841048846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 msgr2=0x5633fc6b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b87fff640 1 -- 172.21.15.17:0/841048846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633fc330920 con 0x5633fc6b9040 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.661+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4b7c0a22a0 0x7f4b7c0a4760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009370 tx=0x7f4b88009ea0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.662+0000 7f4b87fff640 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c16e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005ab0 tx=0x7f4b7802c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e37f9dcc12173e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.662+0000 7f4b84ff9640 1 -- 172.21.15.17:0/841048846 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004800 con 0x5633fc6b9040 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.662+0000 7f4b84ff9640 1 -- 172.21.15.17:0/841048846 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b78004960 con 0x5633fc6b9040 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.662+0000 7f4b84ff9640 1 -- 172.21.15.17:0/841048846 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780050e0 con 0x5633fc6b9040 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- 172.21.15.17:0/841048846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 msgr2=0x5633fc6c16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c16e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005ab0 tx=0x7f4b7802c680 comp rx=0 tx=0).stop 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- 172.21.15.17:0/841048846 shutdown_connections 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c16e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 --2- 172.21.15.17:0/841048846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- 172.21.15.17:0/841048846 >> 172.21.15.17:0/841048846 conn(0x5633fc6b2e50 msgr2=0x5633fc6b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- 172.21.15.17:0/841048846 shutdown_connections 2024-02-14T15:11:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- 172.21.15.17:0/841048846 wait complete. 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- start start 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 Processor -- start 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6c1c20 0x5633fc6bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc5bbfa0 con 0x5633fc6c1c20 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc2f3250 con 0x5633fc6b8720 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc5a5200 con 0x5633fc6b9040 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b97779740 10 client.0 _ll_register_callbacks cb 0x5633fc614b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b8ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6c1c20 0x5633fc6bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6be910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:39336/0 (socket says 172.21.15.17:39336) 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:39960/0 (socket says 172.21.15.17:39960) 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.663+0000 7f4b87fff640 1 -- 172.21.15.17:0/3184164822 learned_addr learned my addr 172.21.15.17:0/3184164822 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b8ed9d640 1 -- 172.21.15.17:0/3184164822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 msgr2=0x5633fc6be910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6be910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b8ed9d640 1 -- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 msgr2=0x5633fc6c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b8ed9d640 1 -- 172.21.15.17:0/3184164822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9010e8f0 con 0x5633fc6c1c20 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6c1c20 0x5633fc6bee70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fd0 tx=0x7f4b88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ca059fed473ab9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b75ffb640 1 -- 172.21.15.17:0/3184164822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88014e80 con 0x5633fc6c1c20 2024-02-14T15:11:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b75ffb640 1 -- 172.21.15.17:0/3184164822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f4b88003970 con 0x5633fc6c1c20 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633fc31f600 con 0x5633fc6c1c20 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b75ffb640 1 -- 172.21.15.17:0/3184164822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b88003b30 con 0x5633fc6c1c20 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b97779740 10 client.16986 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.664+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5633fc61dd70 con 0x5633fc6c1c20 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b75ffb640 1 -- 172.21.15.17:0/3184164822 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5633fc31f600 con 0x5633fc6c1c20 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b75ffb640 1 -- 172.21.15.17:0/3184164822 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4b88027030 con 0x5633fc6c1c20 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b97779740 10 client.16986 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b97779740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b97779740 20 client.16986 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b97779740 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc6d19f0 0x5633fc6d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b97779740 4 client.16986 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633fc3701b0 con 0x5633fc6d19f0 2024-02-14T15:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc6d19f0 0x5633fc6d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b75ffb640 10 client.16986 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.665+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc6d19f0 0x5633fc6d3e10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0cb4b09024e9c5 server_cookie=235148ee4dd18652 in_seq=0 out_seq=0 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b75ffb640 1 -- 172.21.15.17:0/3184164822 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633fc3701b0 con 0x5633fc6d19f0 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b75ffb640 10 client.16986 handle_command_reply: tid=0 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 client.16986 shutdown 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6c1c20 msgr2=0x5633fc6bee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6c1c20 0x5633fc6bee70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fd0 tx=0x7f4b88004ab0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 shutdown_connections 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc6d19f0 0x5633fc6d3e10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b75ffb640 0 client.16986 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6c1c20 0x5633fc6bee70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6be910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 --2- 172.21.15.17:0/3184164822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 >> 172.21.15.17:0/3184164822 conn(0x5633fc6b2e50 msgr2=0x5633fc6b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 shutdown_connections 2024-02-14T15:11:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 1 -- 172.21.15.17:0/3184164822 wait complete. 2024-02-14T15:11:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.666+0000 7f4b97779740 20 client.16986 trim_cache size 0 max 16384 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 Processor -- start 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 -- start start 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc6ae0e0 con 0x5633fc6b8720 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc6b78f0 con 0x5633fc6c1c20 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b97779740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc6b7ec0 con 0x5633fc6b9040 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6bd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35130/0 (socket says 172.21.15.17:35130) 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 -- 172.21.15.17:0/4209418790 learned_addr learned my addr 172.21.15.17:0/4209418790 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b87fff640 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 -- 172.21.15.17:0/4209418790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 msgr2=0x5633fc6c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 -- 172.21.15.17:0/4209418790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 msgr2=0x5633fc6c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8e59c640 1 -- 172.21.15.17:0/4209418790 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633fc5ba820 con 0x5633fc6b8720 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.680+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6bd870 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4b800972a0 tx=0x7f4b80004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06855ccc2142aef server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b77fff640 1 -- 172.21.15.17:0/4209418790 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80099140 con 0x5633fc6b8720 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b77fff640 1 -- 172.21.15.17:0/4209418790 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b800aee90 con 0x5633fc6b8720 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b77fff640 1 -- 172.21.15.17:0/4209418790 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b800ce370 con 0x5633fc6b8720 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b97779740 1 -- 172.21.15.17:0/4209418790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 msgr2=0x5633fc6bd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b97779740 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6bd870 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4b800972a0 tx=0x7f4b80004b30 comp rx=0 tx=0).stop 2024-02-14T15:11:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b97779740 1 -- 172.21.15.17:0/4209418790 shutdown_connections 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b97779740 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6c1c20 0x5633fc6c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b97779740 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6b9040 0x5633fc6c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.681+0000 7f4b97779740 1 --2- 172.21.15.17:0/4209418790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b8720 0x5633fc6bd870 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 -- 172.21.15.17:0/4209418790 >> 172.21.15.17:0/4209418790 conn(0x5633fc6b2e50 msgr2=0x5633fc6b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 -- 172.21.15.17:0/4209418790 shutdown_connections 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 -- 172.21.15.17:0/4209418790 wait complete. 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 -- start start 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 Processor -- start 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b9040 0x5633fc6c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6c1c20 0x5633fc727f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc6c8550 con 0x5633fc6b9040 2024-02-14T15:11:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc6c86c0 con 0x5633fc6b8720 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fc728460 con 0x5633fc6c1c20 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b97779740 10 client.0 _ll_register_callbacks cb 0x7f4b78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b9040 0x5633fc6c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b9040 0x5633fc6c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35134/0 (socket says 172.21.15.17:35134) 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b87fff640 1 -- 172.21.15.17:0/1282129761 learned_addr learned my addr 172.21.15.17:0/1282129761 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b8ed9d640 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6c1c20 0x5633fc727f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.682+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b87fff640 1 -- 172.21.15.17:0/1282129761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6c1c20 msgr2=0x5633fc727f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b87fff640 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6c1c20 0x5633fc727f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b87fff640 1 -- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 msgr2=0x5633fc6c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b87fff640 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b87fff640 1 -- 172.21.15.17:0/1282129761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b901a4450 con 0x5633fc6b9040 2024-02-14T15:11:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b87fff640 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b9040 0x5633fc6c7df0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009900 tx=0x7f4b78004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f6b06f9399c2a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b75ffb640 1 -- 172.21.15.17:0/1282129761 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78005200 con 0x5633fc6b9040 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b75ffb640 1 -- 172.21.15.17:0/1282129761 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f4b78005360 con 0x5633fc6b9040 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b78004960 con 0x5633fc6b9040 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b75ffb640 1 -- 172.21.15.17:0/1282129761 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b78004ad0 con 0x5633fc6b9040 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b97779740 10 client.16992 fetch_fsmap learned FSMap version 31 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5633fc5e0720 con 0x5633fc6b9040 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.683+0000 7f4b75ffb640 1 -- 172.21.15.17:0/1282129761 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b78004960 con 0x5633fc6b9040 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b75ffb640 1 -- 172.21.15.17:0/1282129761 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4b7803c050 con 0x5633fc6b9040 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b97779740 10 client.16992 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b97779740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b97779740 20 client.16992 populate_metadata read hostname 'smithi017' 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b97779740 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc729880 0x5633fc72bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:11:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b97779740 4 client.16992 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:11:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633fc35c550 con 0x5633fc729880 2024-02-14T15:11:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc729880 0x5633fc72bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:11:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b75ffb640 10 client.16992 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.684+0000 7f4b8e59c640 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc729880 0x5633fc72bca0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbbb2e2f0cc03457 server_cookie=6a700b2e9a1dbc5d in_seq=0 out_seq=0 2024-02-14T15:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b75ffb640 1 -- 172.21.15.17:0/1282129761 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633fc35c550 con 0x5633fc729880 2024-02-14T15:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b75ffb640 10 client.16992 handle_command_reply: tid=0 2024-02-14T15:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 client.16992 shutdown 2024-02-14T15:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b9040 msgr2=0x5633fc6c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b9040 0x5633fc6c7df0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009900 tx=0x7f4b78004570 comp rx=0 tx=0).stop 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 shutdown_connections 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5633fc729880 0x5633fc72bca0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5633fc6b9040 0x5633fc6c7df0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b75ffb640 0 client.16992 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5633fc6c1c20 0x5633fc727f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 --2- 172.21.15.17:0/1282129761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5633fc6b8720 0x5633fc6c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 >> 172.21.15.17:0/1282129761 conn(0x5633fc6b2e50 msgr2=0x5633fc6b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 shutdown_connections 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 1 -- 172.21.15.17:0/1282129761 wait complete. 2024-02-14T15:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.695+0000 7f4b97779740 20 client.16992 trim_cache size 0 max 16384 2024-02-14T15:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4b7c0a22a0 msgr2=0x7f4b7c0a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4b7c0a22a0 0x7f4b7c0a4760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009370 tx=0x7f4b88009ea0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b90068d40 msgr2=0x7f4b9019d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b90068d40 0x7f4b9019d5f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000e9f0 tx=0x7f4b8000edf0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 shutdown_connections 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f4b7c0a22a0 0x7f4b7c0a4760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b901976b0 0x7f4b90197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b90106510 0x7f4b9019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 --2- 172.21.15.17:0/2713136377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b90068d40 0x7f4b9019d5f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 >> 172.21.15.17:0/2713136377 conn(0x7f4b90077220 msgr2=0x7f4b900fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:11:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 shutdown_connections 2024-02-14T15:11:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:11:55.696+0000 7f4b94ad4640 1 -- 172.21.15.17:0/2713136377 wait complete. 2024-02-14T15:11:55.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 881013 11.27 MB/sec execute 297 sec latency 15.326 ms 2024-02-14T15:11:56.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 883369 11.28 MB/sec execute 298 sec latency 18.572 ms 2024-02-14T15:11:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:56 smithi050 ceph-mon[36512]: pgmap v264: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-14T15:11:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:56 smithi076 ceph-mon[37057]: pgmap v264: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-14T15:11:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:56 smithi017 ceph-mon[32549]: pgmap v264: 129 pgs: 129 active+clean; 489 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 179 op/s 2024-02-14T15:11:57.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 884750 11.26 MB/sec execute 299 sec latency 26.475 ms 2024-02-14T15:11:58.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 886620 11.25 MB/sec execute 300 sec latency 128.335 ms 2024-02-14T15:11:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:11:58 smithi050 ceph-mon[36512]: pgmap v265: 129 pgs: 129 active+clean; 497 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-02-14T15:11:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:11:58 smithi076 ceph-mon[37057]: pgmap v265: 129 pgs: 129 active+clean; 497 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-02-14T15:11:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:11:58 smithi017 ceph-mon[32549]: pgmap v265: 129 pgs: 129 active+clean; 497 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-02-14T15:11:59.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 887966 11.25 MB/sec execute 301 sec latency 107.042 ms 2024-02-14T15:12:00.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 888599 11.22 MB/sec execute 302 sec latency 61.311 ms 2024-02-14T15:12:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:00 smithi050 ceph-mon[36512]: pgmap v266: 129 pgs: 129 active+clean; 497 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 176 op/s 2024-02-14T15:12:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:00 smithi076 ceph-mon[37057]: pgmap v266: 129 pgs: 129 active+clean; 497 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 176 op/s 2024-02-14T15:12:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:00 smithi017 ceph-mon[32549]: pgmap v266: 129 pgs: 129 active+clean; 497 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 176 op/s 2024-02-14T15:12:01.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 890864 11.22 MB/sec execute 303 sec latency 56.783 ms 2024-02-14T15:12:02.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 891721 11.20 MB/sec execute 304 sec latency 67.639 ms 2024-02-14T15:12:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:02 smithi050 ceph-mon[36512]: pgmap v267: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2024-02-14T15:12:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:02 smithi076 ceph-mon[37057]: pgmap v267: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2024-02-14T15:12:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:02 smithi017 ceph-mon[32549]: pgmap v267: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 294 op/s 2024-02-14T15:12:03.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 894009 11.21 MB/sec execute 305 sec latency 66.601 ms 2024-02-14T15:12:04.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 895125 11.19 MB/sec execute 306 sec latency 87.346 ms 2024-02-14T15:12:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:04 smithi050 ceph-mon[36512]: pgmap v268: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-02-14T15:12:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:04 smithi076 ceph-mon[37057]: pgmap v268: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-02-14T15:12:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:04 smithi017 ceph-mon[32549]: pgmap v268: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-02-14T15:12:05.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 897148 11.18 MB/sec execute 307 sec latency 16.364 ms 2024-02-14T15:12:06.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 898757 11.18 MB/sec execute 308 sec latency 26.443 ms 2024-02-14T15:12:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:06 smithi050 ceph-mon[36512]: pgmap v269: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-02-14T15:12:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:06 smithi076 ceph-mon[37057]: pgmap v269: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-02-14T15:12:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:06 smithi017 ceph-mon[32549]: pgmap v269: 129 pgs: 129 active+clean; 505 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 206 op/s 2024-02-14T15:12:07.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 901099 11.18 MB/sec execute 309 sec latency 43.617 ms 2024-02-14T15:12:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.786+0000 7fba501c9640 1 -- 172.21.15.17:0/2249620718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba48101860 msgr2=0x7fba48104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.786+0000 7fba501c9640 1 --2- 172.21.15.17:0/2249620718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba48101860 0x7fba48104d90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009900 tx=0x7fba3c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:12:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.786+0000 7fba501c9640 1 -- 172.21.15.17:0/2249620718 shutdown_connections 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.786+0000 7fba501c9640 1 --2- 172.21.15.17:0/2249620718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba48101860 0x7fba48104d90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.786+0000 7fba501c9640 1 --2- 172.21.15.17:0/2249620718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba48105470 0x7fba4810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.786+0000 7fba501c9640 1 --2- 172.21.15.17:0/2249620718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba48100eb0 0x7fba48101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.786+0000 7fba501c9640 1 -- 172.21.15.17:0/2249620718 >> 172.21.15.17:0/2249620718 conn(0x7fba48075440 msgr2=0x7fba48075850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 -- 172.21.15.17:0/2249620718 shutdown_connections 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 -- 172.21.15.17:0/2249620718 wait complete. 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 Processor -- start 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 -- start start 2024-02-14T15:12:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba48100eb0 0x7fba4819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba48105470 0x7fba4819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba481976c0 0x7fba48197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48198190 con 0x7fba481976c0 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48198300 con 0x7fba48105470 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba501c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481984c0 con 0x7fba48100eb0 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba4e766640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba481976c0 0x7fba48197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba4e766640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba481976c0 0x7fba48197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45074/0 (socket says 172.21.15.17:45074) 2024-02-14T15:12:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba4e766640 1 -- 172.21.15.17:0/3059455372 learned_addr learned my addr 172.21.15.17:0/3059455372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba4df65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba48100eb0 0x7fba4819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.787+0000 7fba4d764640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba48105470 0x7fba4819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba4e766640 1 -- 172.21.15.17:0/3059455372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba48100eb0 msgr2=0x7fba4819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba4e766640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba48100eb0 0x7fba4819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba4e766640 1 -- 172.21.15.17:0/3059455372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba48105470 msgr2=0x7fba4819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba4e766640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba48105470 0x7fba4819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba4e766640 1 -- 172.21.15.17:0/3059455372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c009560 con 0x7fba481976c0 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba4df65640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba48100eb0 0x7fba4819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba4e766640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba481976c0 0x7fba48197b40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fba4000e970 tx=0x7fba4000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba46ffd640 1 -- 172.21.15.17:0/3059455372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40009720 con 0x7fba481976c0 2024-02-14T15:12:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba46ffd640 1 -- 172.21.15.17:0/3059455372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba40005500 con 0x7fba481976c0 2024-02-14T15:12:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba481a4410 con 0x7fba481976c0 2024-02-14T15:12:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba46ffd640 1 -- 172.21.15.17:0/3059455372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40010430 con 0x7fba481976c0 2024-02-14T15:12:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.788+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba481a48f0 con 0x7fba481976c0 2024-02-14T15:12:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba46ffd640 1 -- 172.21.15.17:0/3059455372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fba40004ff0 con 0x7fba481976c0 2024-02-14T15:12:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 Processor -- start 2024-02-14T15:12:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 -- start start 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bb1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bbd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359b1f9e0 con 0x55c359bbac20 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359b1cf10 con 0x55c359bb2040 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.791+0000 7fba52d97740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359b186f0 con 0x55c359bb1720 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4d764640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bb1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4df65640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4d764640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bb1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:55656/0 (socket says 172.21.15.17:55656) 2024-02-14T15:12:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4d764640 1 -- 172.21.15.17:0/2217437569 learned_addr learned my addr 172.21.15.17:0/2217437569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba46ffd640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba300a2220 0x7fba300a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4d764640 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bbd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4e766640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba300a2220 0x7fba300a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4df65640 1 -- 172.21.15.17:0/2217437569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 msgr2=0x55c359bb1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4df65640 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bb1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4df65640 1 -- 172.21.15.17:0/2217437569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 msgr2=0x55c359bbd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4df65640 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bbd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4df65640 1 -- 172.21.15.17:0/2217437569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c359829920 con 0x55c359bb2040 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4d764640 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bb1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba46ffd640 1 -- 172.21.15.17:0/3059455372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fba40014070 con 0x7fba481976c0 2024-02-14T15:12:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4df65640 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bba6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba38005fd0 tx=0x7fba38011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a6fd4bd7e370c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba4e766640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba300a2220 0x7fba300a46e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba400cbb80 tx=0x7fba40090440 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba2ffff640 1 -- 172.21.15.17:0/2217437569 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38003c30 con 0x55c359bb2040 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba2ffff640 1 -- 172.21.15.17:0/2217437569 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba3800fe80 con 0x55c359bb2040 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.792+0000 7fba2ffff640 1 -- 172.21.15.17:0/2217437569 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380193e0 con 0x55c359bb2040 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 -- 172.21.15.17:0/2217437569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 msgr2=0x55c359bba6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bba6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba38005fd0 tx=0x7fba38011c70 comp rx=0 tx=0).stop 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 -- 172.21.15.17:0/2217437569 shutdown_connections 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bbd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bba6e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 --2- 172.21.15.17:0/2217437569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bb1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 -- 172.21.15.17:0/2217437569 >> 172.21.15.17:0/2217437569 conn(0x55c359babe50 msgr2=0x55c359bac230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 -- 172.21.15.17:0/2217437569 shutdown_connections 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 -- 172.21.15.17:0/2217437569 wait complete. 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.793+0000 7fba52d97740 1 -- start start 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 1 Processor -- start 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bb7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bb7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359ab4fa0 con 0x55c359bbac20 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3597ec250 con 0x55c359bb2040 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359a9e200 con 0x55c359bb1720 2024-02-14T15:12:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba52d97740 10 client.0 _ll_register_callbacks cb 0x55c359b0db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bb7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bb7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45090/0 (socket says 172.21.15.17:45090) 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 -- 172.21.15.17:0/1035900947 learned_addr learned my addr 172.21.15.17:0/1035900947 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4df65640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bb7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 -- 172.21.15.17:0/1035900947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 msgr2=0x55c359bba590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 -- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 msgr2=0x55c359bb7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bb7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4d764640 1 -- 172.21.15.17:0/1035900947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba38011440 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.794+0000 7fba4df65640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bb7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba4d764640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bb7e80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02c170 tx=0x7fba3c045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16b20825a9d9486 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba2d7fa640 1 -- 172.21.15.17:0/1035900947 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c048070 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba2d7fa640 1 -- 172.21.15.17:0/1035900947 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fba3c02cbc0 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c359818600 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba2d7fa640 1 -- 172.21.15.17:0/1035900947 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba3c004480 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 10 client.17004 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c359b16d70 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba2d7fa640 1 -- 172.21.15.17:0/1035900947 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c359818600 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba2d7fa640 1 -- 172.21.15.17:0/1035900947 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fba3c003710 con 0x55c359bbac20 2024-02-14T15:12:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 10 client.17004 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 20 client.17004 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359bca920 0x55c359bccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 4 client.17004 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3598691b0 con 0x55c359bca920 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.795+0000 7fba4df65640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359bca920 0x55c359bccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.796+0000 7fba2d7fa640 10 client.17004 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.796+0000 7fba4df65640 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359bca920 0x55c359bccd10 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8098fdb5764560a server_cookie=70f22cd612f433f5 in_seq=0 out_seq=0 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba2d7fa640 1 -- 172.21.15.17:0/1035900947 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3598691b0 con 0x55c359bca920 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba2d7fa640 10 client.17004 handle_command_reply: tid=0 2024-02-14T15:12:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 client.17004 shutdown 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 msgr2=0x55c359bb7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bb7e80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02c170 tx=0x7fba3c045040 comp rx=0 tx=0).stop 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 shutdown_connections 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359bca920 0x55c359bccd10 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bb7e80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb2040 0x55c359bb7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba2d7fa640 0 client.17004 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 --2- 172.21.15.17:0/1035900947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 >> 172.21.15.17:0/1035900947 conn(0x55c359babe50 msgr2=0x55c359badba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 shutdown_connections 2024-02-14T15:12:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 1 -- 172.21.15.17:0/1035900947 wait complete. 2024-02-14T15:12:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.797+0000 7fba52d97740 20 client.17004 trim_cache size 0 max 16384 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.810+0000 7fba52d97740 1 Processor -- start 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.810+0000 7fba52d97740 1 -- start start 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba52d97740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba52d97740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bb2040 0x55c359bba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba52d97740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bbac20 0x55c359bb4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba52d97740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359bca720 con 0x55c359bb2040 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba52d97740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359bb1380 con 0x55c359bbac20 2024-02-14T15:12:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba52d97740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359baca20 con 0x55c359bb1720 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bb2040 0x55c359bba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bb2040 0x55c359bba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45100/0 (socket says 172.21.15.17:45100) 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 -- 172.21.15.17:0/663281242 learned_addr learned my addr 172.21.15.17:0/663281242 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4d764640 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bbac20 0x55c359bb4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4d764640 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 -- 172.21.15.17:0/663281242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 msgr2=0x55c359bba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 -- 172.21.15.17:0/663281242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bbac20 msgr2=0x55c359bb4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bbac20 0x55c359bb4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 -- 172.21.15.17:0/663281242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c359ab3820 con 0x55c359bb2040 2024-02-14T15:12:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4d764640 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba4df65640 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bb2040 0x55c359bba590 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fba38005c40 tx=0x7fba380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10723dcb36dec32 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba2f7fe640 1 -- 172.21.15.17:0/663281242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800fcf0 con 0x55c359bb2040 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba2f7fe640 1 -- 172.21.15.17:0/663281242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba38004e50 con 0x55c359bb2040 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.811+0000 7fba2f7fe640 1 -- 172.21.15.17:0/663281242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38003f10 con 0x55c359bb2040 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 -- 172.21.15.17:0/663281242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bb2040 msgr2=0x55c359bba590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bb2040 0x55c359bba590 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fba38005c40 tx=0x7fba380044c0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 -- 172.21.15.17:0/663281242 shutdown_connections 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bb2040 0x55c359bba590 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bbac20 0x55c359bb4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 --2- 172.21.15.17:0/663281242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb1720 0x55c359bba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 -- 172.21.15.17:0/663281242 >> 172.21.15.17:0/663281242 conn(0x55c359babe50 msgr2=0x55c359bb78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 -- 172.21.15.17:0/663281242 shutdown_connections 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 -- 172.21.15.17:0/663281242 wait complete. 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 -- start start 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.812+0000 7fba52d97740 1 Processor -- start 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba52d97740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb1720 0x55c359bc0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba52d97740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb2040 0x55c359bc0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba52d97740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bc10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba52d97740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359c20e00 con 0x55c359bbac20 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba52d97740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359c20f40 con 0x55c359bb1720 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba52d97740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c359c21080 con 0x55c359bb2040 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba52d97740 10 client.0 _ll_register_callbacks cb 0x7fba38009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bc10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bc10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45112/0 (socket says 172.21.15.17:45112) 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 -- 172.21.15.17:0/2219853586 learned_addr learned my addr 172.21.15.17:0/2219853586 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4df65640 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb2040 0x55c359bc0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb1720 0x55c359bc0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 -- 172.21.15.17:0/2219853586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb2040 msgr2=0x55c359bc0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb2040 0x55c359bc0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 -- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb1720 msgr2=0x55c359bc0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb1720 0x55c359bc0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 -- 172.21.15.17:0/2219853586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3801e040 con 0x55c359bbac20 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba4d764640 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bc10e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fba3c001eb0 tx=0x7fba3c045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e60bef809b1209e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.813+0000 7fba2d7fa640 1 -- 172.21.15.17:0/2219853586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c048040 con 0x55c359bbac20 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba2d7fa640 1 -- 172.21.15.17:0/2219853586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fba3c000e40 con 0x55c359bbac20 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c359bb7750 con 0x55c359bbac20 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba2d7fa640 1 -- 172.21.15.17:0/2219853586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba3c003e70 con 0x55c359bbac20 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 10 client.17010 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c359ad9720 con 0x55c359bbac20 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba2d7fa640 1 -- 172.21.15.17:0/2219853586 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c359bb7750 con 0x55c359bbac20 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba2d7fa640 1 -- 172.21.15.17:0/2219853586 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fba3c05b040 con 0x55c359bbac20 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 10 client.17010 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 20 client.17010 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359c22490 0x55c359c24880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 4 client.17010 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:12:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fba3800fe80 con 0x55c359c22490 2024-02-14T15:12:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.814+0000 7fba4df65640 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359c22490 0x55c359c24880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.815+0000 7fba2d7fa640 10 client.17010 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.815+0000 7fba4df65640 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359c22490 0x55c359c24880 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a34600ab04966719 server_cookie=784ade81158466d in_seq=0 out_seq=0 2024-02-14T15:12:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.826+0000 7fba2d7fa640 1 -- 172.21.15.17:0/2219853586 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fba3800fe80 con 0x55c359c22490 2024-02-14T15:12:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.826+0000 7fba2d7fa640 10 client.17010 handle_command_reply: tid=0 2024-02-14T15:12:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 client.17010 shutdown 2024-02-14T15:12:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 msgr2=0x55c359bc10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bc10e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fba3c001eb0 tx=0x7fba3c045040 comp rx=0 tx=0).stop 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 shutdown_connections 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c359c22490 0x55c359c24880 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c359bbac20 0x55c359bc10e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c359bb2040 0x55c359bc0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba2d7fa640 0 client.17010 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 --2- 172.21.15.17:0/2219853586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c359bb1720 0x55c359bc0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 >> 172.21.15.17:0/2219853586 conn(0x55c359babe50 msgr2=0x55c359bb2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 shutdown_connections 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 1 -- 172.21.15.17:0/2219853586 wait complete. 2024-02-14T15:12:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba52d97740 20 client.17010 trim_cache size 0 max 16384 2024-02-14T15:12:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:12:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:12:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:12:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:12:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba300a2220 msgr2=0x7fba300a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba501c9640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba300a2220 0x7fba300a46e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba400cbb80 tx=0x7fba40090440 comp rx=0 tx=0).stop 2024-02-14T15:12:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba481976c0 msgr2=0x7fba48197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.827+0000 7fba501c9640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba481976c0 0x7fba48197b40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fba4000e970 tx=0x7fba4000ed70 comp rx=0 tx=0).stop 2024-02-14T15:12:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 shutdown_connections 2024-02-14T15:12:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba300a2220 0x7fba300a46e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba481976c0 0x7fba48197b40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba48105470 0x7fba4819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 --2- 172.21.15.17:0/3059455372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba48100eb0 0x7fba4819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 >> 172.21.15.17:0/3059455372 conn(0x7fba48075440 msgr2=0x7fba480fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 shutdown_connections 2024-02-14T15:12:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:08.828+0000 7fba501c9640 1 -- 172.21.15.17:0/3059455372 wait complete. 2024-02-14T15:12:08.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 902410 11.17 MB/sec execute 310 sec latency 22.292 ms 2024-02-14T15:12:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:08 smithi050 ceph-mon[36512]: pgmap v270: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 296 op/s 2024-02-14T15:12:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:08 smithi076 ceph-mon[37057]: pgmap v270: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 296 op/s 2024-02-14T15:12:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:08 smithi017 ceph-mon[32549]: pgmap v270: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 296 op/s 2024-02-14T15:12:09.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 904310 11.16 MB/sec execute 311 sec latency 110.098 ms 2024-02-14T15:12:10.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:09 smithi050 ceph-mon[36512]: osdmap e101: 12 total, 12 up, 12 in 2024-02-14T15:12:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:09 smithi076 ceph-mon[37057]: osdmap e101: 12 total, 12 up, 12 in 2024-02-14T15:12:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:09 smithi017 ceph-mon[32549]: osdmap e101: 12 total, 12 up, 12 in 2024-02-14T15:12:10.879 INFO:tasks.workunit.client.0.smithi017.stdout: 1 905702 11.16 MB/sec execute 312 sec latency 19.028 ms 2024-02-14T15:12:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:10 smithi050 ceph-mon[36512]: pgmap v272: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2024-02-14T15:12:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:10 smithi076 ceph-mon[37057]: pgmap v272: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2024-02-14T15:12:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:10 smithi017 ceph-mon[32549]: pgmap v272: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2024-02-14T15:12:11.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 907680 11.15 MB/sec execute 313 sec latency 72.292 ms 2024-02-14T15:12:12.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 909369 11.15 MB/sec execute 314 sec latency 16.287 ms 2024-02-14T15:12:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:12 smithi050 ceph-mon[36512]: pgmap v273: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:12 smithi076 ceph-mon[37057]: pgmap v273: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:12 smithi017 ceph-mon[32549]: pgmap v273: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:13.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 911330 11.14 MB/sec execute 315 sec latency 16.183 ms 2024-02-14T15:12:14.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 912751 11.14 MB/sec execute 316 sec latency 92.233 ms 2024-02-14T15:12:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:14 smithi050 ceph-mon[36512]: pgmap v274: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:14 smithi076 ceph-mon[37057]: pgmap v274: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:14 smithi017 ceph-mon[32549]: pgmap v274: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:15.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 914031 11.12 MB/sec execute 317 sec latency 171.684 ms 2024-02-14T15:12:16.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 916065 11.13 MB/sec execute 318 sec latency 25.612 ms 2024-02-14T15:12:16.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:16 smithi076 ceph-mon[37057]: pgmap v275: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:16.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:16 smithi076 ceph-mon[37057]: osdmap e102: 12 total, 12 up, 12 in 2024-02-14T15:12:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:16 smithi017 ceph-mon[32549]: pgmap v275: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:16 smithi017 ceph-mon[32549]: osdmap e102: 12 total, 12 up, 12 in 2024-02-14T15:12:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:16 smithi050 ceph-mon[36512]: pgmap v275: 129 pgs: 129 active+clean; 522 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-02-14T15:12:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:16 smithi050 ceph-mon[36512]: osdmap e102: 12 total, 12 up, 12 in 2024-02-14T15:12:17.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 917093 11.14 MB/sec execute 319 sec latency 54.616 ms 2024-02-14T15:12:18.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 919292 11.15 MB/sec execute 320 sec latency 17.122 ms 2024-02-14T15:12:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:18 smithi076 ceph-mon[37057]: pgmap v277: 129 pgs: 129 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-02-14T15:12:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:18 smithi017 ceph-mon[32549]: pgmap v277: 129 pgs: 129 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-02-14T15:12:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:18 smithi050 ceph-mon[36512]: pgmap v277: 129 pgs: 129 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-02-14T15:12:19.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 920854 11.15 MB/sec execute 321 sec latency 41.600 ms 2024-02-14T15:12:20.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 923012 11.15 MB/sec execute 322 sec latency 53.321 ms 2024-02-14T15:12:20.983 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:20 smithi076 ceph-mon[37057]: pgmap v278: 129 pgs: 129 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2024-02-14T15:12:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:20 smithi017 ceph-mon[32549]: pgmap v278: 129 pgs: 129 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2024-02-14T15:12:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:20 smithi050 ceph-mon[36512]: pgmap v278: 129 pgs: 129 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2024-02-14T15:12:21.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 924726 11.15 MB/sec execute 323 sec latency 16.229 ms 2024-02-14T15:12:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 -- 172.21.15.17:0/515245207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 msgr2=0x7f50641001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 --2- 172.21.15.17:0/515245207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f50641001d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5054009930 tx=0x7f505402bc30 comp rx=0 tx=0).stop 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 -- 172.21.15.17:0/515245207 shutdown_connections 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 --2- 172.21.15.17:0/515245207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50641065b0 0x7f506410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 --2- 172.21.15.17:0/515245207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50641007a0 0x7f5064105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 --2- 172.21.15.17:0/515245207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f50641001d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 -- 172.21.15.17:0/515245207 >> 172.21.15.17:0/515245207 conn(0x7f5064075410 msgr2=0x7f5064075820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 -- 172.21.15.17:0/515245207 shutdown_connections 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.925+0000 7f5069a15640 1 -- 172.21.15.17:0/515245207 wait complete. 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 Processor -- start 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 -- start start 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f506419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50641065b0 0x7f506419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5064199990 0x7f5064199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506419a380 con 0x7f50640ffdf0 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506419a4f0 con 0x7f5064199990 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5069a15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506419a6b0 con 0x7f50641065b0 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f506419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f506419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44208/0 (socket says 172.21.15.17:44208) 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f5062ffd640 1 -- 172.21.15.17:0/1909276756 learned_addr learned my addr 172.21.15.17:0/1909276756 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f505bfff640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50641065b0 0x7f506419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.926+0000 7f50637fe640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5064199990 0x7f5064199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5062ffd640 1 -- 172.21.15.17:0/1909276756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50641065b0 msgr2=0x7f506419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5062ffd640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50641065b0 0x7f506419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5062ffd640 1 -- 172.21.15.17:0/1909276756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5064199990 msgr2=0x7f5064199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5062ffd640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5064199990 0x7f5064199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5062ffd640 1 -- 172.21.15.17:0/1909276756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c009490 con 0x7f50640ffdf0 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f50637fe640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5064199990 0x7f5064199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5062ffd640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f506419f7e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5054009230 tx=0x7f505402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5068a13640 1 -- 172.21.15.17:0/1909276756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054004350 con 0x7f50640ffdf0 2024-02-14T15:12:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5054009560 con 0x7f50640ffdf0 2024-02-14T15:12:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5068a13640 1 -- 172.21.15.17:0/1909276756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50540044b0 con 0x7f50640ffdf0 2024-02-14T15:12:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5068a13640 1 -- 172.21.15.17:0/1909276756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054003d00 con 0x7f50640ffdf0 2024-02-14T15:12:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.927+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50641a6a10 con 0x7f50640ffdf0 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.929+0000 7f5068a13640 1 -- 172.21.15.17:0/1909276756 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5054004df0 con 0x7f50640ffdf0 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 Processor -- start 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 -- start start 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd422b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd42b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd42e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd3909e0 con 0x5645fd423040 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd38df10 con 0x5645fd42bc20 2024-02-14T15:12:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f506c5e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd3896f0 con 0x5645fd422720 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f5068a13640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f50500a2220 0x7f50500a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd42b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f50637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd422b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f50637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd422b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:38856/0 (socket says 172.21.15.17:38856) 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd42b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44218/0 (socket says 172.21.15.17:44218) 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f50637fe640 1 -- 172.21.15.17:0/3498657315 learned_addr learned my addr 172.21.15.17:0/3498657315 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f50637fe640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f50500a2220 0x7f50500a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f505bfff640 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd42e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.930+0000 7f5068a13640 1 -- 172.21.15.17:0/1909276756 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f50540f02c0 con 0x7f50640ffdf0 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f50637fe640 1 -- 172.21.15.17:0/3498657315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 msgr2=0x5645fd42e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f50637fe640 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd42e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f50637fe640 1 -- 172.21.15.17:0/3498657315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 msgr2=0x5645fd42b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f50637fe640 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd42b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f50637fe640 1 -- 172.21.15.17:0/3498657315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fd09a920 con 0x5645fd422720 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f505bfff640 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd42e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f50637fe640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f50500a2220 0x7f50500a46e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f505c0099e0 tx=0x7f505c018000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f50637fe640 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd422b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f505c009fd0 tx=0x7f505c00f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54845451a5677159 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f5059ffb640 1 -- 172.21.15.17:0/3498657315 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c004450 con 0x5645fd422720 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f5059ffb640 1 -- 172.21.15.17:0/3498657315 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f505c011610 con 0x5645fd422720 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.931+0000 7f5059ffb640 1 -- 172.21.15.17:0/3498657315 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c026400 con 0x5645fd422720 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 -- 172.21.15.17:0/3498657315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 msgr2=0x5645fd422b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd422b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f505c009fd0 tx=0x7f505c00f940 comp rx=0 tx=0).stop 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 -- 172.21.15.17:0/3498657315 shutdown_connections 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd42b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd42e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3498657315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd422b00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 -- 172.21.15.17:0/3498657315 >> 172.21.15.17:0/3498657315 conn(0x5645fd41ce50 msgr2=0x5645fd41d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 -- 172.21.15.17:0/3498657315 shutdown_connections 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 -- 172.21.15.17:0/3498657315 wait complete. 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 -- start start 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.932+0000 7f506c5e4740 1 Processor -- start 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd422720 0x5645fd42b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd428990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd42bc20 0x5645fd428fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f506c5e4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd325fa0 con 0x5645fd423040 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f506c5e4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd05d250 con 0x5645fd422720 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f506c5e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd30f200 con 0x5645fd42bc20 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f506c5e4740 10 client.0 _ll_register_callbacks cb 0x5645fd37eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd428990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f50637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd42bc20 0x5645fd428fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd428990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44230/0 (socket says 172.21.15.17:44230) 2024-02-14T15:12:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd422720 0x5645fd42b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 -- 172.21.15.17:0/2517087803 learned_addr learned my addr 172.21.15.17:0/2517087803 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 -- 172.21.15.17:0/2517087803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd42bc20 msgr2=0x5645fd428fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd42bc20 0x5645fd428fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 -- 172.21.15.17:0/2517087803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd422720 msgr2=0x5645fd42b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd422720 0x5645fd42b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 -- 172.21.15.17:0/2517087803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c019040 con 0x5645fd423040 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f505bfff640 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd428990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f504c0126d0 tx=0x7f504c012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb17a4b5ce3c479 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f5042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.933+0000 7f5042ffd640 1 -- 172.21.15.17:0/2517087803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c017070 con 0x5645fd423040 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f5042ffd640 1 -- 172.21.15.17:0/2517087803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f504c004050 con 0x5645fd423040 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645fd089600 con 0x5645fd423040 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f5042ffd640 1 -- 172.21.15.17:0/2517087803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f504c004210 con 0x5645fd423040 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 10 client.17022 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5645fd387d70 con 0x5645fd423040 2024-02-14T15:12:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f5042ffd640 1 -- 172.21.15.17:0/2517087803 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5645fd089600 con 0x5645fd423040 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f5042ffd640 1 -- 172.21.15.17:0/2517087803 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f504c003a10 con 0x5645fd423040 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 10 client.17022 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 20 client.17022 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd43de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 4 client.17022 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645fd0da1b0 con 0x5645fd43ba20 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.934+0000 7f5062ffd640 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd43de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.935+0000 7f5042ffd640 10 client.17022 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.935+0000 7f5062ffd640 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd43de40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69ee96a7b64b6fb server_cookie=b316c5a06bbad26 in_seq=0 out_seq=0 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.935+0000 7f5042ffd640 1 -- 172.21.15.17:0/2517087803 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645fd0da1b0 con 0x5645fd43ba20 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.935+0000 7f5042ffd640 10 client.17022 handle_command_reply: tid=0 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.935+0000 7f506c5e4740 1 client.17022 shutdown 2024-02-14T15:12:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 msgr2=0x5645fd428990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd428990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f504c0126d0 tx=0x7f504c012b90 comp rx=0 tx=0).stop 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 shutdown_connections 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd43de40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f5042ffd640 0 client.17022 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd423040 0x5645fd428990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd42bc20 0x5645fd428fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 --2- 172.21.15.17:0/2517087803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd422720 0x5645fd42b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 >> 172.21.15.17:0/2517087803 conn(0x5645fd41ce50 msgr2=0x5645fd41eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 shutdown_connections 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 1 -- 172.21.15.17:0/2517087803 wait complete. 2024-02-14T15:12:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.936+0000 7f506c5e4740 20 client.17022 trim_cache size 0 max 16384 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 Processor -- start 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 -- start start 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd428930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd423040 0x5645fd42b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd42bc20 0x5645fd42b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd443050 con 0x5645fd42bc20 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd438400 con 0x5645fd423040 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f506c5e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd41f160 con 0x5645fd422720 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd42bc20 0x5645fd42b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd42bc20 0x5645fd42b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44234/0 (socket says 172.21.15.17:44234) 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 -- 172.21.15.17:0/1949661651 learned_addr learned my addr 172.21.15.17:0/1949661651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f505bfff640 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd423040 0x5645fd42b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f5062ffd640 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd428930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 -- 172.21.15.17:0/1949661651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 msgr2=0x5645fd428930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd428930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 -- 172.21.15.17:0/1949661651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd423040 msgr2=0x5645fd42b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd423040 0x5645fd42b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 -- 172.21.15.17:0/1949661651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fd324820 con 0x5645fd42bc20 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.951+0000 7f50637fe640 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd42bc20 0x5645fd42b600 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f505c00ec00 tx=0x7f505c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bba39574f2ec513 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.952+0000 7f50597fa640 1 -- 172.21.15.17:0/1949661651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00ec30 con 0x5645fd42bc20 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.952+0000 7f50597fa640 1 -- 172.21.15.17:0/1949661651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f505c00ed90 con 0x5645fd42bc20 2024-02-14T15:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.952+0000 7f50597fa640 1 -- 172.21.15.17:0/1949661651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c005640 con 0x5645fd42bc20 2024-02-14T15:12:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.952+0000 7f506c5e4740 1 -- 172.21.15.17:0/1949661651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd42bc20 msgr2=0x5645fd42b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.952+0000 7f506c5e4740 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd42bc20 0x5645fd42b600 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f505c00ec00 tx=0x7f505c003eb0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 -- 172.21.15.17:0/1949661651 shutdown_connections 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd42bc20 0x5645fd42b600 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd423040 0x5645fd42b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 --2- 172.21.15.17:0/1949661651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd422720 0x5645fd428930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 -- 172.21.15.17:0/1949661651 >> 172.21.15.17:0/1949661651 conn(0x5645fd41ce50 msgr2=0x5645fd428220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 -- 172.21.15.17:0/1949661651 shutdown_connections 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 -- 172.21.15.17:0/1949661651 wait complete. 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 -- start start 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 Processor -- start 2024-02-14T15:12:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd422720 0x5645fd432c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.953+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd423040 0x5645fd430ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f506c5e4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd431530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f506c5e4740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd431a70 con 0x5645fd422720 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f506c5e4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd431be0 con 0x5645fd42bc20 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f506c5e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd431d50 con 0x5645fd423040 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f506c5e4740 10 client.0 _ll_register_callbacks cb 0x7f505c0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd422720 0x5645fd432c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f505bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd423040 0x5645fd430ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd422720 0x5645fd432c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44250/0 (socket says 172.21.15.17:44250) 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 -- 172.21.15.17:0/3572332352 learned_addr learned my addr 172.21.15.17:0/3572332352 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f50637fe640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd431530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 -- 172.21.15.17:0/3572332352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd423040 msgr2=0x5645fd430ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd423040 0x5645fd430ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 -- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 msgr2=0x5645fd431530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd431530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 -- 172.21.15.17:0/3572332352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645fd09a920 con 0x5645fd422720 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f505bfff640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd423040 0x5645fd430ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f50637fe640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd431530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5062ffd640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd422720 0x5645fd432c00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f50540ed7c0 tx=0x7f50540dc760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2106a61b72ec79fa server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5042ffd640 1 -- 172.21.15.17:0/3572332352 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540035f0 con 0x5645fd422720 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5042ffd640 1 -- 172.21.15.17:0/3572332352 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f50540bd100 con 0x5645fd422720 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f505c011610 con 0x5645fd422720 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.954+0000 7f5042ffd640 1 -- 172.21.15.17:0/3572332352 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50540bca00 con 0x5645fd422720 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 10 client.17028 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5645fd34a720 con 0x5645fd422720 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f5042ffd640 1 -- 172.21.15.17:0/3572332352 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f505c011610 con 0x5645fd422720 2024-02-14T15:12:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f5042ffd640 1 -- 172.21.15.17:0/3572332352 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f505403c070 con 0x5645fd422720 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 10 client.17028 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 20 client.17028 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd495320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 4 client.17028 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645fd0c6550 con 0x5645fd43ba20 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.955+0000 7f505bfff640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd495320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.956+0000 7f505bfff640 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd495320 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a5798acd09ab4d server_cookie=10d6d1fbcb8cf9c5 in_seq=0 out_seq=0 2024-02-14T15:12:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.956+0000 7f5042ffd640 10 client.17028 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f5042ffd640 1 -- 172.21.15.17:0/3572332352 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645fd0c6550 con 0x5645fd43ba20 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f5042ffd640 10 client.17028 handle_command_reply: tid=0 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 client.17028 shutdown 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd422720 msgr2=0x5645fd432c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd422720 0x5645fd432c00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f50540ed7c0 tx=0x7f50540dc760 comp rx=0 tx=0).stop 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 shutdown_connections 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5645fd43ba20 0x5645fd495320 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5645fd42bc20 0x5645fd431530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f5042ffd640 0 client.17028 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5645fd423040 0x5645fd430ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 --2- 172.21.15.17:0/3572332352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5645fd422720 0x5645fd432c00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.966+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 >> 172.21.15.17:0/3572332352 conn(0x5645fd41ce50 msgr2=0x5645fd41e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 shutdown_connections 2024-02-14T15:12:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f506c5e4740 1 -- 172.21.15.17:0/3572332352 wait complete. 2024-02-14T15:12:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f506c5e4740 20 client.17028 trim_cache size 0 max 16384 2024-02-14T15:12:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:12:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:12:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:12:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f50500a2220 msgr2=0x7f50500a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f50500a2220 0x7f50500a46e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f505c0099e0 tx=0x7f505c018000 comp rx=0 tx=0).stop 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 msgr2=0x7f506419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f506419f7e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5054009230 tx=0x7f505402c650 comp rx=0 tx=0).stop 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 shutdown_connections 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f50500a2220 0x7f50500a46e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5064199990 0x7f5064199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50641065b0 0x7f506419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 --2- 172.21.15.17:0/1909276756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50640ffdf0 0x7f506419f7e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 >> 172.21.15.17:0/1909276756 conn(0x7f5064075410 msgr2=0x7f50640fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 shutdown_connections 2024-02-14T15:12:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:21.967+0000 7f5069a15640 1 -- 172.21.15.17:0/1909276756 wait complete. 2024-02-14T15:12:22.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 927485 11.16 MB/sec execute 324 sec latency 12.360 ms 2024-02-14T15:12:22.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:22 smithi076 ceph-mon[37057]: pgmap v279: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:22 smithi017 ceph-mon[32549]: pgmap v279: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:22 smithi050 ceph-mon[36512]: pgmap v279: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:23.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 929540 11.15 MB/sec execute 325 sec latency 15.871 ms 2024-02-14T15:12:24.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 931293 11.16 MB/sec execute 326 sec latency 39.513 ms 2024-02-14T15:12:24.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:24 smithi076 ceph-mon[37057]: pgmap v280: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:24 smithi017 ceph-mon[32549]: pgmap v280: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:24 smithi050 ceph-mon[36512]: pgmap v280: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:25.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 933217 11.14 MB/sec execute 327 sec latency 30.041 ms 2024-02-14T15:12:26.880 INFO:tasks.workunit.client.0.smithi017.stdout: 1 935034 11.15 MB/sec execute 328 sec latency 20.900 ms 2024-02-14T15:12:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:26 smithi017 ceph-mon[32549]: pgmap v281: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:26 smithi050 ceph-mon[36512]: pgmap v281: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:27.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:26 smithi076 ceph-mon[37057]: pgmap v281: 129 pgs: 129 active+clean; 541 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 206 op/s 2024-02-14T15:12:27.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 937759 11.16 MB/sec execute 329 sec latency 13.685 ms 2024-02-14T15:12:28.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 939422 11.15 MB/sec execute 330 sec latency 25.815 ms 2024-02-14T15:12:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:28 smithi017 ceph-mon[32549]: pgmap v282: 129 pgs: 129 active+clean; 551 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 274 op/s 2024-02-14T15:12:29.035 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:28 smithi076 ceph-mon[37057]: pgmap v282: 129 pgs: 129 active+clean; 551 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 274 op/s 2024-02-14T15:12:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:28 smithi050 ceph-mon[36512]: pgmap v282: 129 pgs: 129 active+clean; 551 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 274 op/s 2024-02-14T15:12:29.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 941622 11.16 MB/sec execute 331 sec latency 63.972 ms 2024-02-14T15:12:30.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 943093 11.14 MB/sec execute 332 sec latency 18.756 ms 2024-02-14T15:12:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:30 smithi017 ceph-mon[32549]: pgmap v283: 129 pgs: 129 active+clean; 551 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2024-02-14T15:12:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:30 smithi050 ceph-mon[36512]: pgmap v283: 129 pgs: 129 active+clean; 551 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2024-02-14T15:12:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:30 smithi076 ceph-mon[37057]: pgmap v283: 129 pgs: 129 active+clean; 551 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 175 op/s 2024-02-14T15:12:31.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 945479 11.15 MB/sec execute 333 sec latency 14.323 ms 2024-02-14T15:12:32.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 946764 11.13 MB/sec execute 334 sec latency 183.637 ms 2024-02-14T15:12:33.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:32 smithi017 ceph-mon[32549]: pgmap v284: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-02-14T15:12:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:32 smithi050 ceph-mon[36512]: pgmap v284: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-02-14T15:12:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:32 smithi076 ceph-mon[37057]: pgmap v284: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 267 op/s 2024-02-14T15:12:33.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 949026 11.14 MB/sec execute 335 sec latency 18.847 ms 2024-02-14T15:12:34.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 950852 11.13 MB/sec execute 336 sec latency 107.093 ms 2024-02-14T15:12:35.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:34 smithi017 ceph-mon[32549]: pgmap v285: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-02-14T15:12:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 -- 172.21.15.17:0/3851332025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 msgr2=0x7f27fc10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 --2- 172.21.15.17:0/3851332025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc10cfe0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009cd0 tx=0x7f27f402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 -- 172.21.15.17:0/3851332025 shutdown_connections 2024-02-14T15:12:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 --2- 172.21.15.17:0/3851332025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc10cfe0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 --2- 172.21.15.17:0/3851332025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27fc100cf0 0x7f27fc104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 --2- 172.21.15.17:0/3851332025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27fc100340 0x7f27fc100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 -- 172.21.15.17:0/3851332025 >> 172.21.15.17:0/3851332025 conn(0x7f27fc077220 msgr2=0x7f27fc077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 -- 172.21.15.17:0/3851332025 shutdown_connections 2024-02-14T15:12:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.060+0000 7f2803815640 1 -- 172.21.15.17:0/3851332025 wait complete. 2024-02-14T15:12:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 Processor -- start 2024-02-14T15:12:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 -- start start 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27fc100340 0x7f27fc19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27fc197690 0x7f27fc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc198160 con 0x7f27fc105410 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc1982d0 con 0x7f27fc197690 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2803815640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc198490 con 0x7f27fc100340 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2800db0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2800db0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc19db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39798/0 (socket says 172.21.15.17:39798) 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2800db0640 1 -- 172.21.15.17:0/1208841957 learned_addr learned my addr 172.21.15.17:0/1208841957 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f28015b1640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27fc100340 0x7f27fc19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.061+0000 7f2801db2640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27fc197690 0x7f27fc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2800db0640 1 -- 172.21.15.17:0/1208841957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27fc100340 msgr2=0x7f27fc19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2800db0640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27fc100340 0x7f27fc19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2800db0640 1 -- 172.21.15.17:0/1208841957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27fc197690 msgr2=0x7f27fc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2800db0640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27fc197690 0x7f27fc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2800db0640 1 -- 172.21.15.17:0/1208841957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f4009900 con 0x7f27fc105410 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2800db0640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc19db10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f27ec005fd0 tx=0x7f27ec00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f27f27fc640 1 -- 172.21.15.17:0/1208841957 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec009c30 con 0x7f27fc105410 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f27f27fc640 1 -- 172.21.15.17:0/1208841957 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27ec009d90 con 0x7f27fc105410 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27fc1a4430 con 0x7f27fc105410 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f27f27fc640 1 -- 172.21.15.17:0/1208841957 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec010460 con 0x7f27fc105410 2024-02-14T15:12:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.062+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc1a4910 con 0x7f27fc105410 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.064+0000 7f27f27fc640 1 -- 172.21.15.17:0/1208841957 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f27ec003f20 con 0x7f27fc105410 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 Processor -- start 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f27f27fc640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27e80a2220 0x7f27e80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 -- start start 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d336eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f27f27fc640 1 -- 172.21.15.17:0/1208841957 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f27ec0ce000 con 0x7f27fc105410 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d33776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f2801db2640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27e80a2220 0x7f27e80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d337a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d32dc9e0 con 0x55f3d336f040 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d32d9f10 con 0x55f3d3377c20 2024-02-14T15:12:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28063e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d32d56f0 con 0x55f3d336e720 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d336eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f2800db0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d33776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d336eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:53430/0 (socket says 172.21.15.17:53430) 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 -- 172.21.15.17:0/508643155 learned_addr learned my addr 172.21.15.17:0/508643155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f2801db2640 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d337a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 -- 172.21.15.17:0/508643155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 msgr2=0x55f3d337a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d337a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 -- 172.21.15.17:0/508643155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 msgr2=0x55f3d33776e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d33776e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f28015b1640 1 -- 172.21.15.17:0/508643155 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3d2fe6920 con 0x55f3d336e720 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.065+0000 7f2801db2640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27e80a2220 0x7f27e80a46e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27f4005cc0 tx=0x7f27f4032ef0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.066+0000 7f28015b1640 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d336eb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009d20 tx=0x7f27f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50a62ec959fbfc12 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.066+0000 7f27e77fe640 1 -- 172.21.15.17:0/508643155 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8019630 con 0x55f3d336e720 2024-02-14T15:12:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.066+0000 7f27e77fe640 1 -- 172.21.15.17:0/508643155 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f8019c70 con 0x55f3d336e720 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.066+0000 7f27e77fe640 1 -- 172.21.15.17:0/508643155 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800ec70 con 0x55f3d336e720 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 -- 172.21.15.17:0/508643155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 msgr2=0x55f3d336eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d336eb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009d20 tx=0x7f27f8011810 comp rx=0 tx=0).stop 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 -- 172.21.15.17:0/508643155 shutdown_connections 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d33776e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d337a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 --2- 172.21.15.17:0/508643155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d336eb00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 -- 172.21.15.17:0/508643155 >> 172.21.15.17:0/508643155 conn(0x55f3d3368e50 msgr2=0x55f3d3369230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 -- 172.21.15.17:0/508643155 shutdown_connections 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 -- 172.21.15.17:0/508643155 wait complete. 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 -- start start 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.067+0000 7f28063e1740 1 Processor -- start 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28063e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d3377570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28063e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336f040 0x55f3d3374950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28063e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3374eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28063e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d3271fa0 con 0x55f3d336e720 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28063e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d2fa9250 con 0x55f3d3377c20 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28063e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d325b200 con 0x55f3d336f040 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28063e1740 10 client.0 _ll_register_callbacks cb 0x55f3d32cab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d3377570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d3377570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39806/0 (socket says 172.21.15.17:39806) 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f2801db2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3374eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 -- 172.21.15.17:0/987885450 learned_addr learned my addr 172.21.15.17:0/987885450 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f2800db0640 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336f040 0x55f3d3374950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 -- 172.21.15.17:0/987885450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336f040 msgr2=0x55f3d3374950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336f040 0x55f3d3374950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 -- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 msgr2=0x55f3d3374eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3374eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 -- 172.21.15.17:0/987885450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f8009950 con 0x55f3d336e720 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f2801db2640 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3374eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f28015b1640 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d3377570 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e950 tx=0x7f27f8009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc510c6f08cdc727 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.068+0000 7f27e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f27e4ff9640 1 -- 172.21.15.17:0/987885450 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8004050 con 0x55f3d336e720 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f27e4ff9640 1 -- 172.21.15.17:0/987885450 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f27f80041b0 con 0x55f3d336e720 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3d2fd5600 con 0x55f3d336e720 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f27e4ff9640 1 -- 172.21.15.17:0/987885450 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f8004370 con 0x55f3d336e720 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 10 client.17040 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f3d32d3d70 con 0x55f3d336e720 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f27e4ff9640 1 -- 172.21.15.17:0/987885450 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f3d2fd5600 con 0x55f3d336e720 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f27e4ff9640 1 -- 172.21.15.17:0/987885450 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f27f80046c0 con 0x55f3d336e720 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 10 client.17040 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 10 client.17040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 20 client.17040 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d3387910 0x55f3d3389d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 4 client.17040 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.069+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3d30261b0 con 0x55f3d3387910 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.070+0000 7f2800db0640 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d3387910 0x55f3d3389d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.070+0000 7f2800db0640 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d3387910 0x55f3d3389d00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d7c9831c81a44b server_cookie=73666c5fc512325e in_seq=0 out_seq=0 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.070+0000 7f27e4ff9640 10 client.17040 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f27e4ff9640 1 -- 172.21.15.17:0/987885450 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3d30261b0 con 0x55f3d3387910 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f27e4ff9640 10 client.17040 handle_command_reply: tid=0 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 client.17040 shutdown 2024-02-14T15:12:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 msgr2=0x55f3d3377570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d3377570 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e950 tx=0x7f27f8009ef0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 shutdown_connections 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d3387910 0x55f3d3389d00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3374eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f27e4ff9640 0 client.17040 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336f040 0x55f3d3374950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 --2- 172.21.15.17:0/987885450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d3377570 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 >> 172.21.15.17:0/987885450 conn(0x55f3d3368e50 msgr2=0x55f3d336aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 shutdown_connections 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 1 -- 172.21.15.17:0/987885450 wait complete. 2024-02-14T15:12:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.071+0000 7f28063e1740 20 client.17040 trim_cache size 0 max 16384 2024-02-14T15:12:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.086+0000 7f28063e1740 1 Processor -- start 2024-02-14T15:12:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.087+0000 7f28063e1740 1 -- start start 2024-02-14T15:12:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.087+0000 7f28063e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d3377030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.087+0000 7f28063e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d3377570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.087+0000 7f28063e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3371bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.087+0000 7f28063e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d3364060 con 0x55f3d336f040 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.087+0000 7f28063e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d33672d0 con 0x55f3d3377c20 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.087+0000 7f28063e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d338eee0 con 0x55f3d336e720 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3371bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3371bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:52402/0 (socket says 172.21.15.17:52402) 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 -- 172.21.15.17:0/344801871 learned_addr learned my addr 172.21.15.17:0/344801871 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2800db0640 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d3377570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 -- 172.21.15.17:0/344801871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 msgr2=0x55f3d3377030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-14T15:12:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d3377030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 -- 172.21.15.17:0/344801871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 msgr2=0x55f3d3377570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d3377570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.088+0000 7f2801db2640 1 -- 172.21.15.17:0/344801871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3d3270820 con 0x55f3d3377c20 2024-02-14T15:12:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.089+0000 7f2801db2640 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3371bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009cd0 tx=0x7f27f4043040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae1384804ea8dd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.089+0000 7f27e6ffd640 1 -- 172.21.15.17:0/344801871 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4034d50 con 0x55f3d3377c20 2024-02-14T15:12:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.090+0000 7f27e6ffd640 1 -- 172.21.15.17:0/344801871 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f4003970 con 0x55f3d3377c20 2024-02-14T15:12:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.090+0000 7f27e6ffd640 1 -- 172.21.15.17:0/344801871 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f40044a0 con 0x55f3d3377c20 2024-02-14T15:12:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.091+0000 7f28063e1740 1 -- 172.21.15.17:0/344801871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 msgr2=0x55f3d3371bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.091+0000 7f28063e1740 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3371bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009cd0 tx=0x7f27f4043040 comp rx=0 tx=0).stop 2024-02-14T15:12:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.091+0000 7f28063e1740 1 -- 172.21.15.17:0/344801871 shutdown_connections 2024-02-14T15:12:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.092+0000 7f28063e1740 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336f040 0x55f3d3377570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.092+0000 7f28063e1740 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d3377c20 0x55f3d3371bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.092+0000 7f28063e1740 1 --2- 172.21.15.17:0/344801871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d336e720 0x55f3d3377030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.092+0000 7f28063e1740 1 -- 172.21.15.17:0/344801871 >> 172.21.15.17:0/344801871 conn(0x55f3d3368e50 msgr2=0x55f3d3373240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.093+0000 7f28063e1740 1 -- 172.21.15.17:0/344801871 shutdown_connections 2024-02-14T15:12:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.094+0000 7f28063e1740 1 -- 172.21.15.17:0/344801871 wait complete. 2024-02-14T15:12:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.094+0000 7f28063e1740 1 -- start start 2024-02-14T15:12:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.094+0000 7f28063e1740 1 Processor -- start 2024-02-14T15:12:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.095+0000 7f28063e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d337cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28063e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d336f040 0x55f3d337d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28063e1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d3377c20 0x55f3d33d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28063e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d33d7350 con 0x55f3d336e720 2024-02-14T15:12:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28063e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d33d7490 con 0x55f3d336f040 2024-02-14T15:12:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28063e1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d33d7650 con 0x55f3d3377c20 2024-02-14T15:12:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28063e1740 10 client.0 _ll_register_callbacks cb 0x7f27f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f2800db0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d336f040 0x55f3d337d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f2800db0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d336f040 0x55f3d337d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:52408/0 (socket says 172.21.15.17:52408) 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f2800db0640 1 -- 172.21.15.17:0/2217506920 learned_addr learned my addr 172.21.15.17:0/2217506920 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28015b1640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d337cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f2801db2640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d3377c20 0x55f3d33d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28015b1640 1 -- 172.21.15.17:0/2217506920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d3377c20 msgr2=0x55f3d33d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28015b1640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d3377c20 0x55f3d33d6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28015b1640 1 -- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d336f040 msgr2=0x55f3d337d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28015b1640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d336f040 0x55f3d337d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.096+0000 7f28015b1640 1 -- 172.21.15.17:0/2217506920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc10d7f0 con 0x55f3d336e720 2024-02-14T15:12:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.097+0000 7f2800db0640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d336f040 0x55f3d337d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.097+0000 7f28015b1640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d337cde0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f27f8004ad0 tx=0x7f27f8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78527c66fd826e70 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.097+0000 7f27e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:12:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.097+0000 7f27e4ff9640 1 -- 172.21.15.17:0/2217506920 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8011dc0 con 0x55f3d336e720 2024-02-14T15:12:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.097+0000 7f27e4ff9640 1 -- 172.21.15.17:0/2217506920 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f27f8005180 con 0x55f3d336e720 2024-02-14T15:12:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.098+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27f8019c70 con 0x55f3d336e720 2024-02-14T15:12:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.098+0000 7f27e4ff9640 1 -- 172.21.15.17:0/2217506920 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f800e980 con 0x55f3d336e720 2024-02-14T15:12:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 10 client.17046 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f3d3296720 con 0x55f3d336e720 2024-02-14T15:12:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f27e4ff9640 1 -- 172.21.15.17:0/2217506920 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f8019c70 con 0x55f3d336e720 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f27e4ff9640 1 -- 172.21.15.17:0/2217506920 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f27f8004040 con 0x55f3d336e720 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 10 client.17046 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 20 client.17046 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d33deba0 0x55f3d33e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 4 client.17046 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.099+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f3d3012550 con 0x55f3d33deba0 2024-02-14T15:12:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.100+0000 7f2800db0640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d33deba0 0x55f3d33e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.101+0000 7f2800db0640 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d33deba0 0x55f3d33e0fc0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48fc54b6b84b31b7 server_cookie=e43ff70635e935ff in_seq=0 out_seq=0 2024-02-14T15:12:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.101+0000 7f27e4ff9640 10 client.17046 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.124+0000 7f27e4ff9640 1 -- 172.21.15.17:0/2217506920 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f3d3012550 con 0x55f3d33deba0 2024-02-14T15:12:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.124+0000 7f27e4ff9640 10 client.17046 handle_command_reply: tid=0 2024-02-14T15:12:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.124+0000 7f28063e1740 1 client.17046 shutdown 2024-02-14T15:12:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 msgr2=0x55f3d337cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d337cde0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f27f8004ad0 tx=0x7f27f8005d30 comp rx=0 tx=0).stop 2024-02-14T15:12:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 shutdown_connections 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f3d33deba0 0x55f3d33e0fc0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3d3377c20 0x55f3d33d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f3d336f040 0x55f3d337d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 --2- 172.21.15.17:0/2217506920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f3d336e720 0x55f3d337cde0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 >> 172.21.15.17:0/2217506920 conn(0x55f3d3368e50 msgr2=0x55f3d3373eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f27e4ff9640 0 client.17046 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 shutdown_connections 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 1 -- 172.21.15.17:0/2217506920 wait complete. 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.125+0000 7f28063e1740 20 client.17046 trim_cache size 0 max 16384 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:12:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:12:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:12:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:12:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27e80a2220 msgr2=0x7f27e80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27e80a2220 0x7f27e80a46e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27f4005cc0 tx=0x7f27f4032ef0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 msgr2=0x7f27fc19db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc19db10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f27ec005fd0 tx=0x7f27ec00dde0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 shutdown_connections 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f27e80a2220 0x7f27e80a46e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27fc105410 0x7f27fc19db10 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27fc197690 0x7f27fc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 --2- 172.21.15.17:0/1208841957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27fc100340 0x7f27fc19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 >> 172.21.15.17:0/1208841957 conn(0x7f27fc077220 msgr2=0x7f27fc1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 shutdown_connections 2024-02-14T15:12:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:35.126+0000 7f2803815640 1 -- 172.21.15.17:0/1208841957 wait complete. 2024-02-14T15:12:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:34 smithi050 ceph-mon[36512]: pgmap v285: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-02-14T15:12:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:34 smithi076 ceph-mon[37057]: pgmap v285: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-02-14T15:12:35.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 952661 11.13 MB/sec execute 337 sec latency 22.448 ms 2024-02-14T15:12:36.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 955167 11.14 MB/sec execute 338 sec latency 14.419 ms 2024-02-14T15:12:37.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:36 smithi017 ceph-mon[32549]: pgmap v286: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-02-14T15:12:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:36 smithi050 ceph-mon[36512]: pgmap v286: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-02-14T15:12:37.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:36 smithi076 ceph-mon[37057]: pgmap v286: 129 pgs: 129 active+clean; 561 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-02-14T15:12:37.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 956687 11.13 MB/sec execute 339 sec latency 85.699 ms 2024-02-14T15:12:38.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 959295 11.14 MB/sec execute 340 sec latency 45.055 ms 2024-02-14T15:12:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:38 smithi017 ceph-mon[32549]: pgmap v287: 129 pgs: 129 active+clean; 527 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-02-14T15:12:39.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:38 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:12:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:38 smithi050 ceph-mon[36512]: pgmap v287: 129 pgs: 129 active+clean; 527 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-02-14T15:12:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:38 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:12:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:38 smithi076 ceph-mon[37057]: pgmap v287: 129 pgs: 129 active+clean; 527 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-02-14T15:12:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:38 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:12:39.881 INFO:tasks.workunit.client.0.smithi017.stdout: 1 960536 11.12 MB/sec execute 341 sec latency 82.619 ms 2024-02-14T15:12:40.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 962911 11.13 MB/sec execute 342 sec latency 42.156 ms 2024-02-14T15:12:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:40 smithi017 ceph-mon[32549]: pgmap v288: 129 pgs: 129 active+clean; 527 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 174 op/s 2024-02-14T15:12:41.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:40 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:12:41.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:40 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:12:41.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:40 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:12:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:40 smithi050 ceph-mon[36512]: pgmap v288: 129 pgs: 129 active+clean; 527 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 174 op/s 2024-02-14T15:12:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:40 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:12:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:40 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:12:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:40 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:12:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:40 smithi076 ceph-mon[37057]: pgmap v288: 129 pgs: 129 active+clean; 527 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 174 op/s 2024-02-14T15:12:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:40 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:12:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:40 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:12:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:40 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:12:41.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 964530 11.12 MB/sec execute 343 sec latency 15.821 ms 2024-02-14T15:12:42.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 966655 11.13 MB/sec execute 344 sec latency 105.036 ms 2024-02-14T15:12:43.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:42 smithi017 ceph-mon[32549]: pgmap v289: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-02-14T15:12:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:42 smithi050 ceph-mon[36512]: pgmap v289: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-02-14T15:12:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:42 smithi076 ceph-mon[37057]: pgmap v289: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 261 op/s 2024-02-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 968358 11.11 MB/sec execute 345 sec latency 14.137 ms 2024-02-14T15:12:44.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 970318 11.12 MB/sec execute 346 sec latency 53.190 ms 2024-02-14T15:12:45.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:44 smithi017 ceph-mon[32549]: pgmap v290: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-02-14T15:12:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:44 smithi050 ceph-mon[36512]: pgmap v290: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-02-14T15:12:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:44 smithi076 ceph-mon[37057]: pgmap v290: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-02-14T15:12:45.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 972015 11.10 MB/sec execute 347 sec latency 43.300 ms 2024-02-14T15:12:46.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 974050 11.11 MB/sec execute 348 sec latency 15.504 ms 2024-02-14T15:12:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:46 smithi017 ceph-mon[32549]: pgmap v291: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-02-14T15:12:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:46 smithi050 ceph-mon[36512]: pgmap v291: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-02-14T15:12:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:46 smithi076 ceph-mon[37057]: pgmap v291: 129 pgs: 129 active+clean; 537 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-02-14T15:12:47.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 976448 11.12 MB/sec execute 349 sec latency 46.909 ms 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.242+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3056179671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 msgr2=0x7f5a14100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.242+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3056179671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a14100720 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009930 tx=0x7f5a0402bc30 comp rx=0 tx=0).stop 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3056179671 shutdown_connections 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3056179671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a14105410 0x7f5a1410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3056179671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a14100cf0 0x7f5a14104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3056179671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a14100720 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3056179671 >> 172.21.15.17:0/3056179671 conn(0x7f5a14077220 msgr2=0x7f5a14077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3056179671 shutdown_connections 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3056179671 wait complete. 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 Processor -- start 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.243+0000 7f5a1a0fb640 1 -- start start 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a1a0fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a1419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a1a0fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a14105410 0x7f5a1419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a1a0fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a141976b0 0x7f5a14197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a1a0fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14198180 con 0x7f5a14100340 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a1a0fb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141982f0 con 0x7f5a141976b0 2024-02-14T15:12:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a1a0fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141984b0 con 0x7f5a14105410 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a1419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a1419d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44390/0 (socket says 172.21.15.17:44390) 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a0affd640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a14105410 0x7f5a1419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 -- 172.21.15.17:0/3704515983 learned_addr learned my addr 172.21.15.17:0/3704515983 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a13fff640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a141976b0 0x7f5a14197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 -- 172.21.15.17:0/3704515983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a14105410 msgr2=0x7f5a1419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a14105410 0x7f5a1419db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 -- 172.21.15.17:0/3704515983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a141976b0 msgr2=0x7f5a14197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a141976b0 0x7f5a14197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 -- 172.21.15.17:0/3704515983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a04009560 con 0x7f5a14100340 2024-02-14T15:12:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a1419d5f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009230 tx=0x7f5a0402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a117fa640 1 -- 172.21.15.17:0/3704515983 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04004350 con 0x7f5a14100340 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a117fa640 1 -- 172.21.15.17:0/3704515983 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a040044b0 con 0x7f5a14100340 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a117fa640 1 -- 172.21.15.17:0/3704515983 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04003d00 con 0x7f5a14100340 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.244+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a141a43f0 con 0x7f5a14100340 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.245+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a141a48d0 con 0x7f5a14100340 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a117fa640 1 -- 172.21.15.17:0/3704515983 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5a04004df0 con 0x7f5a14100340 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 Processor -- start 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 -- start start 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a117fa640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a000a2220 0x7f5a000a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8be34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bf3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b519e0 con 0x563fe8bf1030 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b4ef10 con 0x563fe8be3100 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8b4a6f0 con 0x563fe8be3ab0 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a13fff640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a000a2220 0x7f5a000a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8be34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:47310/0 (socket says 172.21.15.17:47310) 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a13fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bf3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.247+0000 7f5a0affd640 1 -- 172.21.15.17:0/2261983729 learned_addr learned my addr 172.21.15.17:0/2261983729 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a117fa640 1 -- 172.21.15.17:0/3704515983 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f5a040f0320 con 0x7f5a14100340 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a137fe640 1 -- 172.21.15.17:0/2261983729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 msgr2=0x563fe8bf0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a137fe640 1 --2- 172.21.15.17:0/2261983729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a137fe640 1 -- 172.21.15.17:0/2261983729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 msgr2=0x563fe8bf3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a137fe640 1 --2- 172.21.15.17:0/2261983729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bf3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a137fe640 1 -- 172.21.15.17:0/2261983729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe885b920 con 0x563fe8be3100 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a137fe640 1 --2- 172.21.15.17:0/2261983729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8be34e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a040ed7c0 tx=0x7f5a04031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c5ab637dd923d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a13fff640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a000a2220 0x7f5a000a46e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005fd0 tx=0x7f5a0c009e90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a09ffb640 1 -- 172.21.15.17:0/2261983729 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040bd160 con 0x563fe8be3100 2024-02-14T15:12:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a09ffb640 1 -- 172.21.15.17:0/2261983729 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a04003e60 con 0x563fe8be3100 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.248+0000 7f5a09ffb640 1 -- 172.21.15.17:0/2261983729 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040bdd90 con 0x563fe8be3100 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 -- 172.21.15.17:0/2261983729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 msgr2=0x563fe8be34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/2261983729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8be34e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a040ed7c0 tx=0x7f5a04031810 comp rx=0 tx=0).stop 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 -- 172.21.15.17:0/2261983729 shutdown_connections 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/2261983729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bf3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/2261983729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/2261983729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8be34e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 -- 172.21.15.17:0/2261983729 >> 172.21.15.17:0/2261983729 conn(0x563fe8bdde50 msgr2=0x563fe8bde230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 -- 172.21.15.17:0/2261983729 shutdown_connections 2024-02-14T15:12:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 -- 172.21.15.17:0/2261983729 wait complete. 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 -- start start 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.249+0000 7f5a1cda5740 1 Processor -- start 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bedce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8ae6fa0 con 0x563fe8bf1030 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe881e250 con 0x563fe8be3100 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8ad0200 con 0x563fe8be3ab0 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 10 client.0 _ll_register_callbacks cb 0x563fe8b3fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a13fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bedce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:50100/0 (socket says 172.21.15.17:50100) 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 -- 172.21.15.17:0/3524527865 learned_addr learned my addr 172.21.15.17:0/3524527865 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 -- 172.21.15.17:0/3524527865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 msgr2=0x563fe8bedce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bedce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 -- 172.21.15.17:0/3524527865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 msgr2=0x563fe8bee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 -- 172.21.15.17:0/3524527865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a141a48d0 con 0x563fe8be3100 2024-02-14T15:12:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a13fff640 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009900 tx=0x7f5a040092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489f58f88898326c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f59f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f59f2ffd640 1 -- 172.21.15.17:0/3524527865 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040c60c0 con 0x563fe8be3100 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f59f2ffd640 1 -- 172.21.15.17:0/3524527865 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f5a040bd160 con 0x563fe8be3100 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.250+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fe884a600 con 0x563fe8be3100 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f59f2ffd640 1 -- 172.21.15.17:0/3524527865 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a040c0bb0 con 0x563fe8be3100 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 10 client.26414 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563fe8b48d70 con 0x563fe8be3100 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f59f2ffd640 1 -- 172.21.15.17:0/3524527865 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563fe884a600 con 0x563fe8be3100 2024-02-14T15:12:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f59f2ffd640 1 -- 172.21.15.17:0/3524527865 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a0403c070 con 0x563fe8be3100 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 10 client.26414 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 20 client.26414 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c00710 0x563fe8c02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 4 client.26414 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fe889b1b0 con 0x563fe8c00710 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a0affd640 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c00710 0x563fe8c02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f59f2ffd640 10 client.26414 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.251+0000 7f5a0affd640 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c00710 0x563fe8c02b00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d56ff593fd2652f server_cookie=b2e2cf4845a4c924 in_seq=0 out_seq=0 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f59f2ffd640 1 -- 172.21.15.17:0/3524527865 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fe889b1b0 con 0x563fe8c00710 2024-02-14T15:12:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f59f2ffd640 10 client.26414 handle_command_reply: tid=0 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 client.26414 shutdown 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 msgr2=0x563fe8bf0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009900 tx=0x7f5a040092d0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 shutdown_connections 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c00710 0x563fe8c02b00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8bf1030 0x563fe8bee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8be3ab0 0x563fe8bedce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f59f2ffd640 0 client.26414 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3524527865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0950 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 >> 172.21.15.17:0/3524527865 conn(0x563fe8bdde50 msgr2=0x563fe8bdfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 shutdown_connections 2024-02-14T15:12:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3524527865 wait complete. 2024-02-14T15:12:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.252+0000 7f5a1cda5740 20 client.26414 trim_cache size 0 max 16384 2024-02-14T15:12:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.266+0000 7f5a1cda5740 1 Processor -- start 2024-02-14T15:12:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a1cda5740 1 -- start start 2024-02-14T15:12:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8beaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8be2890 con 0x563fe8be3ab0 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8bd9090 con 0x563fe8be3100 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8bdc300 con 0x563fe8bf1030 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a13fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8beaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44428/0 (socket says 172.21.15.17:44428) 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:50104/0 (socket says 172.21.15.17:50104) 2024-02-14T15:12:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a0affd640 1 -- 172.21.15.17:0/3623217151 learned_addr learned my addr 172.21.15.17:0/3623217151 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a137fe640 1 -- 172.21.15.17:0/3623217151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 msgr2=0x563fe8beaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8beaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a137fe640 1 -- 172.21.15.17:0/3623217151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 msgr2=0x563fe8bf0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a137fe640 1 -- 172.21.15.17:0/3623217151 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe8ae5820 con 0x563fe8be3100 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a13fff640 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8beaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.267+0000 7f5a0affd640 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a137fe640 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a040ed870 tx=0x7f5a040037d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0893c22a17fc8d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a097fa640 1 -- 172.21.15.17:0/3623217151 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040bcf50 con 0x563fe8be3100 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a097fa640 1 -- 172.21.15.17:0/3623217151 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a040bc3c0 con 0x563fe8be3100 2024-02-14T15:12:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a097fa640 1 -- 172.21.15.17:0/3623217151 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040bccc0 con 0x563fe8be3100 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3623217151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 msgr2=0x563fe8bf0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a040ed870 tx=0x7f5a040037d0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3623217151 shutdown_connections 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8beaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/3623217151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf0410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3623217151 >> 172.21.15.17:0/3623217151 conn(0x563fe8bdde50 msgr2=0x563fe8bec5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.268+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3623217151 shutdown_connections 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 -- 172.21.15.17:0/3623217151 wait complete. 2024-02-14T15:12:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 -- start start 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 Processor -- start 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8bf67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8bf6dd0 con 0x563fe8be3ab0 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8bf6f40 con 0x563fe8be3100 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe8bf7100 con 0x563fe8bf1030 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a1cda5740 10 client.0 _ll_register_callbacks cb 0x7f5a0404afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44436/0 (socket says 172.21.15.17:44436) 2024-02-14T15:12:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a0affd640 1 -- 172.21.15.17:0/971235597 learned_addr learned my addr 172.21.15.17:0/971235597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.269+0000 7f5a13fff640 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8bf67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a137fe640 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a0affd640 1 -- 172.21.15.17:0/971235597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 msgr2=0x563fe8bf67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a0affd640 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8bf67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a0affd640 1 -- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 msgr2=0x563fe8bf7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a0affd640 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a0affd640 1 -- 172.21.15.17:0/971235597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc009b90 con 0x563fe8be3ab0 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a0affd640 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf62b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x563fe8c03410 tx=0x7f59fc01f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913facbd11cf062a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f59f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f59f2ffd640 1 -- 172.21.15.17:0/971235597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc017040 con 0x563fe8be3ab0 2024-02-14T15:12:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f59f2ffd640 1 -- 172.21.15.17:0/971235597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f59fc003ca0 con 0x563fe8be3ab0 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a04003e60 con 0x563fe8be3ab0 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f59f2ffd640 1 -- 172.21.15.17:0/971235597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59fc003e60 con 0x563fe8be3ab0 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a1cda5740 10 client.17064 fetch_fsmap learned FSMap version 31 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f5a0c019040 con 0x563fe8be3ab0 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.270+0000 7f59f2ffd640 1 -- 172.21.15.17:0/971235597 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a04003e60 con 0x563fe8be3ab0 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f59f2ffd640 1 -- 172.21.15.17:0/971235597 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f59fc028590 con 0x563fe8be3ab0 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a1cda5740 10 client.17064 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a1cda5740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a1cda5740 20 client.17064 populate_metadata read hostname 'smithi017' 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c58350 0x563fe8c5a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:12:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a1cda5740 4 client.17064 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:12:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563fe8887550 con 0x563fe8c58350 2024-02-14T15:12:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a137fe640 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c58350 0x563fe8c5a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:12:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f59f2ffd640 10 client.17064 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.271+0000 7f5a137fe640 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c58350 0x563fe8c5a770 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5126d028d1be80b1 server_cookie=d84f6dbcac20de01 in_seq=0 out_seq=0 2024-02-14T15:12:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f59f2ffd640 1 -- 172.21.15.17:0/971235597 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563fe8887550 con 0x563fe8c58350 2024-02-14T15:12:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f59f2ffd640 10 client.17064 handle_command_reply: tid=0 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 client.17064 shutdown 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 msgr2=0x563fe8bf62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf62b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x563fe8c03410 tx=0x7f59fc01f770 comp rx=0 tx=0).stop 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 shutdown_connections 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x563fe8c58350 0x563fe8c5a770 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563fe8be3ab0 0x563fe8bf62b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fe8bf1030 0x563fe8bf67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 --2- 172.21.15.17:0/971235597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563fe8be3100 0x563fe8bf7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f59f2ffd640 0 client.17064 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:12:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 >> 172.21.15.17:0/971235597 conn(0x563fe8bdde50 msgr2=0x563fe8be8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 shutdown_connections 2024-02-14T15:12:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 1 -- 172.21.15.17:0/971235597 wait complete. 2024-02-14T15:12:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.283+0000 7f5a1cda5740 20 client.17064 trim_cache size 0 max 16384 2024-02-14T15:12:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:12:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:12:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:12:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:12:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a000a2220 msgr2=0x7f5a000a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a000a2220 0x7f5a000a46e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005fd0 tx=0x7f5a0c009e90 comp rx=0 tx=0).stop 2024-02-14T15:12:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 msgr2=0x7f5a1419d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:12:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a1419d5f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009230 tx=0x7f5a0402c650 comp rx=0 tx=0).stop 2024-02-14T15:12:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 shutdown_connections 2024-02-14T15:12:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5a000a2220 0x7f5a000a46e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a141976b0 0x7f5a14197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a14105410 0x7f5a1419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 --2- 172.21.15.17:0/3704515983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a14100340 0x7f5a1419d5f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:12:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 >> 172.21.15.17:0/3704515983 conn(0x7f5a14077220 msgr2=0x7f5a140fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:12:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 shutdown_connections 2024-02-14T15:12:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:12:48.284+0000 7f5a1a0fb640 1 -- 172.21.15.17:0/3704515983 wait complete. 2024-02-14T15:12:48.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 977930 11.10 MB/sec execute 350 sec latency 65.073 ms 2024-02-14T15:12:49.017 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:48 smithi017 ceph-mon[32549]: pgmap v292: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-02-14T15:12:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:48 smithi050 ceph-mon[36512]: pgmap v292: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-02-14T15:12:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:48 smithi076 ceph-mon[37057]: pgmap v292: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-02-14T15:12:49.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 980542 11.12 MB/sec execute 351 sec latency 15.947 ms 2024-02-14T15:12:50.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 981151 11.09 MB/sec execute 352 sec latency 104.315 ms 2024-02-14T15:12:51.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:50 smithi017 ceph-mon[32549]: pgmap v293: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-02-14T15:12:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:50 smithi050 ceph-mon[36512]: pgmap v293: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-02-14T15:12:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:50 smithi076 ceph-mon[37057]: pgmap v293: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 173 op/s 2024-02-14T15:12:51.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 983482 11.10 MB/sec execute 353 sec latency 47.114 ms 2024-02-14T15:12:52.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 984819 11.09 MB/sec execute 354 sec latency 15.124 ms 2024-02-14T15:12:53.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:52 smithi017 ceph-mon[32549]: pgmap v294: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 258 op/s 2024-02-14T15:12:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:52 smithi050 ceph-mon[36512]: pgmap v294: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 258 op/s 2024-02-14T15:12:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:52 smithi076 ceph-mon[37057]: pgmap v294: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 258 op/s 2024-02-14T15:12:53.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 987445 11.09 MB/sec execute 355 sec latency 17.617 ms 2024-02-14T15:12:54.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 988574 11.08 MB/sec execute 356 sec latency 38.691 ms 2024-02-14T15:12:55.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:54 smithi017 ceph-mon[32549]: pgmap v295: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 171 op/s 2024-02-14T15:12:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:54 smithi050 ceph-mon[36512]: pgmap v295: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 171 op/s 2024-02-14T15:12:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:54 smithi076 ceph-mon[37057]: pgmap v295: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 171 op/s 2024-02-14T15:12:55.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 990890 11.08 MB/sec execute 357 sec latency 67.246 ms 2024-02-14T15:12:56.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 991908 11.07 MB/sec execute 358 sec latency 50.675 ms 2024-02-14T15:12:57.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:56 smithi017 ceph-mon[32549]: pgmap v296: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 171 op/s 2024-02-14T15:12:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:56 smithi050 ceph-mon[36512]: pgmap v296: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 171 op/s 2024-02-14T15:12:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:56 smithi076 ceph-mon[37057]: pgmap v296: 129 pgs: 129 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 171 op/s 2024-02-14T15:12:57.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 994001 11.07 MB/sec execute 359 sec latency 55.419 ms 2024-02-14T15:12:58.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 995441 11.06 MB/sec execute 360 sec latency 17.509 ms 2024-02-14T15:12:59.035 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:12:58 smithi076 ceph-mon[37057]: pgmap v297: 129 pgs: 129 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 253 op/s 2024-02-14T15:12:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:12:58 smithi050 ceph-mon[36512]: pgmap v297: 129 pgs: 129 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 253 op/s 2024-02-14T15:12:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:12:58 smithi017 ceph-mon[32549]: pgmap v297: 129 pgs: 129 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 253 op/s 2024-02-14T15:12:59.882 INFO:tasks.workunit.client.0.smithi017.stdout: 1 997378 11.05 MB/sec execute 361 sec latency 123.566 ms 2024-02-14T15:13:00.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 998497 11.05 MB/sec execute 362 sec latency 181.848 ms 2024-02-14T15:13:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:00 smithi050 ceph-mon[36512]: pgmap v298: 129 pgs: 129 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 166 op/s 2024-02-14T15:13:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:00 smithi076 ceph-mon[37057]: pgmap v298: 129 pgs: 129 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 166 op/s 2024-02-14T15:13:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:00 smithi017 ceph-mon[32549]: pgmap v298: 129 pgs: 129 active+clean; 596 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 166 op/s 2024-02-14T15:13:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 -- 172.21.15.17:0/268897264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100cf0 msgr2=0x7fe3a4104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/268897264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100cf0 0x7fe3a4104d30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe398009890 tx=0x7fe39802bb70 comp rx=0 tx=0).stop 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 -- 172.21.15.17:0/268897264 shutdown_connections 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/268897264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a4105410 0x7fe3a410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/268897264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100cf0 0x7fe3a4104d30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/268897264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4100340 0x7fe3a4100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 -- 172.21.15.17:0/268897264 >> 172.21.15.17:0/268897264 conn(0x7fe3a4077220 msgr2=0x7fe3a4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 -- 172.21.15.17:0/268897264 shutdown_connections 2024-02-14T15:13:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.397+0000 7fe3aa66f640 1 -- 172.21.15.17:0/268897264 wait complete. 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 Processor -- start 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 -- start start 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100340 0x7fe3a419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4105410 0x7fe3a419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41965b0 0x7fe3a4196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4196fa0 con 0x7fe3a4105410 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4197110 con 0x7fe3a4100340 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3aa66f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41972d0 con 0x7fe3a41965b0 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4105410 0x7fe3a419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4105410 0x7fe3a419c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32892/0 (socket says 172.21.15.17:32892) 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3a37fe640 1 -- 172.21.15.17:0/2627679032 learned_addr learned my addr 172.21.15.17:0/2627679032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3a8c0c640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41965b0 0x7fe3a4196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.398+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100340 0x7fe3a419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a8c0c640 1 -- 172.21.15.17:0/2627679032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100340 msgr2=0x7fe3a419c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a8c0c640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100340 0x7fe3a419c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a8c0c640 1 -- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4105410 msgr2=0x7fe3a419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a8c0c640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4105410 0x7fe3a419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a8c0c640 1 -- 172.21.15.17:0/2627679032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe394009560 con 0x7fe3a41965b0 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4105410 0x7fe3a419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100340 0x7fe3a419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a8c0c640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41965b0 0x7fe3a4196a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009fd0 tx=0x7fe39c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a17fa640 1 -- 172.21.15.17:0/2627679032 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ce20 con 0x7fe3a41965b0 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a17fa640 1 -- 172.21.15.17:0/2627679032 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe39c005020 con 0x7fe3a41965b0 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe398009490 con 0x7fe3a41965b0 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3a17fa640 1 -- 172.21.15.17:0/2627679032 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c010690 con 0x7fe3a41965b0 2024-02-14T15:13:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.399+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a41a3610 con 0x7fe3a41965b0 2024-02-14T15:13:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.402+0000 7fe3a17fa640 1 -- 172.21.15.17:0/2627679032 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe39c005500 con 0x7fe3a41965b0 2024-02-14T15:13:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.402+0000 7fe3ad23e740 1 Processor -- start 2024-02-14T15:13:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3ad23e740 1 -- start start 2024-02-14T15:13:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644750d9e0 con 0x5564475a38b0 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644750af10 con 0x5564475a2ef0 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564475066f0 con 0x5564475a8aa0 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a17fa640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe38c0a22a0 0x7fe38c0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475a3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:37638/0 (socket says 172.21.15.17:37638) 2024-02-14T15:13:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1575053932 learned_addr learned my addr 172.21.15.17:0/1575053932 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32900/0 (socket says 172.21.15.17:32900) 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a8c0c640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe38c0a22a0 0x7fe38c0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1575053932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 msgr2=0x5564475a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1575053932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 msgr2=0x5564475af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1575053932 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556447217920 con 0x5564475a2ef0 2024-02-14T15:13:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.403+0000 7fe3a17fa640 1 -- 172.21.15.17:0/2627679032 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe39c014070 con 0x7fe3a41965b0 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.404+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475a3370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe394011680 tx=0x7fe394011b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee5daacad5e566 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.404+0000 7fe3a8c0c640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe38c0a22a0 0x7fe38c0a4760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe39c09b090 tx=0x7fe39c064040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.404+0000 7fe38a7fc640 1 -- 172.21.15.17:0/1575053932 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400fb40 con 0x5564475a2ef0 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.404+0000 7fe38a7fc640 1 -- 172.21.15.17:0/1575053932 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe394005500 con 0x5564475a2ef0 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.404+0000 7fe38a7fc640 1 -- 172.21.15.17:0/1575053932 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394019430 con 0x5564475a2ef0 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1575053932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 msgr2=0x5564475a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475a3370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe394011680 tx=0x7fe394011b40 comp rx=0 tx=0).stop 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1575053932 shutdown_connections 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475a3370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1575053932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1575053932 >> 172.21.15.17:0/1575053932 conn(0x556447599b90 msgr2=0x556447599f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1575053932 shutdown_connections 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1575053932 wait complete. 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.405+0000 7fe3ad23e740 1 -- start start 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 1 Processor -- start 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564474a2fa0 con 0x5564475a2ef0 2024-02-14T15:13:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564471da250 con 0x5564475a38b0 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644748c200 con 0x5564475a8aa0 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32908/0 (socket says 172.21.15.17:32908) 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3a37fe640 1 -- 172.21.15.17:0/445303785 learned_addr learned my addr 172.21.15.17:0/445303785 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3ad23e740 10 client.0 _ll_register_callbacks cb 0x5564474fbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.406+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3a3fff640 1 -- 172.21.15.17:0/445303785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 msgr2=0x5564475ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3a3fff640 1 -- 172.21.15.17:0/445303785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 msgr2=0x5564475af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3a3fff640 1 -- 172.21.15.17:0/445303785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe394009e70 con 0x5564475a38b0 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475acac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe39401b040 tx=0x7fe39400ff30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e358562f6ba6b1eb server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3837fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3837fe640 1 -- 172.21.15.17:0/445303785 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394004220 con 0x5564475a38b0 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556447206600 con 0x5564475a38b0 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3837fe640 1 -- 172.21.15.17:0/445303785 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe394004380 con 0x5564475a38b0 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3837fe640 1 -- 172.21.15.17:0/445303785 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3940035f0 con 0x5564475a38b0 2024-02-14T15:13:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3ad23e740 10 client.26429 fetch_fsmap learned FSMap version 31 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.407+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556447504d70 con 0x5564475a38b0 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3837fe640 1 -- 172.21.15.17:0/445303785 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556447206600 con 0x5564475a38b0 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3837fe640 1 -- 172.21.15.17:0/445303785 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe394021040 con 0x5564475a38b0 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3ad23e740 10 client.26429 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3ad23e740 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3ad23e740 20 client.26429 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5564475bcdf0 0x5564475bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3ad23e740 4 client.26429 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564472571b0 con 0x5564475bcdf0 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.408+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5564475bcdf0 0x5564475bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.409+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5564475bcdf0 0x5564475bf210 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7865fcd2fe05149 server_cookie=923259688650b336 in_seq=0 out_seq=0 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.409+0000 7fe3837fe640 10 client.26429 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.409+0000 7fe3837fe640 1 -- 172.21.15.17:0/445303785 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5564472571b0 con 0x5564475bcdf0 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.409+0000 7fe3837fe640 10 client.26429 handle_command_reply: tid=0 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.409+0000 7fe3ad23e740 1 client.26429 shutdown 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 msgr2=0x5564475acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475acac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe39401b040 tx=0x7fe39400ff30 comp rx=0 tx=0).stop 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 shutdown_connections 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5564475bcdf0 0x5564475bf210 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475acac0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/445303785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3837fe640 0 client.26429 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 >> 172.21.15.17:0/445303785 conn(0x556447599b90 msgr2=0x556447598ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 shutdown_connections 2024-02-14T15:13:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 1 -- 172.21.15.17:0/445303785 wait complete. 2024-02-14T15:13:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.410+0000 7fe3ad23e740 20 client.26429 trim_cache size 0 max 16384 2024-02-14T15:13:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 Processor -- start 2024-02-14T15:13:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 -- start start 2024-02-14T15:13:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475aa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556447206600 con 0x5564475a2ef0 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564475982d0 con 0x5564475a38b0 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564475a8690 con 0x5564475a8aa0 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475aa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32918/0 (socket says 172.21.15.17:32918) 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3a37fe640 1 -- 172.21.15.17:0/1741320032 learned_addr learned my addr 172.21.15.17:0/1741320032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.427+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1741320032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 msgr2=0x5564475af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1741320032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 msgr2=0x5564475aa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475aa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1741320032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564474a1820 con 0x5564475a38b0 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475a2400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe394005020 tx=0x7fe394000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa084c7685d46c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe389ffb640 1 -- 172.21.15.17:0/1741320032 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394009d10 con 0x5564475a38b0 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe389ffb640 1 -- 172.21.15.17:0/1741320032 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe394005390 con 0x5564475a38b0 2024-02-14T15:13:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.428+0000 7fe389ffb640 1 -- 172.21.15.17:0/1741320032 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394003870 con 0x5564475a38b0 2024-02-14T15:13:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1741320032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 msgr2=0x5564475a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475a2400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe394005020 tx=0x7fe394000bc0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1741320032 shutdown_connections 2024-02-14T15:13:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a38b0 0x5564475a2400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1741320032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a2ef0 0x5564475aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1741320032 >> 172.21.15.17:0/1741320032 conn(0x556447599b90 msgr2=0x5564475a1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1741320032 shutdown_connections 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.429+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1741320032 wait complete. 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 -- start start 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 Processor -- start 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564476134c0 con 0x5564475a38b0 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556447613630 con 0x5564475a2ef0 2024-02-14T15:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564476137f0 con 0x5564475a8aa0 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3ad23e740 10 client.0 _ll_register_callbacks cb 0x7fe394009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3a3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3a3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32920/0 (socket says 172.21.15.17:32920) 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3a37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:37654/0 (socket says 172.21.15.17:37654) 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3a3fff640 1 -- 172.21.15.17:0/1371694062 learned_addr learned my addr 172.21.15.17:0/1371694062 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.430+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3a37fe640 1 -- 172.21.15.17:0/1371694062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 msgr2=0x5564475b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3a37fe640 1 -- 172.21.15.17:0/1371694062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 msgr2=0x5564475b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3a37fe640 1 -- 172.21.15.17:0/1371694062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a410d7f0 con 0x5564475a2ef0 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3a37fe640 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475b2dc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe398000c00 tx=0x7fe398044040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708365b2c307ba66 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3837fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3837fe640 1 -- 172.21.15.17:0/1371694062 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398048040 con 0x5564475a2ef0 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3837fe640 1 -- 172.21.15.17:0/1371694062 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe39804f4f0 con 0x5564475a2ef0 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe394005500 con 0x5564475a2ef0 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3837fe640 1 -- 172.21.15.17:0/1371694062 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe39804f6b0 con 0x5564475a2ef0 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3ad23e740 10 client.26435 fetch_fsmap learned FSMap version 31 2024-02-14T15:13:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.431+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556447504d70 con 0x5564475a2ef0 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3837fe640 1 -- 172.21.15.17:0/1371694062 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe398061080 con 0x5564475a2ef0 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3837fe640 1 -- 172.21.15.17:0/1371694062 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe398032800 con 0x5564475a2ef0 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3ad23e740 10 client.26435 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3ad23e740 10 client.26435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3ad23e740 20 client.26435 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55644759be50 0x55644759e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3ad23e740 4 client.26435 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe394005500 con 0x55644759be50 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55644759be50 0x55644759e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3837fe640 10 client.26435 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:01.432+0000 7fe3a3fff640 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55644759be50 0x55644759e240 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc6ad7e5a37095 server_cookie=9b6de9002c03069f in_seq=0 out_seq=0 2024-02-14T15:13:01.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 999012 11.02 MB/sec execute 363 sec latency 109.370 ms 2024-02-14T15:13:02.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1001339 11.03 MB/sec execute 364 sec latency 18.911 ms 2024-02-14T15:13:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:02 smithi050 ceph-mon[36512]: pgmap v299: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 274 op/s 2024-02-14T15:13:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.178+0000 7fe3837fe640 1 -- 172.21.15.17:0/1371694062 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe394005500 con 0x55644759be50 2024-02-14T15:13:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.178+0000 7fe3837fe640 10 client.26435 handle_command_reply: tid=0 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.178+0000 7fe3ad23e740 1 client.26435 shutdown 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 msgr2=0x5564475b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475b2dc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe398000c00 tx=0x7fe398044040 comp rx=0 tx=0).stop 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 shutdown_connections 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55644759be50 0x55644759e240 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5564475a38b0 0x5564475b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564475a8aa0 0x5564475b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3837fe640 0 client.26435 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 --2- 172.21.15.17:0/1371694062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564475a2ef0 0x5564475b2dc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 >> 172.21.15.17:0/1371694062 conn(0x556447599b90 msgr2=0x5564475a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 shutdown_connections 2024-02-14T15:13:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 1 -- 172.21.15.17:0/1371694062 wait complete. 2024-02-14T15:13:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3ad23e740 20 client.26435 trim_cache size 0 max 16384 2024-02-14T15:13:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:13:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:13:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:13:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:13:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe38c0a22a0 msgr2=0x7fe38c0a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe38c0a22a0 0x7fe38c0a4760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe39c09b090 tx=0x7fe39c064040 comp rx=0 tx=0).stop 2024-02-14T15:13:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41965b0 msgr2=0x7fe3a4196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41965b0 0x7fe3a4196a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009fd0 tx=0x7fe39c00ef10 comp rx=0 tx=0).stop 2024-02-14T15:13:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 shutdown_connections 2024-02-14T15:13:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe38c0a22a0 0x7fe38c0a4760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3a4105410 0x7fe3a419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3a41965b0 0x7fe3a4196a60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 --2- 172.21.15.17:0/2627679032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3a4100340 0x7fe3a419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.179+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 >> 172.21.15.17:0/2627679032 conn(0x7fe3a4077220 msgr2=0x7fe3a40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.180+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 shutdown_connections 2024-02-14T15:13:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:03.180+0000 7fe3aa66f640 1 -- 172.21.15.17:0/2627679032 wait complete. 2024-02-14T15:13:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:02 smithi076 ceph-mon[37057]: pgmap v299: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 274 op/s 2024-02-14T15:13:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:02 smithi017 ceph-mon[32549]: pgmap v299: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 274 op/s 2024-02-14T15:13:03.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1002553 11.01 MB/sec execute 365 sec latency 50.267 ms 2024-02-14T15:13:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: *** Caught signal (Segmentation fault) ** 2024-02-14T15:13:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: in thread 7f36fbb9e700 thread_name:md_log_replay 2024-02-14T15:13:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:13:04.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f370cf5fd20] 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5646936b0d5f] 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5646936b2f93] 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 4: (MDLog::_replay_thread()+0x647) [0x56469370f7b7] 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564693719651] 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f370cf551ca] 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 7: clone() 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2024-02-14T15:13:03.865+0000 7f36fbb9e700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: in thread 7f36fbb9e700 thread_name:md_log_replay 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2024-02-14T15:13:04.144 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f370cf5fd20] 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5646936b0d5f] 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5646936b2f93] 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 4: (MDLog::_replay_thread()+0x647) [0x56469370f7b7] 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564693719651] 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f370cf551ca] 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 7: clone() 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2024-02-14T15:13:04.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 0> 2024-02-14T15:13:03.865+0000 7f36fbb9e700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: in thread 7f36fbb9e700 thread_name:md_log_replay 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f370cf5fd20] 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5646936b0d5f] 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5646936b2f93] 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 4: (MDLog::_replay_thread()+0x647) [0x56469370f7b7] 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564693719651] 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f370cf551ca] 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 7: clone() 2024-02-14T15:13:04.146 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 0> 2024-02-14T15:13:03.865+0000 7f36fbb9e700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: in thread 7f36fbb9e700 thread_name:md_log_replay 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f370cf5fd20] 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x5646936b0d5f] 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5646936b2f93] 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 4: (MDLog::_replay_thread()+0x647) [0x56469370f7b7] 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564693719651] 2024-02-14T15:13:04.147 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f370cf551ca] 2024-02-14T15:13:04.148 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 7: clone() 2024-02-14T15:13:04.148 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:13:04.148 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[71815]: 2024-02-14T15:13:04.148 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:03 smithi050 podman[72250]: 2024-02-14 15:13:03.918922364 +0000 UTC m=+0.022408781 container died 843e3bb22bc41ce9b425ceae67f2ef39fd0eb15c461c35a4c4e46c9060f313d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-14T15:13:04.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1004785 11.02 MB/sec execute 366 sec latency 46.770 ms 2024-02-14T15:13:04.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:04 smithi050 ceph-mon[36512]: pgmap v300: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-02-14T15:13:04.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:04 smithi050 ceph-mon[36512]: osdmap e103: 12 total, 12 up, 12 in 2024-02-14T15:13:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:04 smithi076 ceph-mon[37057]: pgmap v300: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-02-14T15:13:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:04 smithi076 ceph-mon[37057]: osdmap e103: 12 total, 12 up, 12 in 2024-02-14T15:13:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:04 smithi017 ceph-mon[32549]: pgmap v300: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-02-14T15:13:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:04 smithi017 ceph-mon[32549]: osdmap e103: 12 total, 12 up, 12 in 2024-02-14T15:13:05.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:04 smithi050 podman[72250]: 2024-02-14 15:13:04.925591528 +0000 UTC m=+1.029077930 container remove 843e3bb22bc41ce9b425ceae67f2ef39fd0eb15c461c35a4c4e46c9060f313d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-14T15:13:05.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:04 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-02-14T15:13:05.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:05 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-02-14T15:13:05.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1005888 11.01 MB/sec execute 367 sec latency 180.314 ms 2024-02-14T15:13:06.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1007905 11.00 MB/sec execute 368 sec latency 18.803 ms 2024-02-14T15:13:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:06 smithi050 ceph-mon[36512]: pgmap v302: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-02-14T15:13:07.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:06 smithi076 ceph-mon[37057]: pgmap v302: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-02-14T15:13:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:06 smithi017 ceph-mon[32549]: pgmap v302: 129 pgs: 129 active+clean; 604 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-02-14T15:13:07.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1009514 11.00 MB/sec execute 369 sec latency 47.394 ms 2024-02-14T15:13:08.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1011517 10.99 MB/sec execute 370 sec latency 73.978 ms 2024-02-14T15:13:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:08 smithi050 ceph-mon[36512]: pgmap v303: 129 pgs: 129 active+clean; 612 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-02-14T15:13:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:08 smithi076 ceph-mon[37057]: pgmap v303: 129 pgs: 129 active+clean; 612 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-02-14T15:13:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:08 smithi017 ceph-mon[32549]: pgmap v303: 129 pgs: 129 active+clean; 612 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-02-14T15:13:09.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1012937 10.99 MB/sec execute 371 sec latency 24.480 ms 2024-02-14T15:13:10.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1014644 10.98 MB/sec execute 372 sec latency 58.217 ms 2024-02-14T15:13:11.068 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:10 smithi017 ceph-mon[32549]: pgmap v304: 129 pgs: 129 active+clean; 612 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-02-14T15:13:11.068 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:10 smithi017 ceph-mon[32549]: osdmap e104: 12 total, 12 up, 12 in 2024-02-14T15:13:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:10 smithi050 ceph-mon[36512]: pgmap v304: 129 pgs: 129 active+clean; 612 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-02-14T15:13:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:10 smithi050 ceph-mon[36512]: osdmap e104: 12 total, 12 up, 12 in 2024-02-14T15:13:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:10 smithi076 ceph-mon[37057]: pgmap v304: 129 pgs: 129 active+clean; 612 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 241 op/s 2024-02-14T15:13:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:10 smithi076 ceph-mon[37057]: osdmap e104: 12 total, 12 up, 12 in 2024-02-14T15:13:11.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1016345 10.98 MB/sec execute 373 sec latency 19.795 ms 2024-02-14T15:13:12.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1018140 10.97 MB/sec execute 374 sec latency 42.008 ms 2024-02-14T15:13:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:12 smithi050 ceph-mon[36512]: pgmap v306: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-02-14T15:13:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:12 smithi076 ceph-mon[37057]: pgmap v306: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-02-14T15:13:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:12 smithi017 ceph-mon[32549]: pgmap v306: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 270 op/s 2024-02-14T15:13:13.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1019837 10.97 MB/sec execute 375 sec latency 86.115 ms 2024-02-14T15:13:14.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1021578 10.96 MB/sec execute 376 sec latency 26.405 ms 2024-02-14T15:13:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:14 smithi050 ceph-mon[36512]: pgmap v307: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 232 op/s 2024-02-14T15:13:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:14 smithi076 ceph-mon[37057]: pgmap v307: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 232 op/s 2024-02-14T15:13:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:14 smithi017 ceph-mon[32549]: pgmap v307: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 232 op/s 2024-02-14T15:13:15.665 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 3. 2024-02-14T15:13:15.665 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 systemd[1]: Stopped Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:13:15.665 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 systemd[1]: Starting Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:13:15.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1023343 10.97 MB/sec execute 377 sec latency 64.082 ms 2024-02-14T15:13:15.955 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 podman[72353]: 2024-02-14 15:13:15.567755454 +0000 UTC m=+0.022395816 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:13:15.955 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 podman[72353]: 2024-02-14 15:13:15.693295218 +0000 UTC m=+0.147935556 container create ae581fba06637b172763cd7b23ff86e4779013ed504bd322540049940ac59ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-14T15:13:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.271+0000 7f1a5276b640 1 -- 172.21.15.17:0/3353255305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c105410 msgr2=0x7f1a4c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.271+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3353255305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c105410 0x7f1a4c10cfe0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009cd0 tx=0x7f1a4402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 -- 172.21.15.17:0/3353255305 shutdown_connections 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3353255305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c105410 0x7f1a4c10cfe0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3353255305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c100cf0 0x7f1a4c104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3353255305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a4c100340 0x7f1a4c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 -- 172.21.15.17:0/3353255305 >> 172.21.15.17:0/3353255305 conn(0x7f1a4c077220 msgr2=0x7f1a4c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 -- 172.21.15.17:0/3353255305 shutdown_connections 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 -- 172.21.15.17:0/3353255305 wait complete. 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 Processor -- start 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.272+0000 7f1a5276b640 1 -- start start 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c100340 0x7f1a4c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a4c105410 0x7f1a4c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c197710 0x7f1a4c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1981e0 con 0x7f1a4c100340 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c198350 con 0x7f1a4c105410 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c198510 con 0x7f1a4c197710 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c100340 0x7f1a4c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c100340 0x7f1a4c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39110/0 (socket says 172.21.15.17:39110) 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 -- 172.21.15.17:0/3402149257 learned_addr learned my addr 172.21.15.17:0/3402149257 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a50d08640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c197710 0x7f1a4c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a4c105410 0x7f1a4c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 -- 172.21.15.17:0/3402149257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c197710 msgr2=0x7f1a4c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c197710 0x7f1a4c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 -- 172.21.15.17:0/3402149257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a4c105410 msgr2=0x7f1a4c19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a4c105410 0x7f1a4c19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 -- 172.21.15.17:0/3402149257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44009900 con 0x7f1a4c100340 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a50d08640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c197710 0x7f1a4c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c100340 0x7f1a4c19d650 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e9f0 tx=0x7f1a4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a497fa640 1 -- 172.21.15.17:0/3402149257 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40009700 con 0x7f1a4c100340 2024-02-14T15:13:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c1a44b0 con 0x7f1a4c100340 2024-02-14T15:13:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a497fa640 1 -- 172.21.15.17:0/3402149257 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a4000cd80 con 0x7f1a4c100340 2024-02-14T15:13:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a497fa640 1 -- 172.21.15.17:0/3402149257 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40018920 con 0x7f1a4c100340 2024-02-14T15:13:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.273+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c1a49e0 con 0x7f1a4c100340 2024-02-14T15:13:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.276+0000 7f1a497fa640 1 -- 172.21.15.17:0/3402149257 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1a40018a80 con 0x7f1a4c100340 2024-02-14T15:13:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.276+0000 7f1a5540e740 1 Processor -- start 2024-02-14T15:13:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.276+0000 7f1a5540e740 1 -- start start 2024-02-14T15:13:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bcdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 0x5611b1bd9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a5540e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1b3b9e0 con 0x5611b1bce040 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a5540e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1b38f10 con 0x5611b1bd6c20 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a5540e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1b346f0 con 0x5611b1bcd720 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a497fa640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a340a2220 0x7f1a340a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a4bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bcdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bcdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:36984/0 (socket says 172.21.15.17:36984) 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a4bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39112/0 (socket says 172.21.15.17:39112) 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 -- 172.21.15.17:0/1365735190 learned_addr learned my addr 172.21.15.17:0/1365735190 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a340a2220 0x7f1a340a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 0x5611b1bd9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a497fa640 1 -- 172.21.15.17:0/3402149257 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f1a40014070 con 0x7f1a4c100340 2024-02-14T15:13:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 -- 172.21.15.17:0/1365735190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 msgr2=0x5611b1bd9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 0x5611b1bd9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 -- 172.21.15.17:0/1365735190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 msgr2=0x5611b1bd66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 -- 172.21.15.17:0/1365735190 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611b1845920 con 0x5611b1bcd720 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bcdb00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009430 tx=0x7f1a44034d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6340a662c5feb28 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.277+0000 7f1a50d08640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a340a2220 0x7f1a340a46e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1a44033000 tx=0x7f1a4403c000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a327fc640 1 -- 172.21.15.17:0/1365735190 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44048070 con 0x5611b1bcd720 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a327fc640 1 -- 172.21.15.17:0/1365735190 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a44034a90 con 0x5611b1bcd720 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a327fc640 1 -- 172.21.15.17:0/1365735190 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440045a0 con 0x5611b1bcd720 2024-02-14T15:13:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a5540e740 1 -- 172.21.15.17:0/1365735190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 msgr2=0x5611b1bcdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a5540e740 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bcdb00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009430 tx=0x7f1a44034d70 comp rx=0 tx=0).stop 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a5540e740 1 -- 172.21.15.17:0/1365735190 shutdown_connections 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a5540e740 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a5540e740 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 0x5611b1bd9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a5540e740 1 --2- 172.21.15.17:0/1365735190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bcdb00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.278+0000 7f1a5540e740 1 -- 172.21.15.17:0/1365735190 >> 172.21.15.17:0/1365735190 conn(0x5611b1bc7e50 msgr2=0x5611b1bc8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 -- 172.21.15.17:0/1365735190 shutdown_connections 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 -- 172.21.15.17:0/1365735190 wait complete. 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 -- start start 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 Processor -- start 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bcd720 0x5611b1bd6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bd4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1ad0fa0 con 0x5611b1bce040 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1808250 con 0x5611b1bcd720 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1aba200 con 0x5611b1bd6c20 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a5540e740 10 client.0 _ll_register_callbacks cb 0x5611b1b29b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a4b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a4b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39118/0 (socket says 172.21.15.17:39118) 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/13376584 learned_addr learned my addr 172.21.15.17:0/13376584 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.279+0000 7f1a50d08640 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bd4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bcd720 0x5611b1bd6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/13376584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 msgr2=0x5611b1bd4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bd4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bcd720 msgr2=0x5611b1bd6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bcd720 0x5611b1bd6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/13376584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c068a20 con 0x5611b1bce040 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd3a50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c012700 tx=0x7f1a3c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1437d48ce9e7ee26 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/13376584 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c017070 con 0x5611b1bce040 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/13376584 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f1a3c00f920 con 0x5611b1bce040 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611b18851b0 con 0x5611b1bce040 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/13376584 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a3c00fae0 con 0x5611b1bce040 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a5540e740 10 client.17097 fetch_fsmap learned FSMap version 31 2024-02-14T15:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.280+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5611b1b32d70 con 0x5611b1bce040 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/13376584 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611b18851b0 con 0x5611b1bce040 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/13376584 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1a3c010430 con 0x5611b1bce040 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a5540e740 10 client.17097 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a5540e740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a5540e740 20 client.17097 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a5540e740 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1be6a40 0x5611b1be8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a5540e740 4 client.17097 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611b1871550 con 0x5611b1be6a40 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.281+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1be6a40 0x5611b1be8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.282+0000 7f1a2b7fe640 10 client.17097 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.282+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1be6a40 0x5611b1be8e60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9930698a0e3a2e5 server_cookie=6006ac2b2c09702c in_seq=0 out_seq=0 2024-02-14T15:13:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.283+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/13376584 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611b1871550 con 0x5611b1be6a40 2024-02-14T15:13:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.283+0000 7f1a2b7fe640 10 client.17097 handle_command_reply: tid=0 2024-02-14T15:13:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.283+0000 7f1a5540e740 1 client.17097 shutdown 2024-02-14T15:13:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.284+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 msgr2=0x5611b1bd3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.284+0000 7f1a5540e740 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd3a50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c012700 tx=0x7f1a3c012bc0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 shutdown_connections 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1be6a40 0x5611b1be8e60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd3a50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bd4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 --2- 172.21.15.17:0/13376584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bcd720 0x5611b1bd6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 >> 172.21.15.17:0/13376584 conn(0x5611b1bc7e50 msgr2=0x5611b1bc9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a2b7fe640 0 client.17097 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 shutdown_connections 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 1 -- 172.21.15.17:0/13376584 wait complete. 2024-02-14T15:13:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.285+0000 7f1a5540e740 20 client.17097 trim_cache size 0 max 16384 2024-02-14T15:13:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 Processor -- start 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 -- start start 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bd39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 0x5611b1bd6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1bc76f0 con 0x5611b1bce040 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1bcd160 con 0x5611b1bd6c20 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a5540e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1bc8c50 con 0x5611b1bcd720 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a4b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a4b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39122/0 (socket says 172.21.15.17:39122) 2024-02-14T15:13:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/612381424 learned_addr learned my addr 172.21.15.17:0/612381424 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.310+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bd39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/612381424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 msgr2=0x5611b1bd39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bd39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/612381424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 msgr2=0x5611b1bd6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 0x5611b1bd6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/612381424 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611b1acf820 con 0x5611b1bce040 2024-02-14T15:13:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd6130 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c010840 tx=0x7f1a3c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b828f45bb0e480b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a31ffb640 1 -- 172.21.15.17:0/612381424 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c017040 con 0x5611b1bce040 2024-02-14T15:13:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a31ffb640 1 -- 172.21.15.17:0/612381424 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c011f30 con 0x5611b1bce040 2024-02-14T15:13:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.311+0000 7f1a31ffb640 1 -- 172.21.15.17:0/612381424 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c005370 con 0x5611b1bce040 2024-02-14T15:13:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.313+0000 7f1a5540e740 1 -- 172.21.15.17:0/612381424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 msgr2=0x5611b1bd6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.313+0000 7f1a5540e740 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd6130 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c010840 tx=0x7f1a3c009f60 comp rx=0 tx=0).stop 2024-02-14T15:13:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.313+0000 7f1a5540e740 1 -- 172.21.15.17:0/612381424 shutdown_connections 2024-02-14T15:13:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.313+0000 7f1a5540e740 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bce040 0x5611b1bd6130 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.313+0000 7f1a5540e740 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bd6c20 0x5611b1bd6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.313+0000 7f1a5540e740 1 --2- 172.21.15.17:0/612381424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bcd720 0x5611b1bd39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.313+0000 7f1a5540e740 1 -- 172.21.15.17:0/612381424 >> 172.21.15.17:0/612381424 conn(0x5611b1bc7e50 msgr2=0x5611b1bd32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.314+0000 7f1a5540e740 1 -- 172.21.15.17:0/612381424 shutdown_connections 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.314+0000 7f1a5540e740 1 -- 172.21.15.17:0/612381424 wait complete. 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.314+0000 7f1a5540e740 1 -- start start 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 1 Processor -- start 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bcd720 0x5611b1bdde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bce040 0x5611b1bdc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bdc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1bdcef0 con 0x5611b1bcd720 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1bdd060 con 0x5611b1bce040 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611b1c3cf90 con 0x5611b1bd6c20 2024-02-14T15:13:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a4b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bce040 0x5611b1bdc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a5540e740 10 client.0 _ll_register_callbacks cb 0x7f1a40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a4b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bce040 0x5611b1bdc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:36770/0 (socket says 172.21.15.17:36770) 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a4b7fe640 1 -- 172.21.15.17:0/3980776314 learned_addr learned my addr 172.21.15.17:0/3980776314 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bcd720 0x5611b1bdde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.315+0000 7f1a50d08640 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bdc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.316+0000 7f1a4bfff640 1 -- 172.21.15.17:0/3980776314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 msgr2=0x5611b1bdc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.316+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bdc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.316+0000 7f1a4bfff640 1 -- 172.21.15.17:0/3980776314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bce040 msgr2=0x5611b1bdc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.316+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bce040 0x5611b1bdc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.316+0000 7f1a4bfff640 1 -- 172.21.15.17:0/3980776314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c021040 con 0x5611b1bcd720 2024-02-14T15:13:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.316+0000 7f1a4bfff640 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bcd720 0x5611b1bdde00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1a400cbb50 tx=0x7f1a40097180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f77e33cf1f255c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.316+0000 7f1a2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:13:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.317+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/3980776314 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a400b2070 con 0x5611b1bcd720 2024-02-14T15:13:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.317+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611b1834600 con 0x5611b1bcd720 2024-02-14T15:13:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.317+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/3980776314 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f1a400aee40 con 0x5611b1bcd720 2024-02-14T15:13:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.317+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/3980776314 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a400a0140 con 0x5611b1bcd720 2024-02-14T15:13:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.318+0000 7f1a5540e740 10 client.17103 fetch_fsmap learned FSMap version 31 2024-02-14T15:13:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.318+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5611b1af5720 con 0x5611b1bcd720 2024-02-14T15:13:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/3980776314 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611b1834600 con 0x5611b1bcd720 2024-02-14T15:13:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/3980776314 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1a400b9120 con 0x5611b1bcd720 2024-02-14T15:13:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a5540e740 10 client.17103 fetch_fsmap finished waiting for FSMap version 31 2024-02-14T15:13:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a5540e740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a5540e740 20 client.17103 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a5540e740 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1c3e700 0x5611b1c40af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a5540e740 4 client.17103 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:13:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.319+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611b17e0520 con 0x5611b1c3e700 2024-02-14T15:13:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.320+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1c3e700 0x5611b1c40af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.320+0000 7f1a2b7fe640 10 client.17103 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.321+0000 7f1a4b7fe640 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1c3e700 0x5611b1c40af0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aae9fe5124cc437 server_cookie=6564d4bfe80841f in_seq=0 out_seq=0 2024-02-14T15:13:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.350+0000 7f1a2b7fe640 1 -- 172.21.15.17:0/3980776314 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611b17e0520 con 0x5611b1c3e700 2024-02-14T15:13:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.350+0000 7f1a2b7fe640 10 client.17103 handle_command_reply: tid=0 2024-02-14T15:13:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.350+0000 7f1a5540e740 1 client.17103 shutdown 2024-02-14T15:13:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bcd720 msgr2=0x5611b1bdde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bcd720 0x5611b1bdde00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1a400cbb50 tx=0x7f1a40097180 comp rx=0 tx=0).stop 2024-02-14T15:13:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 shutdown_connections 2024-02-14T15:13:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5611b1c3e700 0x5611b1c40af0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611b1bd6c20 0x5611b1bdc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611b1bce040 0x5611b1bdc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 --2- 172.21.15.17:0/3980776314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5611b1bcd720 0x5611b1bdde00 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 >> 172.21.15.17:0/3980776314 conn(0x5611b1bc7e50 msgr2=0x5611b1bc9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a2b7fe640 0 client.17103 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 shutdown_connections 2024-02-14T15:13:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.351+0000 7f1a5540e740 1 -- 172.21.15.17:0/3980776314 wait complete. 2024-02-14T15:13:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.352+0000 7f1a5540e740 20 client.17103 trim_cache size 0 max 16384 2024-02-14T15:13:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:13:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:13:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:13:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:13:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.354+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a340a2220 msgr2=0x7f1a340a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.354+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a340a2220 0x7f1a340a46e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1a44033000 tx=0x7f1a4403c000 comp rx=0 tx=0).stop 2024-02-14T15:13:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.354+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c100340 msgr2=0x7f1a4c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.354+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c100340 0x7f1a4c19d650 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e9f0 tx=0x7f1a4000eeb0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 shutdown_connections 2024-02-14T15:13:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a340a2220 0x7f1a340a46e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a4c197710 0x7f1a4c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a4c105410 0x7f1a4c19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 --2- 172.21.15.17:0/3402149257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a4c100340 0x7f1a4c19d650 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 >> 172.21.15.17:0/3402149257 conn(0x7f1a4c077220 msgr2=0x7f1a4c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 shutdown_connections 2024-02-14T15:13:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:16.355+0000 7f1a5276b640 1 -- 172.21.15.17:0/3402149257 wait complete. 2024-02-14T15:13:16.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 podman[72353]: 2024-02-14 15:13:15.955423515 +0000 UTC m=+0.410063869 container init ae581fba06637b172763cd7b23ff86e4779013ed504bd322540049940ac59ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD) 2024-02-14T15:13:16.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 podman[72353]: 2024-02-14 15:13:15.959021515 +0000 UTC m=+0.413661872 container start ae581fba06637b172763cd7b23ff86e4779013ed504bd322540049940ac59ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-14T15:13:16.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 ceph-mds[72372]: -- 172.21.15.50:0/1541120500 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1a3145520 con 0x55e1a311a800 2024-02-14T15:13:16.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: starting mds.h at 2024-02-14T15:13:16.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:16 smithi050 bash[72353]: ae581fba06637b172763cd7b23ff86e4779013ed504bd322540049940ac59ce8 2024-02-14T15:13:16.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:13:16 smithi050 systemd[1]: Started Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:13:16.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1024422 10.94 MB/sec execute 378 sec latency 127.330 ms 2024-02-14T15:13:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:16 smithi050 ceph-mon[36512]: pgmap v308: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2024-02-14T15:13:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:16 smithi050 ceph-mon[36512]: daemon mds.h restarted 2024-02-14T15:13:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:16 smithi076 ceph-mon[37057]: pgmap v308: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2024-02-14T15:13:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:16 smithi076 ceph-mon[37057]: daemon mds.h restarted 2024-02-14T15:13:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:16 smithi017 ceph-mon[32549]: pgmap v308: 129 pgs: 129 active+clean; 572 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 216 op/s 2024-02-14T15:13:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:16 smithi017 ceph-mon[32549]: daemon mds.h restarted 2024-02-14T15:13:17.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1026764 10.96 MB/sec execute 379 sec latency 36.632 ms 2024-02-14T15:13:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:17 smithi050 ceph-mon[36512]: osdmap e105: 12 total, 12 up, 12 in 2024-02-14T15:13:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:17 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6836/812364823,v1:172.21.15.50:6837/812364823] up:boot 2024-02-14T15:13:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:17 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-14T15:13:18.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:17 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:13:18.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:17 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:13:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:17 smithi076 ceph-mon[37057]: osdmap e105: 12 total, 12 up, 12 in 2024-02-14T15:13:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:17 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6836/812364823,v1:172.21.15.50:6837/812364823] up:boot 2024-02-14T15:13:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:17 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-14T15:13:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:17 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:13:18.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:17 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:13:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:17 smithi017 ceph-mon[32549]: osdmap e105: 12 total, 12 up, 12 in 2024-02-14T15:13:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:17 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6836/812364823,v1:172.21.15.50:6837/812364823] up:boot 2024-02-14T15:13:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:17 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-14T15:13:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:17 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:13:18.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:17 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:13:18.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1028113 10.94 MB/sec execute 380 sec latency 15.576 ms 2024-02-14T15:13:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:18 smithi050 ceph-mon[36512]: pgmap v310: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-02-14T15:13:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:18 smithi076 ceph-mon[37057]: pgmap v310: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-02-14T15:13:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:18 smithi017 ceph-mon[32549]: pgmap v310: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-02-14T15:13:19.883 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1030347 10.95 MB/sec execute 381 sec latency 12.790 ms 2024-02-14T15:13:20.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1031420 10.93 MB/sec execute 382 sec latency 85.225 ms 2024-02-14T15:13:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:20 smithi050 ceph-mon[36512]: pgmap v311: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-02-14T15:13:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:20 smithi076 ceph-mon[37057]: pgmap v311: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-02-14T15:13:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:20 smithi017 ceph-mon[32549]: pgmap v311: 129 pgs: 129 active+clean; 586 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-02-14T15:13:21.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1033757 10.94 MB/sec execute 383 sec latency 41.195 ms 2024-02-14T15:13:22.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1034823 10.92 MB/sec execute 384 sec latency 20.239 ms 2024-02-14T15:13:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:22 smithi050 ceph-mon[36512]: pgmap v312: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:22 smithi076 ceph-mon[37057]: pgmap v312: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:22 smithi017 ceph-mon[32549]: pgmap v312: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:23.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1037323 10.93 MB/sec execute 385 sec latency 15.384 ms 2024-02-14T15:13:24.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1038553 10.91 MB/sec execute 386 sec latency 20.097 ms 2024-02-14T15:13:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:24 smithi050 ceph-mon[36512]: pgmap v313: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:24 smithi076 ceph-mon[37057]: pgmap v313: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:24 smithi017 ceph-mon[32549]: pgmap v313: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:25.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1040848 10.93 MB/sec execute 387 sec latency 125.661 ms 2024-02-14T15:13:26.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1041678 10.90 MB/sec execute 388 sec latency 28.693 ms 2024-02-14T15:13:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:26 smithi050 ceph-mon[36512]: pgmap v314: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:26 smithi076 ceph-mon[37057]: pgmap v314: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:26 smithi017 ceph-mon[32549]: pgmap v314: 129 pgs: 129 active+clean; 598 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 215 op/s 2024-02-14T15:13:27.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1044236 10.91 MB/sec execute 389 sec latency 24.713 ms 2024-02-14T15:13:28.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1045289 10.90 MB/sec execute 390 sec latency 60.029 ms 2024-02-14T15:13:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:28 smithi050 ceph-mon[36512]: pgmap v315: 129 pgs: 129 active+clean; 606 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 212 op/s 2024-02-14T15:13:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:28 smithi076 ceph-mon[37057]: pgmap v315: 129 pgs: 129 active+clean; 606 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 212 op/s 2024-02-14T15:13:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:28 smithi017 ceph-mon[32549]: pgmap v315: 129 pgs: 129 active+clean; 606 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 212 op/s 2024-02-14T15:13:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.452+0000 7ff4525a0640 1 -- 172.21.15.17:0/2446732473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c100180 msgr2=0x7ff44c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.452+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2446732473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c100180 0x7ff44c100560 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff448009cd0 tx=0x7ff44802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- 172.21.15.17:0/2446732473 shutdown_connections 2024-02-14T15:13:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2446732473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c105250 0x7ff44c10ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2446732473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff44c100b30 0x7ff44c104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2446732473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c100180 0x7ff44c100560 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- 172.21.15.17:0/2446732473 >> 172.21.15.17:0/2446732473 conn(0x7ff44c0fba00 msgr2=0x7ff44c0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- 172.21.15.17:0/2446732473 shutdown_connections 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- 172.21.15.17:0/2446732473 wait complete. 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 Processor -- start 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- start start 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c100180 0x7ff44c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff44c105250 0x7ff44c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c19bf80 0x7ff44c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c19c970 con 0x7ff44c19bf80 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c19cae0 con 0x7ff44c100180 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.453+0000 7ff4525a0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c19cc50 con 0x7ff44c105250 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c19bf80 0x7ff44c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c19bf80 0x7ff44c19c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59308/0 (socket says 172.21.15.17:59308) 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 -- 172.21.15.17:0/2867634474 learned_addr learned my addr 172.21.15.17:0/2867634474 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff45159e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c100180 0x7ff44c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff450d9d640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff44c105250 0x7ff44c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 -- 172.21.15.17:0/2867634474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff44c105250 msgr2=0x7ff44c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff44c105250 0x7ff44c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 -- 172.21.15.17:0/2867634474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c100180 msgr2=0x7ff44c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c100180 0x7ff44c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 -- 172.21.15.17:0/2867634474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448009900 con 0x7ff44c19bf80 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff451d9f640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c19bf80 0x7ff44c19c430 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff44400d950 tx=0x7ff44400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff4427fc640 1 -- 172.21.15.17:0/2867634474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444009a30 con 0x7ff44c19bf80 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff4427fc640 1 -- 172.21.15.17:0/2867634474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff444009b90 con 0x7ff44c19bf80 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c1a4230 con 0x7ff44c19bf80 2024-02-14T15:13:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff4427fc640 1 -- 172.21.15.17:0/2867634474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444003d60 con 0x7ff44c19bf80 2024-02-14T15:13:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.454+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c1a46e0 con 0x7ff44c19bf80 2024-02-14T15:13:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.456+0000 7ff4427fc640 1 -- 172.21.15.17:0/2867634474 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff444009d00 con 0x7ff44c19bf80 2024-02-14T15:13:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 Processor -- start 2024-02-14T15:13:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 -- start start 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4427fc640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ff4380a2290 0x7ff4380a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x560420968ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x5604209718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x5604209741f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604208d69e0 con 0x560420968900 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604208d3f10 con 0x560420969220 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4563d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604208cf6f0 con 0x560420971e00 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff4427fc640 1 -- 172.21.15.17:0/2867634474 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7ff4440cdd50 con 0x7ff44c19bf80 2024-02-14T15:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff45159e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x560420968ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff450d9d640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ff4380a2290 0x7ff4380a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff45159e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x560420968ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59318/0 (socket says 172.21.15.17:59318) 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff45159e640 1 -- 172.21.15.17:0/1762952988 learned_addr learned my addr 172.21.15.17:0/1762952988 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff451d9f640 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x5604209741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.457+0000 7ff450d9d640 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x5604209718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff451d9f640 1 -- 172.21.15.17:0/1762952988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 msgr2=0x5604209718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff451d9f640 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x5604209718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff451d9f640 1 -- 172.21.15.17:0/1762952988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 msgr2=0x560420968ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff451d9f640 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x560420968ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff451d9f640 1 -- 172.21.15.17:0/1762952988 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604205e0920 con 0x560420971e00 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff45159e640 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x560420968ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff450d9d640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ff4380a2290 0x7ff4380a4750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0fbf20 tx=0x7ff43c013880 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff451d9f640 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x5604209741f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4440cb140 tx=0x7ff444096050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13abea32a889dfed server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff4377fe640 1 -- 172.21.15.17:0/1762952988 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44409aae0 con 0x560420971e00 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff4377fe640 1 -- 172.21.15.17:0/1762952988 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44409a3e0 con 0x560420971e00 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.458+0000 7ff4377fe640 1 -- 172.21.15.17:0/1762952988 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44409e120 con 0x560420971e00 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- 172.21.15.17:0/1762952988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 msgr2=0x5604209741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x5604209741f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4440cb140 tx=0x7ff444096050 comp rx=0 tx=0).stop 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- 172.21.15.17:0/1762952988 shutdown_connections 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x5604209741f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x5604209718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1762952988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x560420968ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- 172.21.15.17:0/1762952988 >> 172.21.15.17:0/1762952988 conn(0x560420963030 msgr2=0x560420963410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- 172.21.15.17:0/1762952988 shutdown_connections 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- 172.21.15.17:0/1762952988 wait complete. 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- start start 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 Processor -- start 2024-02-14T15:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x5604209717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x56042096eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x56042096f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042086bfa0 con 0x560420968900 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604205a3250 con 0x560420969220 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.459+0000 7ff4563d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560420855200 con 0x560420971e00 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff4563d1740 10 client.0 _ll_register_callbacks cb 0x5604208c4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x5604209717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff451d9f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x56042096f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x5604209717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59322/0 (socket says 172.21.15.17:59322) 2024-02-14T15:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 -- 172.21.15.17:0/2053817553 learned_addr learned my addr 172.21.15.17:0/2053817553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff450d9d640 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x56042096eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 -- 172.21.15.17:0/2053817553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 msgr2=0x56042096f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x56042096f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 -- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 msgr2=0x56042096eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x56042096eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 -- 172.21.15.17:0/2053817553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c1a46e0 con 0x560420968900 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff450d9d640 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x56042096eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff45159e640 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x5604209717f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff448004af0 tx=0x7ff448034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dcd49674a1dfc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff434ff9640 1 -- 172.21.15.17:0/2053817553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4480328b0 con 0x560420968900 2024-02-14T15:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff434ff9640 1 -- 172.21.15.17:0/2053817553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7ff448046070 con 0x560420968900 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604205cf600 con 0x560420968900 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff434ff9640 1 -- 172.21.15.17:0/2053817553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff448032a10 con 0x560420968900 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff4563d1740 10 client.17115 fetch_fsmap learned FSMap version 33 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.460+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5604208cdd70 con 0x560420968900 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff434ff9640 1 -- 172.21.15.17:0/2053817553 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5604205cf600 con 0x560420968900 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff434ff9640 1 -- 172.21.15.17:0/2053817553 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4480045f0 con 0x560420968900 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff4563d1740 10 client.17115 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff4563d1740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff4563d1740 20 client.17115 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff4563d1740 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x560420981c30 0x560420984050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff4563d1740 4 client.17115 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:13:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604206201b0 con 0x560420981c30 2024-02-14T15:13:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff450d9d640 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x560420981c30 0x560420984050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff434ff9640 10 client.17115 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.461+0000 7ff450d9d640 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x560420981c30 0x560420984050 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bea14aa5a3d9712 server_cookie=d1903dbc634cb570 in_seq=0 out_seq=0 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff434ff9640 1 -- 172.21.15.17:0/2053817553 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604206201b0 con 0x560420981c30 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff434ff9640 10 client.17115 handle_command_reply: tid=0 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 client.17115 shutdown 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 msgr2=0x5604209717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x5604209717f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff448004af0 tx=0x7ff448034670 comp rx=0 tx=0).stop 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 shutdown_connections 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x560420981c30 0x560420984050 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420971e00 0x56042096f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420969220 0x56042096eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff434ff9640 0 client.17115 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 --2- 172.21.15.17:0/2053817553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420968900 0x5604209717f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 >> 172.21.15.17:0/2053817553 conn(0x560420963030 msgr2=0x560420964d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.462+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 shutdown_connections 2024-02-14T15:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.463+0000 7ff4563d1740 1 -- 172.21.15.17:0/2053817553 wait complete. 2024-02-14T15:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.463+0000 7ff4563d1740 20 client.17115 trim_cache size 0 max 16384 2024-02-14T15:13:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 Processor -- start 2024-02-14T15:13:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 -- start start 2024-02-14T15:13:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420971240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420971780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x56042096d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042096efd0 con 0x560420969220 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x560420964260 con 0x560420971e00 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff4563d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560420989220 con 0x560420968900 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff450d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420971780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.476+0000 7ff450d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420971780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59334/0 (socket says 172.21.15.17:59334) 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff451d9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x56042096d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff450d9d640 1 -- 172.21.15.17:0/3430742072 learned_addr learned my addr 172.21.15.17:0/3430742072 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff45159e640 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420971240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff451d9f640 1 -- 172.21.15.17:0/3430742072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 msgr2=0x560420971240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff451d9f640 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420971240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff451d9f640 1 -- 172.21.15.17:0/3430742072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 msgr2=0x560420971780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff451d9f640 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420971780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff451d9f640 1 -- 172.21.15.17:0/3430742072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56042086a820 con 0x560420971e00 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff450d9d640 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420971780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff451d9f640 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x56042096d450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff444009610 tx=0x7ff444003800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec0e9ee8378e6b55 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff436ffd640 1 -- 172.21.15.17:0/3430742072 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44409aae0 con 0x560420971e00 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff436ffd640 1 -- 172.21.15.17:0/3430742072 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44409e530 con 0x560420971e00 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.477+0000 7ff436ffd640 1 -- 172.21.15.17:0/3430742072 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440b8b50 con 0x560420971e00 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 -- 172.21.15.17:0/3430742072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 msgr2=0x56042096d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x56042096d450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff444009610 tx=0x7ff444003800 comp rx=0 tx=0).stop 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 -- 172.21.15.17:0/3430742072 shutdown_connections 2024-02-14T15:13:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420971780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x56042096d450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 --2- 172.21.15.17:0/3430742072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420971240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 -- 172.21.15.17:0/3430742072 >> 172.21.15.17:0/3430742072 conn(0x560420963030 msgr2=0x56042097d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 -- 172.21.15.17:0/3430742072 shutdown_connections 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 -- 172.21.15.17:0/3430742072 wait complete. 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 -- start start 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.478+0000 7ff4563d1740 1 Processor -- start 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420978f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420977330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff4563d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x560420977870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff4563d1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560420977f40 con 0x560420969220 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff4563d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604209780b0 con 0x560420971e00 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff4563d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604209d8100 con 0x560420968900 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff4563d1740 10 client.0 _ll_register_callbacks cb 0x7ff444005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff450d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420977330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff450d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420977330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59344/0 (socket says 172.21.15.17:59344) 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff451d9f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x560420977870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff450d9d640 1 -- 172.21.15.17:0/1245399872 learned_addr learned my addr 172.21.15.17:0/1245399872 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff45159e640 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420978f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff451d9f640 1 -- 172.21.15.17:0/1245399872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 msgr2=0x560420978f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff451d9f640 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420978f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff451d9f640 1 -- 172.21.15.17:0/1245399872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 msgr2=0x560420977330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff451d9f640 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420977330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff451d9f640 1 -- 172.21.15.17:0/1245399872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c012110 con 0x560420971e00 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff450d9d640 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420977330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff451d9f640 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x560420977870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5604209643a0 tx=0x7ff44400dfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46033f77666b8bcf server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff434ff9640 1 -- 172.21.15.17:0/1245399872 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444096ad0 con 0x560420971e00 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.479+0000 7ff434ff9640 1 -- 172.21.15.17:0/1245399872 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7ff44409aae0 con 0x560420971e00 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff44409a3e0 con 0x560420971e00 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff434ff9640 1 -- 172.21.15.17:0/1245399872 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff44409e320 con 0x560420971e00 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 10 client.26471 fetch_fsmap learned FSMap version 33 2024-02-14T15:13:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5604205e0920 con 0x560420971e00 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff434ff9640 1 -- 172.21.15.17:0/1245399872 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44409a3e0 con 0x560420971e00 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff434ff9640 1 -- 172.21.15.17:0/1245399872 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4440992a0 con 0x560420971e00 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 10 client.26471 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 10 client.26471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 20 client.26471 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5604209d8300 0x5604209da720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 4 client.26471 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56042060c550 con 0x5604209d8300 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.480+0000 7ff45159e640 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5604209d8300 0x5604209da720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.481+0000 7ff434ff9640 10 client.26471 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.481+0000 7ff45159e640 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5604209d8300 0x5604209da720 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab155307dda9d4fe server_cookie=206dc08c0941f108 in_seq=0 out_seq=0 2024-02-14T15:13:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff434ff9640 1 -- 172.21.15.17:0/1245399872 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56042060c550 con 0x5604209d8300 2024-02-14T15:13:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff434ff9640 10 client.26471 handle_command_reply: tid=0 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 client.26471 shutdown 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 msgr2=0x560420977870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x560420977870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5604209643a0 tx=0x7ff44400dfb0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 shutdown_connections 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5604209d8300 0x5604209da720 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560420969220 0x560420977330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560420971e00 0x560420977870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 --2- 172.21.15.17:0/1245399872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560420968900 0x560420978f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff434ff9640 0 client.26471 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 >> 172.21.15.17:0/1245399872 conn(0x560420963030 msgr2=0x56042096d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 shutdown_connections 2024-02-14T15:13:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 1 -- 172.21.15.17:0/1245399872 wait complete. 2024-02-14T15:13:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.492+0000 7ff4563d1740 20 client.26471 trim_cache size 0 max 16384 2024-02-14T15:13:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:13:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:13:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:13:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:13:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ff4380a2290 msgr2=0x7ff4380a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ff4380a2290 0x7ff4380a4750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0fbf20 tx=0x7ff43c013880 comp rx=0 tx=0).stop 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c19bf80 msgr2=0x7ff44c19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c19bf80 0x7ff44c19c430 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff44400d950 tx=0x7ff44400de10 comp rx=0 tx=0).stop 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 shutdown_connections 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7ff4380a2290 0x7ff4380a4750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff44c19bf80 0x7ff44c19c430 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff44c105250 0x7ff44c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 --2- 172.21.15.17:0/2867634474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c100180 0x7ff44c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 >> 172.21.15.17:0/2867634474 conn(0x7ff44c0fba00 msgr2=0x7ff44c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 shutdown_connections 2024-02-14T15:13:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:29.493+0000 7ff4525a0640 1 -- 172.21.15.17:0/2867634474 wait complete. 2024-02-14T15:13:29.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1047923 10.91 MB/sec execute 391 sec latency 21.209 ms 2024-02-14T15:13:30.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1048943 10.89 MB/sec execute 392 sec latency 30.962 ms 2024-02-14T15:13:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:30 smithi050 ceph-mon[36512]: pgmap v316: 129 pgs: 129 active+clean; 606 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 182 op/s 2024-02-14T15:13:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:30 smithi076 ceph-mon[37057]: pgmap v316: 129 pgs: 129 active+clean; 606 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 182 op/s 2024-02-14T15:13:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:30 smithi017 ceph-mon[32549]: pgmap v316: 129 pgs: 129 active+clean; 606 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 182 op/s 2024-02-14T15:13:31.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1051174 10.89 MB/sec execute 393 sec latency 83.444 ms 2024-02-14T15:13:32.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1052309 10.88 MB/sec execute 394 sec latency 19.804 ms 2024-02-14T15:13:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:32 smithi050 ceph-mon[36512]: pgmap v317: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-02-14T15:13:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:32 smithi076 ceph-mon[37057]: pgmap v317: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-02-14T15:13:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:32 smithi017 ceph-mon[32549]: pgmap v317: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-02-14T15:13:33.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1054803 10.89 MB/sec execute 395 sec latency 38.100 ms 2024-02-14T15:13:34.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1055979 10.88 MB/sec execute 396 sec latency 18.843 ms 2024-02-14T15:13:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:34 smithi050 ceph-mon[36512]: pgmap v318: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:13:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:34 smithi076 ceph-mon[37057]: pgmap v318: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:13:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:34 smithi017 ceph-mon[32549]: pgmap v318: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:13:35.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1058583 10.89 MB/sec execute 397 sec latency 25.663 ms 2024-02-14T15:13:36.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1059401 10.87 MB/sec execute 398 sec latency 114.505 ms 2024-02-14T15:13:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:36 smithi050 ceph-mon[36512]: pgmap v319: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:13:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:36 smithi076 ceph-mon[37057]: pgmap v319: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:13:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:36 smithi017 ceph-mon[32549]: pgmap v319: 129 pgs: 129 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 173 op/s 2024-02-14T15:13:37.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1062125 10.89 MB/sec execute 399 sec latency 13.435 ms 2024-02-14T15:13:38.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1062614 10.86 MB/sec execute 400 sec latency 246.163 ms 2024-02-14T15:13:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:38 smithi050 ceph-mon[36512]: pgmap v320: 129 pgs: 129 active+clean; 624 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 254 op/s 2024-02-14T15:13:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:38 smithi076 ceph-mon[37057]: pgmap v320: 129 pgs: 129 active+clean; 624 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 254 op/s 2024-02-14T15:13:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:38 smithi017 ceph-mon[32549]: pgmap v320: 129 pgs: 129 active+clean; 624 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 254 op/s 2024-02-14T15:13:39.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1064639 10.85 MB/sec execute 401 sec latency 22.609 ms 2024-02-14T15:13:40.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1066139 10.86 MB/sec execute 402 sec latency 21.420 ms 2024-02-14T15:13:41.091 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:40 smithi050 ceph-mon[36512]: pgmap v321: 129 pgs: 129 active+clean; 624 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-02-14T15:13:41.091 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:40 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:13:41.140 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:40 smithi076 ceph-mon[37057]: pgmap v321: 129 pgs: 129 active+clean; 624 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-02-14T15:13:41.140 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:40 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:13:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:40 smithi017 ceph-mon[32549]: pgmap v321: 129 pgs: 129 active+clean; 624 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-02-14T15:13:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:40 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:13:41.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1067392 10.84 MB/sec execute 403 sec latency 58.977 ms 2024-02-14T15:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 -- 172.21.15.17:0/1736192287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 msgr2=0x7fcb58100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 --2- 172.21.15.17:0/1736192287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 0x7fcb58100720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009930 tx=0x7fcb5402bc30 comp rx=0 tx=0).stop 2024-02-14T15:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 -- 172.21.15.17:0/1736192287 shutdown_connections 2024-02-14T15:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 --2- 172.21.15.17:0/1736192287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 0x7fcb5810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 --2- 172.21.15.17:0/1736192287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb58100cf0 0x7fcb58104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 --2- 172.21.15.17:0/1736192287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 0x7fcb58100720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 -- 172.21.15.17:0/1736192287 >> 172.21.15.17:0/1736192287 conn(0x7fcb58077220 msgr2=0x7fcb58077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 -- 172.21.15.17:0/1736192287 shutdown_connections 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.586+0000 7fcb60025640 1 -- 172.21.15.17:0/1736192287 wait complete. 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 Processor -- start 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 -- start start 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 0x7fcb581967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 0x7fcb58196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb5819b010 0x7fcb5819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5819ba00 con 0x7fcb58105410 2024-02-14T15:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5819bb70 con 0x7fcb58100340 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb60025640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5819bce0 con 0x7fcb5819b010 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 0x7fcb58196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 0x7fcb58196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47576/0 (socket says 172.21.15.17:47576) 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 0x7fcb581967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5e5c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb5819b010 0x7fcb5819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/1836342000 learned_addr learned my addr 172.21.15.17:0/1836342000 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/1836342000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb5819b010 msgr2=0x7fcb5819b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb5819b010 0x7fcb5819b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/1836342000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 msgr2=0x7fcb581967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 0x7fcb581967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/1836342000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009560 con 0x7fcb58105410 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 0x7fcb581967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.587+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 0x7fcb58196ce0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800ead0 tx=0x7fcb4800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.588+0000 7fcb4effd640 1 -- 172.21.15.17:0/1836342000 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48013070 con 0x7fcb58105410 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.588+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb58068a20 con 0x7fcb58105410 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.588+0000 7fcb4effd640 1 -- 172.21.15.17:0/1836342000 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb48005500 con 0x7fcb58105410 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.588+0000 7fcb4effd640 1 -- 172.21.15.17:0/1836342000 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48004d80 con 0x7fcb58105410 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.588+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb58068ed0 con 0x7fcb58105410 2024-02-14T15:13:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.590+0000 7fcb4effd640 1 -- 172.21.15.17:0/1836342000 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcb4800bbd0 con 0x7fcb58105410 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.590+0000 7fcb62bf0740 1 Processor -- start 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.590+0000 7fcb62bf0740 1 -- start start 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce34720 0x56505ce34b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce40010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505cda29e0 con 0x56505ce3dc20 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505cd9ff10 con 0x56505ce34720 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505cd9b6f0 con 0x56505ce35040 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb4effd640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcb440a2270 0x7fcb440a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5d5c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce40010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37170/0 (socket says 172.21.15.17:37170) 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/1257045910 learned_addr learned my addr 172.21.15.17:0/1257045910 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5e5c2640 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce34720 0x56505ce34b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5e5c2640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcb440a2270 0x7fcb440a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/1257045910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce34720 msgr2=0x56505ce34b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce34720 0x56505ce34b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/1257045910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 msgr2=0x56505ce40010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce40010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/1257045910 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56505caac920 con 0x56505ce35040 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb4effd640 1 -- 172.21.15.17:0/1836342000 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fcb480cfac0 con 0x7fcb58105410 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5e5c2640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcb440a2270 0x7fcb440a4730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005fd0 tx=0x7fcb50009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.591+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c240 tx=0x7fcb5402c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9470d706a6d3f42 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb43fff640 1 -- 172.21.15.17:0/1257045910 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5402c7e0 con 0x56505ce35040 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb43fff640 1 -- 172.21.15.17:0/1257045910 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb5402c940 con 0x56505ce35040 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb43fff640 1 -- 172.21.15.17:0/1257045910 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54005430 con 0x56505ce35040 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb62bf0740 1 -- 172.21.15.17:0/1257045910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 msgr2=0x56505ce3d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c240 tx=0x7fcb5402c6e0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb62bf0740 1 -- 172.21.15.17:0/1257045910 shutdown_connections 2024-02-14T15:13:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce40010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d6e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/1257045910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce34720 0x56505ce34b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.592+0000 7fcb62bf0740 1 -- 172.21.15.17:0/1257045910 >> 172.21.15.17:0/1257045910 conn(0x56505ce2ee50 msgr2=0x56505ce2f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 -- 172.21.15.17:0/1257045910 shutdown_connections 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 -- 172.21.15.17:0/1257045910 wait complete. 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 -- start start 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 Processor -- start 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce34720 0x56505ce3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce35040 0x56505ce3a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505cd37fa0 con 0x56505ce3dc20 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505ca6f250 con 0x56505ce35040 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.593+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505cd21200 con 0x56505ce34720 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb62bf0740 10 client.0 _ll_register_callbacks cb 0x56505cd90b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce34720 0x56505ce3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5e5c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce3aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce34720 0x56505ce3d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37186/0 (socket says 172.21.15.17:37186) 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/35711701 learned_addr learned my addr 172.21.15.17:0/35711701 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5d5c0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce35040 0x56505ce3a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/35711701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce35040 msgr2=0x56505ce3a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce35040 0x56505ce3a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/35711701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 msgr2=0x56505ce3aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce3aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/35711701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5810d7f0 con 0x56505ce34720 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce34720 0x56505ce3d5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009850 tx=0x7fcb54004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38442ef727bef70f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb417fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:13:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb417fa640 1 -- 172.21.15.17:0/35711701 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5402c7e0 con 0x56505ce34720 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56505ca9b600 con 0x56505ce34720 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.594+0000 7fcb417fa640 1 -- 172.21.15.17:0/35711701 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fcb5404a650 con 0x56505ce34720 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb417fa640 1 -- 172.21.15.17:0/35711701 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5404a810 con 0x56505ce34720 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 10 client.26653 fetch_fsmap learned FSMap version 33 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56505cd99d70 con 0x56505ce34720 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb417fa640 1 -- 172.21.15.17:0/35711701 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56505ca9b600 con 0x56505ce34720 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb417fa640 1 -- 172.21.15.17:0/35711701 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcb5403c050 con 0x56505ce34720 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 10 client.26653 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 10 client.26653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 20 client.26653 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505ce4d630 0x56505ce4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 4 client.26653 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56505caec1b0 con 0x56505ce4d630 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.595+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505ce4d630 0x56505ce4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb417fa640 10 client.26653 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505ce4d630 0x56505ce4fa20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed7abeba7528c0b server_cookie=d7b2cc9bd86666d6 in_seq=0 out_seq=0 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb417fa640 1 -- 172.21.15.17:0/35711701 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56505caec1b0 con 0x56505ce4d630 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb417fa640 10 client.26653 handle_command_reply: tid=0 2024-02-14T15:13:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb62bf0740 1 client.26653 shutdown 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce34720 msgr2=0x56505ce3d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce34720 0x56505ce3d5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009850 tx=0x7fcb54004f70 comp rx=0 tx=0).stop 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 shutdown_connections 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.596+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505ce4d630 0x56505ce4fa20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce3dc20 0x56505ce3aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb417fa640 0 client.26653 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce35040 0x56505ce3a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/35711701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce34720 0x56505ce3d5a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 >> 172.21.15.17:0/35711701 conn(0x56505ce2ee50 msgr2=0x56505ce30ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 shutdown_connections 2024-02-14T15:13:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb62bf0740 1 -- 172.21.15.17:0/35711701 wait complete. 2024-02-14T15:13:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.597+0000 7fcb62bf0740 20 client.26653 trim_cache size 0 max 16384 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 Processor -- start 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 -- start start 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce3a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505ce40570 con 0x56505ce34720 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505ce4d430 con 0x56505ce3dc20 2024-02-14T15:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.610+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505ce2fe50 con 0x56505ce35040 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce3a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce3a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47606/0 (socket says 172.21.15.17:47606) 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/3831506981 learned_addr learned my addr 172.21.15.17:0/3831506981 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5e5c2640 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/3831506981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 msgr2=0x56505ce3d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/3831506981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 msgr2=0x56505ce3d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce3d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 -- 172.21.15.17:0/3831506981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56505cd36820 con 0x56505ce34720 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5e5c2640 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce3d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce3a8d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c2f0 tx=0x7fcb54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eca54fdeffaeca8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb437fe640 1 -- 172.21.15.17:0/3831506981 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5402c7e0 con 0x56505ce34720 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb437fe640 1 -- 172.21.15.17:0/3831506981 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb5403fc60 con 0x56505ce34720 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.611+0000 7fcb437fe640 1 -- 172.21.15.17:0/3831506981 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5404a7d0 con 0x56505ce34720 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3831506981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 msgr2=0x56505ce3a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce3a8d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c2f0 tx=0x7fcb54009780 comp rx=0 tx=0).stop 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3831506981 shutdown_connections 2024-02-14T15:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce3d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce3d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3831506981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce3a8d0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3831506981 >> 172.21.15.17:0/3831506981 conn(0x56505ce2ee50 msgr2=0x56505ce3a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3831506981 shutdown_connections 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.612+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3831506981 wait complete. 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 -- start start 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 Processor -- start 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce44ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce43390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce43910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505ce44060 con 0x56505ce34720 2024-02-14T15:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505ce441d0 con 0x56505ce3dc20 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505cea3b00 con 0x56505ce35040 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb62bf0740 10 client.0 _ll_register_callbacks cb 0x7fcb48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb5ddc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce44ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb5d5c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce43390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb5e5c2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce43910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb5d5c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce43390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37204/0 (socket says 172.21.15.17:37204) 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/3873545515 learned_addr learned my addr 172.21.15.17:0/3873545515 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.613+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 msgr2=0x56505ce43910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce43910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/3873545515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 msgr2=0x56505ce44ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce44ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb5d5c0640 1 -- 172.21.15.17:0/3873545515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb50019040 con 0x56505ce35040 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce44ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb5e5c2640 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce43910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb5d5c0640 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce43390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb480058c0 tx=0x7fcb480d1f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c01cbb623c4bba4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb417fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb417fa640 1 -- 172.21.15.17:0/3873545515 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480a02a0 con 0x56505ce35040 2024-02-14T15:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb417fa640 1 -- 172.21.15.17:0/3873545515 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7fcb48098870 con 0x56505ce35040 2024-02-14T15:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb5402c940 con 0x56505ce35040 2024-02-14T15:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb417fa640 1 -- 172.21.15.17:0/3873545515 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb48098a30 con 0x56505ce35040 2024-02-14T15:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb62bf0740 10 client.26659 fetch_fsmap learned FSMap version 33 2024-02-14T15:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56505cd99d70 con 0x56505ce35040 2024-02-14T15:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb417fa640 1 -- 172.21.15.17:0/3873545515 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb5402c940 con 0x56505ce35040 2024-02-14T15:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb417fa640 1 -- 172.21.15.17:0/3873545515 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcb480afc10 con 0x56505ce35040 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.614+0000 7fcb62bf0740 10 client.26659 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb62bf0740 10 client.26659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb62bf0740 20 client.26659 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505cea52e0 0x56505cea7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb62bf0740 4 client.26659 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56505cad8550 con 0x56505cea52e0 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505cea52e0 0x56505cea7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb5ddc1640 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505cea52e0 0x56505cea7700 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454cd07f56e68f64 server_cookie=21e3ed916e7798d in_seq=0 out_seq=0 2024-02-14T15:13:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.615+0000 7fcb417fa640 10 client.26659 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb417fa640 1 -- 172.21.15.17:0/3873545515 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56505cad8550 con 0x56505cea52e0 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb417fa640 10 client.26659 handle_command_reply: tid=0 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 client.26659 shutdown 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 msgr2=0x56505ce43390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce43390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb480058c0 tx=0x7fcb480d1f90 comp rx=0 tx=0).stop 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 shutdown_connections 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56505cea52e0 0x56505cea7700 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56505ce3dc20 0x56505ce43910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56505ce35040 0x56505ce43390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb417fa640 0 client.26659 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 --2- 172.21.15.17:0/3873545515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56505ce34720 0x56505ce44ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.628+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 >> 172.21.15.17:0/3873545515 conn(0x56505ce2ee50 msgr2=0x56505ce35560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 shutdown_connections 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb62bf0740 1 -- 172.21.15.17:0/3873545515 wait complete. 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb62bf0740 20 client.26659 trim_cache size 0 max 16384 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:13:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:13:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcb440a2270 msgr2=0x7fcb440a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcb440a2270 0x7fcb440a4730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005fd0 tx=0x7fcb50009500 comp rx=0 tx=0).stop 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 msgr2=0x7fcb58196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 0x7fcb58196ce0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800ead0 tx=0x7fcb4800ef90 comp rx=0 tx=0).stop 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 shutdown_connections 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fcb440a2270 0x7fcb440a4730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58105410 0x7fcb58196ce0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb5819b010 0x7fcb5819b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 --2- 172.21.15.17:0/1836342000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb58100340 0x7fcb581967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 >> 172.21.15.17:0/1836342000 conn(0x7fcb58077220 msgr2=0x7fcb580fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 shutdown_connections 2024-02-14T15:13:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:42.629+0000 7fcb60025640 1 -- 172.21.15.17:0/1836342000 wait complete. 2024-02-14T15:13:42.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1069394 10.85 MB/sec execute 404 sec latency 15.662 ms 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: pgmap v322: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:42 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: pgmap v322: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:42 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: pgmap v322: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-02-14T15:13:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:42 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:43.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1070809 10.84 MB/sec execute 405 sec latency 60.938 ms 2024-02-14T15:13:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi050 to 175.2M 2024-02-14T15:13:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: Unable to set osd_memory_target on smithi050 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-02-14T15:13:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-02-14T15:13:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-14T15:13:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:13:44.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:13:44.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi050 to 175.2M 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: Unable to set osd_memory_target on smithi050 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:13:44.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:13:44.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:44.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi050 to 175.2M 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: Unable to set osd_memory_target on smithi050 to 183744512: error parsing value: Value '183744512' is below minimum 939524096 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:13:44.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:13:44.884 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1072900 10.84 MB/sec execute 406 sec latency 13.745 ms 2024-02-14T15:13:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:44 smithi050 ceph-mon[36512]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-02-14T15:13:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:44 smithi050 ceph-mon[36512]: Unable to set osd_memory_target on smithi017 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-14T15:13:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:44 smithi050 ceph-mon[36512]: pgmap v323: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 153 op/s 2024-02-14T15:13:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:44 smithi076 ceph-mon[37057]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-02-14T15:13:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:44 smithi076 ceph-mon[37057]: Unable to set osd_memory_target on smithi017 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-14T15:13:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:44 smithi076 ceph-mon[37057]: pgmap v323: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 153 op/s 2024-02-14T15:13:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:44 smithi017 ceph-mon[32549]: Adjusting osd_memory_target on smithi017 to 177.2M 2024-02-14T15:13:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:44 smithi017 ceph-mon[32549]: Unable to set osd_memory_target on smithi017 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-02-14T15:13:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:44 smithi017 ceph-mon[32549]: pgmap v323: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 153 op/s 2024-02-14T15:13:45.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1074430 10.83 MB/sec execute 407 sec latency 15.569 ms 2024-02-14T15:13:46.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1076464 10.83 MB/sec execute 408 sec latency 34.573 ms 2024-02-14T15:13:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:46 smithi050 ceph-mon[36512]: pgmap v324: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 153 op/s 2024-02-14T15:13:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:46 smithi076 ceph-mon[37057]: pgmap v324: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 153 op/s 2024-02-14T15:13:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:46 smithi017 ceph-mon[32549]: pgmap v324: 129 pgs: 129 active+clean; 578 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 153 op/s 2024-02-14T15:13:47.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1077886 10.82 MB/sec execute 409 sec latency 28.349 ms 2024-02-14T15:13:48.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1079980 10.83 MB/sec execute 410 sec latency 73.326 ms 2024-02-14T15:13:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:48 smithi050 ceph-mon[36512]: pgmap v325: 129 pgs: 129 active+clean; 588 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:13:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:48 smithi076 ceph-mon[37057]: pgmap v325: 129 pgs: 129 active+clean; 588 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:13:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:48 smithi017 ceph-mon[32549]: pgmap v325: 129 pgs: 129 active+clean; 588 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.3 MiB/s wr, 243 op/s 2024-02-14T15:13:49.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1081436 10.82 MB/sec execute 411 sec latency 19.065 ms 2024-02-14T15:13:50.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1083556 10.82 MB/sec execute 412 sec latency 22.715 ms 2024-02-14T15:13:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:50 smithi050 ceph-mon[36512]: pgmap v326: 129 pgs: 129 active+clean; 588 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-02-14T15:13:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:50 smithi076 ceph-mon[37057]: pgmap v326: 129 pgs: 129 active+clean; 588 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-02-14T15:13:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:50 smithi017 ceph-mon[32549]: pgmap v326: 129 pgs: 129 active+clean; 588 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-02-14T15:13:51.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1084873 10.81 MB/sec execute 413 sec latency 128.845 ms 2024-02-14T15:13:52.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1087041 10.81 MB/sec execute 414 sec latency 19.215 ms 2024-02-14T15:13:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:52 smithi050 ceph-mon[36512]: pgmap v327: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 241 op/s 2024-02-14T15:13:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:52 smithi076 ceph-mon[37057]: pgmap v327: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 241 op/s 2024-02-14T15:13:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:52 smithi017 ceph-mon[32549]: pgmap v327: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.5 MiB/s wr, 241 op/s 2024-02-14T15:13:53.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1088396 10.80 MB/sec execute 415 sec latency 17.376 ms 2024-02-14T15:13:54.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1090640 10.81 MB/sec execute 416 sec latency 13.695 ms 2024-02-14T15:13:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:54 smithi050 ceph-mon[36512]: pgmap v328: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-02-14T15:13:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:54 smithi076 ceph-mon[37057]: pgmap v328: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-02-14T15:13:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:54 smithi017 ceph-mon[32549]: pgmap v328: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-02-14T15:13:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 -- 172.21.15.17:0/4061478847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100cf0 msgr2=0x7f87e0104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 --2- 172.21.15.17:0/4061478847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100cf0 0x7f87e0104d30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009930 tx=0x7f87d402bc30 comp rx=0 tx=0).stop 2024-02-14T15:13:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 -- 172.21.15.17:0/4061478847 shutdown_connections 2024-02-14T15:13:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 --2- 172.21.15.17:0/4061478847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100cf0 0x7f87e0104d30 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 --2- 172.21.15.17:0/4061478847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0105410 0x7f87e010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 --2- 172.21.15.17:0/4061478847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87e0100340 0x7f87e0100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 -- 172.21.15.17:0/4061478847 >> 172.21.15.17:0/4061478847 conn(0x7f87e0077220 msgr2=0x7f87e0077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 -- 172.21.15.17:0/4061478847 shutdown_connections 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 -- 172.21.15.17:0/4061478847 wait complete. 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.719+0000 7f87e5f13640 1 Processor -- start 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87e5f13640 1 -- start start 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87e5f13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100340 0x7f87e019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87e5f13640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0105410 0x7f87e019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87e5f13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87e01998b0 0x7f87e0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87e5f13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e019a2a0 con 0x7f87e0100340 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87e5f13640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e019a410 con 0x7f87e0105410 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87e5f13640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e019a5d0 con 0x7f87e01998b0 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100340 0x7f87e019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100340 0x7f87e019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39250/0 (socket says 172.21.15.17:39250) 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 -- 172.21.15.17:0/3192255501 learned_addr learned my addr 172.21.15.17:0/3192255501 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87dffff640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87e01998b0 0x7f87e0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87deffd640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0105410 0x7f87e019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 -- 172.21.15.17:0/3192255501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87e01998b0 msgr2=0x7f87e0199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87e01998b0 0x7f87e0199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 -- 172.21.15.17:0/3192255501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0105410 msgr2=0x7f87e019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0105410 0x7f87e019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 -- 172.21.15.17:0/3192255501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4009560 con 0x7f87e0100340 2024-02-14T15:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87df7fe640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100340 0x7f87e019f700 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f87d000e9f0 tx=0x7f87d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.720+0000 7f87dcff9640 1 -- 172.21.15.17:0/3192255501 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0009720 con 0x7f87e0100340 2024-02-14T15:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.721+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e01a65c0 con 0x7f87e0100340 2024-02-14T15:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.721+0000 7f87dcff9640 1 -- 172.21.15.17:0/3192255501 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d000cd80 con 0x7f87e0100340 2024-02-14T15:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.721+0000 7f87dcff9640 1 -- 172.21.15.17:0/3192255501 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0010690 con 0x7f87e0100340 2024-02-14T15:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.721+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01a6a50 con 0x7f87e0100340 2024-02-14T15:13:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.723+0000 7f87dcff9640 1 -- 172.21.15.17:0/3192255501 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f87d00046c0 con 0x7f87e0100340 2024-02-14T15:13:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.723+0000 7f87e8bc3740 1 Processor -- start 2024-02-14T15:13:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.723+0000 7f87e8bc3740 1 -- start start 2024-02-14T15:13:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743e6720 0x55e3743e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743efc20 0x55e3743f2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743549e0 con 0x55e3743efc20 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e374351f10 con 0x55e3743e6720 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37434d6f0 con 0x55e3743e7040 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87dcff9640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87c80a2220 0x7f87c80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743efc20 0x55e3743f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87dffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743e6720 0x55e3743e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743efc20 0x55e3743f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39254/0 (socket says 172.21.15.17:39254) 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 -- 172.21.15.17:0/2750509171 learned_addr learned my addr 172.21.15.17:0/2750509171 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:46730/0 (socket says 172.21.15.17:46730) 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87dffff640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87c80a2220 0x7f87c80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87dcff9640 1 -- 172.21.15.17:0/3192255501 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f87d0014070 con 0x7f87e0100340 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 -- 172.21.15.17:0/2750509171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 msgr2=0x55e3743ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 -- 172.21.15.17:0/2750509171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743e6720 msgr2=0x55e3743e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743e6720 0x55e3743e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 -- 172.21.15.17:0/2750509171 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37405e920 con 0x55e3743efc20 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87dffff640 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743e6720 0x55e3743e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87df7fe640 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87dffff640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87c80a2220 0x7f87c80a46e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005fd0 tx=0x7f87d8009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.724+0000 7f87deffd640 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743efc20 0x55e3743f2010 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005ab0 tx=0x7f87d402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbab16716c20d33 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87c5ffb640 1 -- 172.21.15.17:0/2750509171 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d40049c0 con 0x55e3743efc20 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87c5ffb640 1 -- 172.21.15.17:0/2750509171 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d4004040 con 0x55e3743efc20 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87c5ffb640 1 -- 172.21.15.17:0/2750509171 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4005210 con 0x55e3743efc20 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2750509171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743efc20 msgr2=0x55e3743f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743efc20 0x55e3743f2010 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005ab0 tx=0x7f87d402c650 comp rx=0 tx=0).stop 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2750509171 shutdown_connections 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743efc20 0x55e3743f2010 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2750509171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743e6720 0x55e3743e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2750509171 >> 172.21.15.17:0/2750509171 conn(0x55e3743e0e50 msgr2=0x55e3743e1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2750509171 shutdown_connections 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.725+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2750509171 wait complete. 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 -- start start 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 Processor -- start 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3742e9fa0 con 0x55e3743e6720 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e374021250 con 0x55e3743efc20 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3742d3200 con 0x55e3743e7040 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87e8bc3740 10 client.0 _ll_register_callbacks cb 0x55e374342b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39256/0 (socket says 172.21.15.17:39256) 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87df7fe640 1 -- 172.21.15.17:0/2334385124 learned_addr learned my addr 172.21.15.17:0/2334385124 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87deffd640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.726+0000 7f87dffff640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87df7fe640 1 -- 172.21.15.17:0/2334385124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 msgr2=0x55e3743ec960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87df7fe640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ec960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87df7fe640 1 -- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 msgr2=0x55e3743ecf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87df7fe640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ecf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87df7fe640 1 -- 172.21.15.17:0/2334385124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e010d7f0 con 0x55e3743e6720 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87deffd640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87df7fe640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef5d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f87d00cbc90 tx=0x7f87d009a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9091862041dece8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87beffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87beffd640 1 -- 172.21.15.17:0/2334385124 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0064030 con 0x55e3743e6720 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87beffd640 1 -- 172.21.15.17:0/2334385124 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f87d0065070 con 0x55e3743e6720 2024-02-14T15:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e37404d600 con 0x55e3743e6720 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87beffd640 1 -- 172.21.15.17:0/2334385124 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d00cc0c0 con 0x55e3743e6720 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87e8bc3740 10 client.17151 fetch_fsmap learned FSMap version 33 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e37434bd70 con 0x55e3743e6720 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.727+0000 7f87beffd640 1 -- 172.21.15.17:0/2334385124 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e37404d600 con 0x55e3743e6720 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87beffd640 1 -- 172.21.15.17:0/2334385124 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87d001f060 con 0x55e3743e6720 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87e8bc3740 10 client.17151 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87e8bc3740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87e8bc3740 20 client.17151 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e3743ff9b0 0x55e374401da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87e8bc3740 4 client.17151 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e37409e1b0 con 0x55e3743ff9b0 2024-02-14T15:13:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87deffd640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e3743ff9b0 0x55e374401da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87beffd640 10 client.17151 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.728+0000 7f87deffd640 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e3743ff9b0 0x55e374401da0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c1741854558581 server_cookie=e3baaefadb847fa3 in_seq=0 out_seq=0 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87beffd640 1 -- 172.21.15.17:0/2334385124 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e37409e1b0 con 0x55e3743ff9b0 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87beffd640 10 client.17151 handle_command_reply: tid=0 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 client.17151 shutdown 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 msgr2=0x55e3743ef5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef5d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f87d00cbc90 tx=0x7f87d009a830 comp rx=0 tx=0).stop 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 shutdown_connections 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e3743ff9b0 0x55e374401da0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ecf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87beffd640 0 client.17151 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/2334385124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef5d0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 >> 172.21.15.17:0/2334385124 conn(0x55e3743e0e50 msgr2=0x55e3743e2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 shutdown_connections 2024-02-14T15:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 1 -- 172.21.15.17:0/2334385124 wait complete. 2024-02-14T15:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.729+0000 7f87e8bc3740 20 client.17151 trim_cache size 0 max 16384 2024-02-14T15:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.743+0000 7f87e8bc3740 1 Processor -- start 2024-02-14T15:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.743+0000 7f87e8bc3740 1 -- start start 2024-02-14T15:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743ff7b0 con 0x55e3743e6720 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743dc060 con 0x55e3743efc20 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743df2d0 con 0x55e3743e7040 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39266/0 (socket says 172.21.15.17:39266) 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 -- 172.21.15.17:0/1241321837 learned_addr learned my addr 172.21.15.17:0/1241321837 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87deffd640 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87dffff640 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 -- 172.21.15.17:0/1241321837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 msgr2=0x55e3743ef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 -- 172.21.15.17:0/1241321837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 msgr2=0x55e3743ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 -- 172.21.15.17:0/1241321837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3742e8820 con 0x55e3743e6720 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.744+0000 7f87df7fe640 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef090 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f87d0099180 tx=0x7f87d009b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e05f0e4581414d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.745+0000 7f87c57fa640 1 -- 172.21.15.17:0/1241321837 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0064030 con 0x55e3743e6720 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.745+0000 7f87c57fa640 1 -- 172.21.15.17:0/1241321837 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d009b5d0 con 0x55e3743e6720 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.745+0000 7f87c57fa640 1 -- 172.21.15.17:0/1241321837 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d00a31e0 con 0x55e3743e6720 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- 172.21.15.17:0/1241321837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 msgr2=0x55e3743ef090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef090 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f87d0099180 tx=0x7f87d009b050 comp rx=0 tx=0).stop 2024-02-14T15:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- 172.21.15.17:0/1241321837 shutdown_connections 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/1241321837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743ef090 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- 172.21.15.17:0/1241321837 >> 172.21.15.17:0/1241321837 conn(0x55e3743e0e50 msgr2=0x55e3743eb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- 172.21.15.17:0/1241321837 shutdown_connections 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- 172.21.15.17:0/1241321837 wait complete. 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- start start 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 Processor -- start 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743f5580 con 0x55e3743e6720 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743f56f0 con 0x55e3743efc20 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.746+0000 7f87e8bc3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743f58b0 con 0x55e3743e7040 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87e8bc3740 10 client.0 _ll_register_callbacks cb 0x7f87d80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39276/0 (socket says 172.21.15.17:39276) 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 -- 172.21.15.17:0/178036874 learned_addr learned my addr 172.21.15.17:0/178036874 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87deffd640 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87dffff640 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 -- 172.21.15.17:0/178036874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 msgr2=0x55e3743f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 -- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 msgr2=0x55e3743f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 -- 172.21.15.17:0/178036874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e37434bd70 con 0x55e3743e6720 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87df7fe640 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743f6820 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f87d009ae60 tx=0x7f87d00ce920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafb729b98577006 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.747+0000 7f87beffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87beffd640 1 -- 172.21.15.17:0/178036874 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0064070 con 0x55e3743e6720 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87beffd640 1 -- 172.21.15.17:0/178036874 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f87d0065070 con 0x55e3743e6720 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e37408a550 con 0x55e3743e6720 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87beffd640 1 -- 172.21.15.17:0/178036874 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d009b5d0 con 0x55e3743e6720 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 10 client.17157 fetch_fsmap learned FSMap version 33 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e37430e720 con 0x55e3743e6720 2024-02-14T15:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87beffd640 1 -- 172.21.15.17:0/178036874 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e37408a550 con 0x55e3743e6720 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87beffd640 1 -- 172.21.15.17:0/178036874 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87d009ff40 con 0x55e3743e6720 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 10 client.17157 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 20 client.17157 populate_metadata read hostname 'smithi017' 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e374456bc0 0x55e374458fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 4 client.17157 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e373ff9520 con 0x55e374456bc0 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.748+0000 7f87deffd640 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e374456bc0 0x55e374458fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.749+0000 7f87beffd640 10 client.17157 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.749+0000 7f87deffd640 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e374456bc0 0x55e374458fb0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fdd4085adfb0423 server_cookie=7be522a890fa472f in_seq=0 out_seq=0 2024-02-14T15:13:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.760+0000 7f87beffd640 1 -- 172.21.15.17:0/178036874 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e373ff9520 con 0x55e374456bc0 2024-02-14T15:13:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.760+0000 7f87beffd640 10 client.17157 handle_command_reply: tid=0 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.760+0000 7f87e8bc3740 1 client.17157 shutdown 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 msgr2=0x55e3743f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743f6820 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f87d009ae60 tx=0x7f87d00ce920 comp rx=0 tx=0).stop 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 shutdown_connections 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55e374456bc0 0x55e374458fb0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e3743efc20 0x55e3743f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3743e7040 0x55e3743f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87beffd640 0 client.17157 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 --2- 172.21.15.17:0/178036874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e3743e6720 0x55e3743f6820 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 >> 172.21.15.17:0/178036874 conn(0x55e3743e0e50 msgr2=0x55e3743ec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 shutdown_connections 2024-02-14T15:13:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 1 -- 172.21.15.17:0/178036874 wait complete. 2024-02-14T15:13:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e8bc3740 20 client.17157 trim_cache size 0 max 16384 2024-02-14T15:13:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:13:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:13:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:13:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:13:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87c80a2220 msgr2=0x7f87c80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87c80a2220 0x7f87c80a46e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005fd0 tx=0x7f87d8009500 comp rx=0 tx=0).stop 2024-02-14T15:13:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100340 msgr2=0x7f87e019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100340 0x7f87e019f700 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f87d000e9f0 tx=0x7f87d000edf0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 shutdown_connections 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87c80a2220 0x7f87c80a46e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87e01998b0 0x7f87e0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.761+0000 7f87e5f13640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0105410 0x7f87e019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.762+0000 7f87e5f13640 1 --2- 172.21.15.17:0/3192255501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87e0100340 0x7f87e019f700 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.762+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 >> 172.21.15.17:0/3192255501 conn(0x7f87e0077220 msgr2=0x7f87e0068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.762+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 shutdown_connections 2024-02-14T15:13:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:13:55.762+0000 7f87e5f13640 1 -- 172.21.15.17:0/3192255501 wait complete. 2024-02-14T15:13:55.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1092068 10.80 MB/sec execute 417 sec latency 95.755 ms 2024-02-14T15:13:56.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1094142 10.80 MB/sec execute 418 sec latency 68.952 ms 2024-02-14T15:13:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:56 smithi050 ceph-mon[36512]: pgmap v329: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-02-14T15:13:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:56 smithi076 ceph-mon[37057]: pgmap v329: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-02-14T15:13:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:56 smithi017 ceph-mon[32549]: pgmap v329: 129 pgs: 129 active+clean; 598 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 169 op/s 2024-02-14T15:13:57.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1095311 10.79 MB/sec execute 419 sec latency 19.542 ms 2024-02-14T15:13:58.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1097688 10.80 MB/sec execute 420 sec latency 15.069 ms 2024-02-14T15:13:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:13:58 smithi050 ceph-mon[36512]: pgmap v330: 129 pgs: 129 active+clean; 609 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 260 op/s 2024-02-14T15:13:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:13:58 smithi076 ceph-mon[37057]: pgmap v330: 129 pgs: 129 active+clean; 609 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 260 op/s 2024-02-14T15:13:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:13:58 smithi017 ceph-mon[32549]: pgmap v330: 129 pgs: 129 active+clean; 609 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 260 op/s 2024-02-14T15:13:59.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1098767 10.78 MB/sec execute 421 sec latency 28.600 ms 2024-02-14T15:14:00.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1099887 10.76 MB/sec execute 422 sec latency 58.520 ms 2024-02-14T15:14:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:00 smithi050 ceph-mon[36512]: pgmap v331: 129 pgs: 129 active+clean; 609 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-02-14T15:14:01.218 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:00 smithi017 ceph-mon[32549]: pgmap v331: 129 pgs: 129 active+clean; 609 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-02-14T15:14:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:00 smithi076 ceph-mon[37057]: pgmap v331: 129 pgs: 129 active+clean; 609 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-02-14T15:14:01.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1101233 10.77 MB/sec execute 423 sec latency 86.645 ms 2024-02-14T15:14:02.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1102328 10.75 MB/sec execute 424 sec latency 21.650 ms 2024-02-14T15:14:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:02 smithi050 ceph-mon[36512]: pgmap v332: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 280 op/s 2024-02-14T15:14:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:02 smithi076 ceph-mon[37057]: pgmap v332: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 280 op/s 2024-02-14T15:14:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:02 smithi017 ceph-mon[32549]: pgmap v332: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 280 op/s 2024-02-14T15:14:03.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1104690 10.76 MB/sec execute 425 sec latency 16.529 ms 2024-02-14T15:14:04.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1105693 10.74 MB/sec execute 426 sec latency 21.040 ms 2024-02-14T15:14:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:04 smithi050 ceph-mon[36512]: pgmap v333: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 200 op/s 2024-02-14T15:14:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:04 smithi076 ceph-mon[37057]: pgmap v333: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 200 op/s 2024-02-14T15:14:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:04 smithi017 ceph-mon[32549]: pgmap v333: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 200 op/s 2024-02-14T15:14:05.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1108219 10.75 MB/sec execute 427 sec latency 64.186 ms 2024-02-14T15:14:06.885 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1108969 10.73 MB/sec execute 428 sec latency 48.320 ms 2024-02-14T15:14:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:06 smithi050 ceph-mon[36512]: pgmap v334: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 200 op/s 2024-02-14T15:14:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:06 smithi076 ceph-mon[37057]: pgmap v334: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 200 op/s 2024-02-14T15:14:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:06 smithi017 ceph-mon[32549]: pgmap v334: 129 pgs: 129 active+clean; 617 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 200 op/s 2024-02-14T15:14:07.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1111211 10.74 MB/sec execute 429 sec latency 39.987 ms 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 -- 172.21.15.17:0/3689862708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c101450 msgr2=0x7fcd0c101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 --2- 172.21.15.17:0/3689862708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c101450 0x7fcd0c101830 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009930 tx=0x7fcd0802bc30 comp rx=0 tx=0).stop 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 -- 172.21.15.17:0/3689862708 shutdown_connections 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 --2- 172.21.15.17:0/3689862708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd0c106520 0x7fcd0c10e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 --2- 172.21.15.17:0/3689862708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd0c101e00 0x7fcd0c105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 --2- 172.21.15.17:0/3689862708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c101450 0x7fcd0c101830 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 -- 172.21.15.17:0/3689862708 >> 172.21.15.17:0/3689862708 conn(0x7fcd0c0fccf0 msgr2=0x7fcd0c0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 -- 172.21.15.17:0/3689862708 shutdown_connections 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.855+0000 7fcd1474f640 1 -- 172.21.15.17:0/3689862708 wait complete. 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.856+0000 7fcd1474f640 1 Processor -- start 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.856+0000 7fcd1474f640 1 -- start start 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.856+0000 7fcd1474f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd0c101450 0x7fcd0c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.856+0000 7fcd1474f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c106520 0x7fcd0c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.856+0000 7fcd1474f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd0c197790 0x7fcd0c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.856+0000 7fcd1474f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c198240 con 0x7fcd0c106520 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd1474f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c1983b0 con 0x7fcd0c101450 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd1474f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c198520 con 0x7fcd0c197790 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c106520 0x7fcd0c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c106520 0x7fcd0c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53606/0 (socket says 172.21.15.17:53606) 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 -- 172.21.15.17:0/1670150832 learned_addr learned my addr 172.21.15.17:0/1670150832 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd124eb640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd0c101450 0x7fcd0c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd12cec640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd0c197790 0x7fcd0c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 -- 172.21.15.17:0/1670150832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd0c197790 msgr2=0x7fcd0c197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd0c197790 0x7fcd0c197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 -- 172.21.15.17:0/1670150832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd0c101450 msgr2=0x7fcd0c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd0c101450 0x7fcd0c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 -- 172.21.15.17:0/1670150832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08009560 con 0x7fcd0c106520 2024-02-14T15:14:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.857+0000 7fcd11cea640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c106520 0x7fcd0c19db60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e9c0 tx=0x7fccfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.861+0000 7fcd037fe640 1 -- 172.21.15.17:0/1670150832 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc009700 con 0x7fcd0c106520 2024-02-14T15:14:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.862+0000 7fcd037fe640 1 -- 172.21.15.17:0/1670150832 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccfc00cd80 con 0x7fcd0c106520 2024-02-14T15:14:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.862+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c073460 con 0x7fcd0c106520 2024-02-14T15:14:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.862+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c073920 con 0x7fcd0c106520 2024-02-14T15:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.862+0000 7fcd037fe640 1 -- 172.21.15.17:0/1670150832 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc010430 con 0x7fcd0c106520 2024-02-14T15:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 Processor -- start 2024-02-14T15:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 -- start start 2024-02-14T15:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678564c9e0 con 0x5567856e7500 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556785649f10 con 0x5567856f8ea0 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.867+0000 7fcd173fb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567856456f0 con 0x5567856e6be0 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.868+0000 7fcd037fe640 1 -- 172.21.15.17:0/1670150832 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fccfc0106f0 con 0x7fcd0c106520 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.868+0000 7fcd12cec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.868+0000 7fcd12cec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33298/0 (socket says 172.21.15.17:33298) 2024-02-14T15:14:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.868+0000 7fcd12cec640 1 -- 172.21.15.17:0/3029284926 learned_addr learned my addr 172.21.15.17:0/3029284926 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.869+0000 7fcd037fe640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccf80a23f0 0x7fccf80a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.869+0000 7fcd037fe640 1 -- 172.21.15.17:0/1670150832 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9563+0+0 (secure 0 0 0) 0x7fccfc014070 con 0x7fcd0c106520 2024-02-14T15:14:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd124eb640 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.870+0000 7fcd11cea640 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd12cec640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccf80a23f0 0x7fccf80a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd12cec640 1 -- 172.21.15.17:0/3029284926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 msgr2=0x5567856fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd12cec640 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd12cec640 1 -- 172.21.15.17:0/3029284926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 msgr2=0x5567856e0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd12cec640 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd12cec640 1 -- 172.21.15.17:0/3029284926 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556785356920 con 0x5567856e6be0 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.871+0000 7fcd11cea640 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:14:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.872+0000 7fcd12cec640 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e6fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd04011af0 tx=0x7fcd0400f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f6ecde0c976cd8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.872+0000 7fcd12cec640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccf80a23f0 0x7fccf80a48b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd04023000 tx=0x7fcd04020030 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.873+0000 7fcd00ff9640 1 -- 172.21.15.17:0/3029284926 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04004d20 con 0x5567856e6be0 2024-02-14T15:14:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.873+0000 7fcd00ff9640 1 -- 172.21.15.17:0/3029284926 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd04022020 con 0x5567856e6be0 2024-02-14T15:14:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.873+0000 7fcd00ff9640 1 -- 172.21.15.17:0/3029284926 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040108f0 con 0x5567856e6be0 2024-02-14T15:14:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.874+0000 7fcd173fb740 1 -- 172.21.15.17:0/3029284926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 msgr2=0x5567856e6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.874+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e6fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd04011af0 tx=0x7fcd0400f3c0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.874+0000 7fcd173fb740 1 -- 172.21.15.17:0/3029284926 shutdown_connections 2024-02-14T15:14:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.874+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.874+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.874+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3029284926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e6fc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.875+0000 7fcd173fb740 1 -- 172.21.15.17:0/3029284926 >> 172.21.15.17:0/3029284926 conn(0x5567856e1660 msgr2=0x5567856e1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.875+0000 7fcd173fb740 1 -- 172.21.15.17:0/3029284926 shutdown_connections 2024-02-14T15:14:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.876+0000 7fcd173fb740 1 -- 172.21.15.17:0/3029284926 wait complete. 2024-02-14T15:14:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.877+0000 7fcd173fb740 1 -- start start 2024-02-14T15:14:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.877+0000 7fcd173fb740 1 Processor -- start 2024-02-14T15:14:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.878+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.881 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.878+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e7500 0x5567856dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.878+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856f8ea0 0x5567856ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.878+0000 7fcd173fb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567855e1fa0 con 0x5567856f8ea0 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.878+0000 7fcd173fb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556785319250 con 0x5567856e7500 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.878+0000 7fcd173fb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567855cb200 con 0x5567856e6be0 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.878+0000 7fcd173fb740 10 client.0 _ll_register_callbacks cb 0x55678563ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.879+0000 7fcd12cec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856f8ea0 0x5567856ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.879+0000 7fcd12cec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856f8ea0 0x5567856ddf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53612/0 (socket says 172.21.15.17:53612) 2024-02-14T15:14:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.879+0000 7fcd12cec640 1 -- 172.21.15.17:0/2141375033 learned_addr learned my addr 172.21.15.17:0/2141375033 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.880+0000 7fcd11cea640 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e7500 0x5567856dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.880+0000 7fcd12cec640 1 -- 172.21.15.17:0/2141375033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 msgr2=0x5567856e0650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.880+0000 7fcd12cec640 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e0650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.880+0000 7fcd12cec640 1 -- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e7500 msgr2=0x5567856dd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.880+0000 7fcd12cec640 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e7500 0x5567856dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.880+0000 7fcd12cec640 1 -- 172.21.15.17:0/2141375033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd040116a0 con 0x5567856f8ea0 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.881+0000 7fcd12cec640 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856f8ea0 0x5567856ddf20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcd04005fd0 tx=0x7fcd04011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add2367750a5a488 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.881+0000 7fccf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.882+0000 7fccf5ffb640 1 -- 172.21.15.17:0/2141375033 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0402c070 con 0x5567856f8ea0 2024-02-14T15:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.882+0000 7fccf5ffb640 1 -- 172.21.15.17:0/2141375033 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9563+0+0 (secure 0 0 0) 0x7fcd04023030 con 0x5567856f8ea0 2024-02-14T15:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.882+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556785345600 con 0x5567856f8ea0 2024-02-14T15:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.882+0000 7fccf5ffb640 1 -- 172.21.15.17:0/2141375033 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd04010dc0 con 0x5567856f8ea0 2024-02-14T15:14:08.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1112507 10.73 MB/sec execute 430 sec latency 20.948 ms 2024-02-14T15:14:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.885+0000 7fcd173fb740 10 client.17169 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.885+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556785643d70 con 0x5567856f8ea0 2024-02-14T15:14:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fccf5ffb640 1 -- 172.21.15.17:0/2141375033 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556785345600 con 0x5567856f8ea0 2024-02-14T15:14:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fccf5ffb640 1 -- 172.21.15.17:0/2141375033 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd04027550 con 0x5567856f8ea0 2024-02-14T15:14:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fcd173fb740 10 client.17169 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fcd173fb740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fcd173fb740 20 client.17169 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fcd173fb740 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556785708b00 0x55678570af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fcd173fb740 4 client.17169 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:14:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.886+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567853961b0 con 0x556785708b00 2024-02-14T15:14:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.888+0000 7fcd124eb640 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556785708b00 0x55678570af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.889+0000 7fccf5ffb640 10 client.17169 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.889+0000 7fcd124eb640 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556785708b00 0x55678570af20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed585e47d3cdde68 server_cookie=6ebbc04315a99632 in_seq=0 out_seq=0 2024-02-14T15:14:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.891+0000 7fccf5ffb640 1 -- 172.21.15.17:0/2141375033 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567853961b0 con 0x556785708b00 2024-02-14T15:14:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.891+0000 7fccf5ffb640 10 client.17169 handle_command_reply: tid=0 2024-02-14T15:14:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.891+0000 7fcd173fb740 1 client.17169 shutdown 2024-02-14T15:14:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.892+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856f8ea0 msgr2=0x5567856ddf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.892+0000 7fcd173fb740 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856f8ea0 0x5567856ddf20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcd04005fd0 tx=0x7fcd04011620 comp rx=0 tx=0).stop 2024-02-14T15:14:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.894+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 shutdown_connections 2024-02-14T15:14:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.894+0000 7fcd173fb740 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556785708b00 0x55678570af20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.894+0000 7fcd173fb740 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856f8ea0 0x5567856ddf20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.894+0000 7fcd173fb740 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e7500 0x5567856dd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.894+0000 7fcd173fb740 1 --2- 172.21.15.17:0/2141375033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e6be0 0x5567856e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.894+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 >> 172.21.15.17:0/2141375033 conn(0x5567856e1660 msgr2=0x5567856e33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.895+0000 7fccf5ffb640 0 client.17169 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.895+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 shutdown_connections 2024-02-14T15:14:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.896+0000 7fcd173fb740 1 -- 172.21.15.17:0/2141375033 wait complete. 2024-02-14T15:14:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.896+0000 7fcd173fb740 20 client.17169 trim_cache size 0 max 16384 2024-02-14T15:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.932+0000 7fcd173fb740 1 Processor -- start 2024-02-14T15:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd173fb740 1 -- start start 2024-02-14T15:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e6be0 0x5567856e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856f8ea0 0x5567856ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd173fb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556785705570 con 0x5567856e7500 2024-02-14T15:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd173fb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567856e2690 con 0x5567856e6be0 2024-02-14T15:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd173fb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567856e0f20 con 0x5567856f8ea0 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.933+0000 7fcd11cea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.934+0000 7fcd11cea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53624/0 (socket says 172.21.15.17:53624) 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.934+0000 7fcd11cea640 1 -- 172.21.15.17:0/513579882 learned_addr learned my addr 172.21.15.17:0/513579882 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.935+0000 7fcd12cec640 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856f8ea0 0x5567856ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.935+0000 7fcd124eb640 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e6be0 0x5567856e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.935+0000 7fcd12cec640 1 -- 172.21.15.17:0/513579882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e6be0 msgr2=0x5567856e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.935+0000 7fcd12cec640 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e6be0 0x5567856e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.935+0000 7fcd12cec640 1 -- 172.21.15.17:0/513579882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 msgr2=0x5567856e0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.935+0000 7fcd12cec640 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.935+0000 7fcd12cec640 1 -- 172.21.15.17:0/513579882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567855e0820 con 0x5567856f8ea0 2024-02-14T15:14:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.937+0000 7fcd12cec640 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856f8ea0 0x5567856ffbc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcd04000f80 tx=0x7fcd0400f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9406558c0aa683 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.937+0000 7fccf7fff640 1 -- 172.21.15.17:0/513579882 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0402c040 con 0x5567856f8ea0 2024-02-14T15:14:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.937+0000 7fccf7fff640 1 -- 172.21.15.17:0/513579882 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd04003e50 con 0x5567856f8ea0 2024-02-14T15:14:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.937+0000 7fccf7fff640 1 -- 172.21.15.17:0/513579882 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040276c0 con 0x5567856f8ea0 2024-02-14T15:14:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.939+0000 7fcd173fb740 1 -- 172.21.15.17:0/513579882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856f8ea0 msgr2=0x5567856ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.939+0000 7fcd173fb740 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856f8ea0 0x5567856ffbc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcd04000f80 tx=0x7fcd0400f3c0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.942+0000 7fcd173fb740 1 -- 172.21.15.17:0/513579882 shutdown_connections 2024-02-14T15:14:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.942+0000 7fcd173fb740 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e7500 0x5567856e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.942+0000 7fcd173fb740 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856f8ea0 0x5567856ffbc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.942+0000 7fcd173fb740 1 --2- 172.21.15.17:0/513579882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856e6be0 0x5567856e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.942+0000 7fcd173fb740 1 -- 172.21.15.17:0/513579882 >> 172.21.15.17:0/513579882 conn(0x5567856e1660 msgr2=0x5567856dd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.943+0000 7fcd173fb740 1 -- 172.21.15.17:0/513579882 shutdown_connections 2024-02-14T15:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.944+0000 7fcd173fb740 1 -- 172.21.15.17:0/513579882 wait complete. 2024-02-14T15:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.944+0000 7fcd173fb740 1 -- start start 2024-02-14T15:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.944+0000 7fcd173fb740 1 Processor -- start 2024-02-14T15:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.945+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e6be0 0x5567856fff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.945+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e7500 0x5567856fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.945+0000 7fcd173fb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.945+0000 7fcd173fb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567856feff0 con 0x5567856e6be0 2024-02-14T15:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.945+0000 7fcd173fb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567856ff160 con 0x5567856f8ea0 2024-02-14T15:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.945+0000 7fcd173fb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567856ff2d0 con 0x5567856e7500 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.945+0000 7fcd173fb740 10 client.0 _ll_register_callbacks cb 0x7fcd04009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e7500 0x5567856fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e7500 0x5567856fe400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:33324/0 (socket says 172.21.15.17:33324) 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 -- 172.21.15.17:0/3030205184 learned_addr learned my addr 172.21.15.17:0/3030205184 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd124eb640 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e6be0 0x5567856fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 -- 172.21.15.17:0/3030205184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 msgr2=0x5567856fe940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fe940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 -- 172.21.15.17:0/3030205184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e6be0 msgr2=0x5567856fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e6be0 0x5567856fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.946+0000 7fcd11cea640 1 -- 172.21.15.17:0/3030205184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556785643d70 con 0x5567856e7500 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.947+0000 7fcd11cea640 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e7500 0x5567856fe400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccfc09ac70 tx=0x7fccfc09a700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc9802358d3b2a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.947+0000 7fccf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.948+0000 7fccf5ffb640 1 -- 172.21.15.17:0/3030205184 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0b9270 con 0x5567856e7500 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.948+0000 7fccf5ffb640 1 -- 172.21.15.17:0/3030205184 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9563+0+0 (secure 0 0 0) 0x7fccfc0cd040 con 0x5567856e7500 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.948+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd04022020 con 0x5567856e7500 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.948+0000 7fccf5ffb640 1 -- 172.21.15.17:0/3030205184 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccfc0cbed0 con 0x5567856e7500 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.948+0000 7fcd173fb740 10 client.26692 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.948+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556785606720 con 0x5567856e7500 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fccf5ffb640 1 -- 172.21.15.17:0/3030205184 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd04022020 con 0x5567856e7500 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fccf5ffb640 1 -- 172.21.15.17:0/3030205184 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccfc01f060 con 0x5567856e7500 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fcd173fb740 10 client.26692 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fcd173fb740 10 client.26692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fcd173fb740 20 client.26692 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5567857718b0 0x556785773cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fcd173fb740 4 client.26692 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.949+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556785382550 con 0x5567857718b0 2024-02-14T15:14:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.950+0000 7fcd124eb640 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5567857718b0 0x556785773cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.951+0000 7fcd124eb640 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5567857718b0 0x556785773cd0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c015d438b1f40dc server_cookie=d7aee30c56c074d7 in_seq=0 out_seq=0 2024-02-14T15:14:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.951+0000 7fccf5ffb640 10 client.26692 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fccf5ffb640 1 -- 172.21.15.17:0/3030205184 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556785382550 con 0x5567857718b0 2024-02-14T15:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fccf5ffb640 10 client.26692 handle_command_reply: tid=0 2024-02-14T15:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 client.26692 shutdown 2024-02-14T15:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e7500 msgr2=0x5567856fe400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e7500 0x5567856fe400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccfc09ac70 tx=0x7fccfc09a700 comp rx=0 tx=0).stop 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 shutdown_connections 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5567857718b0 0x556785773cd0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567856f8ea0 0x5567856fe940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567856e7500 0x5567856fe400 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fccf5ffb640 0 client.26692 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 --2- 172.21.15.17:0/3030205184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5567856e6be0 0x5567856fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.964+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 >> 172.21.15.17:0/3030205184 conn(0x5567856e1660 msgr2=0x5567856dc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 shutdown_connections 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd173fb740 1 -- 172.21.15.17:0/3030205184 wait complete. 2024-02-14T15:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd173fb740 20 client.26692 trim_cache size 0 max 16384 2024-02-14T15:14:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:14:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:14:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:14:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:14:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccf80a23f0 msgr2=0x7fccf80a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccf80a23f0 0x7fccf80a48b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd04023000 tx=0x7fcd04020030 comp rx=0 tx=0).stop 2024-02-14T15:14:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c106520 msgr2=0x7fcd0c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c106520 0x7fcd0c19db60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e9c0 tx=0x7fccfc00ee80 comp rx=0 tx=0).stop 2024-02-14T15:14:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 shutdown_connections 2024-02-14T15:14:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccf80a23f0 0x7fccf80a48b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd0c106520 0x7fcd0c19db60 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd0c197790 0x7fcd0c197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 --2- 172.21.15.17:0/1670150832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd0c101450 0x7fcd0c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 >> 172.21.15.17:0/1670150832 conn(0x7fcd0c0fccf0 msgr2=0x7fcd0c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 shutdown_connections 2024-02-14T15:14:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:08.965+0000 7fcd1474f640 1 -- 172.21.15.17:0/1670150832 wait complete. 2024-02-14T15:14:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:08 smithi050 ceph-mon[36512]: pgmap v335: 129 pgs: 129 active+clean; 628 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 284 op/s 2024-02-14T15:14:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:08 smithi076 ceph-mon[37057]: pgmap v335: 129 pgs: 129 active+clean; 628 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 284 op/s 2024-02-14T15:14:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:08 smithi017 ceph-mon[32549]: pgmap v335: 129 pgs: 129 active+clean; 628 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.0 MiB/s wr, 284 op/s 2024-02-14T15:14:09.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1114804 10.73 MB/sec execute 431 sec latency 33.024 ms 2024-02-14T15:14:10.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:09 smithi050 ceph-mon[36512]: osdmap e106: 12 total, 12 up, 12 in 2024-02-14T15:14:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:09 smithi076 ceph-mon[37057]: osdmap e106: 12 total, 12 up, 12 in 2024-02-14T15:14:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:09 smithi017 ceph-mon[32549]: osdmap e106: 12 total, 12 up, 12 in 2024-02-14T15:14:10.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1116015 10.72 MB/sec execute 432 sec latency 55.322 ms 2024-02-14T15:14:10.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:10 smithi076 ceph-mon[37057]: pgmap v337: 129 pgs: 129 active+clean; 628 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 232 op/s 2024-02-14T15:14:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:10 smithi050 ceph-mon[36512]: pgmap v337: 129 pgs: 129 active+clean; 628 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 232 op/s 2024-02-14T15:14:11.167 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:10 smithi017 ceph-mon[32549]: pgmap v337: 129 pgs: 129 active+clean; 628 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 232 op/s 2024-02-14T15:14:11.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1118302 10.73 MB/sec execute 433 sec latency 45.396 ms 2024-02-14T15:14:12.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1119459 10.72 MB/sec execute 434 sec latency 41.315 ms 2024-02-14T15:14:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:12 smithi050 ceph-mon[36512]: pgmap v338: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:12 smithi076 ceph-mon[37057]: pgmap v338: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:12 smithi017 ceph-mon[32549]: pgmap v338: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:13.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1121658 10.72 MB/sec execute 435 sec latency 24.438 ms 2024-02-14T15:14:14.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1123011 10.71 MB/sec execute 436 sec latency 26.093 ms 2024-02-14T15:14:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:14 smithi050 ceph-mon[36512]: pgmap v339: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:14 smithi076 ceph-mon[37057]: pgmap v339: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:14 smithi017 ceph-mon[32549]: pgmap v339: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:15.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1125182 10.71 MB/sec execute 437 sec latency 24.932 ms 2024-02-14T15:14:16.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1126560 10.71 MB/sec execute 438 sec latency 19.694 ms 2024-02-14T15:14:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:16 smithi017 ceph-mon[32549]: pgmap v340: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:16 smithi017 ceph-mon[32549]: osdmap e107: 12 total, 12 up, 12 in 2024-02-14T15:14:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:16 smithi050 ceph-mon[36512]: pgmap v340: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:16 smithi050 ceph-mon[36512]: osdmap e107: 12 total, 12 up, 12 in 2024-02-14T15:14:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:16 smithi076 ceph-mon[37057]: pgmap v340: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 211 op/s 2024-02-14T15:14:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:16 smithi076 ceph-mon[37057]: osdmap e107: 12 total, 12 up, 12 in 2024-02-14T15:14:17.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1127658 10.69 MB/sec execute 439 sec latency 220.200 ms 2024-02-14T15:14:18.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1129723 10.70 MB/sec execute 440 sec latency 19.130 ms 2024-02-14T15:14:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:18 smithi017 ceph-mon[32549]: pgmap v342: 129 pgs: 129 active+clean; 572 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-02-14T15:14:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:18 smithi050 ceph-mon[36512]: pgmap v342: 129 pgs: 129 active+clean; 572 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-02-14T15:14:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:18 smithi076 ceph-mon[37057]: pgmap v342: 129 pgs: 129 active+clean; 572 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-02-14T15:14:19.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1131212 10.69 MB/sec execute 441 sec latency 17.030 ms 2024-02-14T15:14:20.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1133379 10.69 MB/sec execute 442 sec latency 16.718 ms 2024-02-14T15:14:21.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:20 smithi017 ceph-mon[32549]: pgmap v343: 129 pgs: 129 active+clean; 572 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-02-14T15:14:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:20 smithi050 ceph-mon[36512]: pgmap v343: 129 pgs: 129 active+clean; 572 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-02-14T15:14:21.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:20 smithi076 ceph-mon[37057]: pgmap v343: 129 pgs: 129 active+clean; 572 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-02-14T15:14:21.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1135251 10.68 MB/sec execute 443 sec latency 24.074 ms 2024-02-14T15:14:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.080+0000 7f164419a640 1 -- 172.21.15.17:0/567272247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c10cf60 msgr2=0x7f163c114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.080+0000 7f164419a640 1 --2- 172.21.15.17:0/567272247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c10cf60 0x7f163c114b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1634005fd0 tx=0x7f163402cba0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.080+0000 7f164419a640 1 -- 172.21.15.17:0/567272247 shutdown_connections 2024-02-14T15:14:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.080+0000 7f164419a640 1 --2- 172.21.15.17:0/567272247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c108e50 0x7f163c10c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.080+0000 7f164419a640 1 --2- 172.21.15.17:0/567272247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c10cf60 0x7f163c114b30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.080+0000 7f164419a640 1 --2- 172.21.15.17:0/567272247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f163c1084a0 0x7f163c108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.080+0000 7f164419a640 1 -- 172.21.15.17:0/567272247 >> 172.21.15.17:0/567272247 conn(0x7f163c06c2e0 msgr2=0x7f163c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.083+0000 7f164419a640 1 -- 172.21.15.17:0/567272247 shutdown_connections 2024-02-14T15:14:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.084+0000 7f164419a640 1 -- 172.21.15.17:0/567272247 wait complete. 2024-02-14T15:14:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.085+0000 7f164419a640 1 Processor -- start 2024-02-14T15:14:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.085+0000 7f164419a640 1 -- start start 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f164419a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1084a0 0x7f163c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f164419a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f163c10cf60 0x7f163c1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f164419a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c1a46f0 0x7f163c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f164419a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a50e0 con 0x7f163c1084a0 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f164419a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a5250 con 0x7f163c10cf60 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f164419a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a53c0 con 0x7f163c1a46f0 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f1642737640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c1a46f0 0x7f163c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f1641f36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1084a0 0x7f163c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f1642737640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c1a46f0 0x7f163c1a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:41848/0 (socket says 172.21.15.17:41848) 2024-02-14T15:14:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.086+0000 7f1642737640 1 -- 172.21.15.17:0/2504583493 learned_addr learned my addr 172.21.15.17:0/2504583493 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.087+0000 7f1641735640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f163c10cf60 0x7f163c1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.087+0000 7f1641735640 1 -- 172.21.15.17:0/2504583493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c1a46f0 msgr2=0x7f163c1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.087+0000 7f1641735640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c1a46f0 0x7f163c1a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.087+0000 7f1641735640 1 -- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1084a0 msgr2=0x7f163c19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.087+0000 7f1641735640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1084a0 0x7f163c19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.087+0000 7f1641735640 1 -- 172.21.15.17:0/2504583493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340073b0 con 0x7f163c10cf60 2024-02-14T15:14:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.088+0000 7f1642737640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c1a46f0 0x7f163c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:14:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.090+0000 7f1641735640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f163c10cf60 0x7f163c1a0370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f163800d6a0 tx=0x7f163800db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.090+0000 7f1632ffd640 1 -- 172.21.15.17:0/2504583493 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638009a20 con 0x7f163c10cf60 2024-02-14T15:14:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.091+0000 7f1632ffd640 1 -- 172.21.15.17:0/2504583493 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1638009b80 con 0x7f163c10cf60 2024-02-14T15:14:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.091+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c1ac9a0 con 0x7f163c10cf60 2024-02-14T15:14:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.091+0000 7f1632ffd640 1 -- 172.21.15.17:0/2504583493 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800f620 con 0x7f163c10cf60 2024-02-14T15:14:22.092 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.091+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c1ace50 con 0x7f163c10cf60 2024-02-14T15:14:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.095+0000 7f1632ffd640 1 -- 172.21.15.17:0/2504583493 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f163800f780 con 0x7f163c10cf60 2024-02-14T15:14:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 Processor -- start 2024-02-14T15:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 -- start start 2024-02-14T15:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b3046c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 0x557b3046cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b30480740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b303da9e0 con 0x557b3047e350 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b303d7f10 con 0x557b3046c190 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.097+0000 7f1646d6d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b303d36f0 con 0x557b3046cb40 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b3046c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b3046c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:48660/0 (socket says 172.21.15.17:48660) 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 -- 172.21.15.17:0/1228854634 learned_addr learned my addr 172.21.15.17:0/1228854634 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1641735640 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b30480740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 -- 172.21.15.17:0/1228854634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 msgr2=0x557b3046cfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 0x557b3046cfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 -- 172.21.15.17:0/1228854634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 msgr2=0x557b30480740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b30480740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.098+0000 7f1642737640 1 -- 172.21.15.17:0/1228854634 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b300e4920 con 0x557b3046c190 2024-02-14T15:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.100+0000 7f1632ffd640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16280a22f0 0x7f16280a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.100+0000 7f1632ffd640 1 -- 172.21.15.17:0/2504583493 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f16380cf150 con 0x7f163c10cf60 2024-02-14T15:14:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.101+0000 7f1642737640 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b3046c570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1634005c40 tx=0x7f1634007b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c462983f387e0fe server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.101+0000 7f1641f36640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16280a22f0 0x7f16280a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.101+0000 7f1641f36640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16280a22f0 0x7f16280a47b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f162c006fd0 tx=0x7f162c008040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.103+0000 7f1627fff640 1 -- 172.21.15.17:0/1228854634 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400a890 con 0x557b3046c190 2024-02-14T15:14:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.103+0000 7f1627fff640 1 -- 172.21.15.17:0/1228854634 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f163400a9f0 con 0x557b3046c190 2024-02-14T15:14:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.104+0000 7f1646d6d740 1 -- 172.21.15.17:0/1228854634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 msgr2=0x557b3046c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.104+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b3046c570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1634005c40 tx=0x7f1634007b90 comp rx=0 tx=0).stop 2024-02-14T15:14:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.106+0000 7f1646d6d740 1 -- 172.21.15.17:0/1228854634 shutdown_connections 2024-02-14T15:14:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.106+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b30480740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.106+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 0x557b3046cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.106+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1228854634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b3046c570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.106+0000 7f1646d6d740 1 -- 172.21.15.17:0/1228854634 >> 172.21.15.17:0/1228854634 conn(0x557b30466e50 msgr2=0x557b30467230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.108+0000 7f1646d6d740 1 -- 172.21.15.17:0/1228854634 shutdown_connections 2024-02-14T15:14:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.108+0000 7f1646d6d740 1 -- 172.21.15.17:0/1228854634 wait complete. 2024-02-14T15:14:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 -- start start 2024-02-14T15:14:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 Processor -- start 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046c190 0x557b30477720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b30477c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3047e350 0x557b3047d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b3036ffa0 con 0x557b3046cb40 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b300a7250 con 0x557b3047e350 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.109+0000 7f1646d6d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30359200 con 0x557b3046c190 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1646d6d740 10 client.0 _ll_register_callbacks cb 0x557b303c8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b30477c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b30477c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44066/0 (socket says 172.21.15.17:44066) 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 -- 172.21.15.17:0/3160640325 learned_addr learned my addr 172.21.15.17:0/3160640325 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1642737640 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3047e350 0x557b3047d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 -- 172.21.15.17:0/3160640325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046c190 msgr2=0x557b30477720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-14T15:14:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046c190 0x557b30477720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 -- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3047e350 msgr2=0x557b3047d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3047e350 0x557b3047d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.110+0000 7f1641735640 1 -- 172.21.15.17:0/3160640325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c115340 con 0x557b3046cb40 2024-02-14T15:14:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.111+0000 7f1641735640 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b30477c60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1638097b80 tx=0x7f1638014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85507f633a76cfe8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.111+0000 7f16257fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:14:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.112+0000 7f16257fa640 1 -- 172.21.15.17:0/3160640325 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380cc900 con 0x557b3046cb40 2024-02-14T15:14:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.112+0000 7f16257fa640 1 -- 172.21.15.17:0/3160640325 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f163809be20 con 0x557b3046cb40 2024-02-14T15:14:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.112+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b300d3600 con 0x557b3046cb40 2024-02-14T15:14:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.113+0000 7f16257fa640 1 -- 172.21.15.17:0/3160640325 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f163809b080 con 0x557b3046cb40 2024-02-14T15:14:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.113+0000 7f1646d6d740 10 client.17187 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.113+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557b303d1d70 con 0x557b3046cb40 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f16257fa640 1 -- 172.21.15.17:0/3160640325 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557b300d3600 con 0x557b3046cb40 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f16257fa640 1 -- 172.21.15.17:0/3160640325 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f16380ccbd0 con 0x557b3046cb40 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f1646d6d740 10 client.17187 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f1646d6d740 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f1646d6d740 20 client.17187 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f1646d6d740 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304968a0 0x557b30498c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f1646d6d740 4 client.17187 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:14:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.114+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b301241b0 con 0x557b304968a0 2024-02-14T15:14:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.115+0000 7f1641f36640 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304968a0 0x557b30498c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.116+0000 7f16257fa640 10 client.17187 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.116+0000 7f1641f36640 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304968a0 0x557b30498c90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4970bc9febaf61 server_cookie=d8496566941e3179 in_seq=0 out_seq=0 2024-02-14T15:14:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.116+0000 7f16257fa640 1 -- 172.21.15.17:0/3160640325 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b301241b0 con 0x557b304968a0 2024-02-14T15:14:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.116+0000 7f16257fa640 10 client.17187 handle_command_reply: tid=0 2024-02-14T15:14:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 client.17187 shutdown 2024-02-14T15:14:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 msgr2=0x557b30477c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b30477c60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1638097b80 tx=0x7f1638014000 comp rx=0 tx=0).stop 2024-02-14T15:14:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 shutdown_connections 2024-02-14T15:14:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304968a0 0x557b30498c90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b30477c60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3047e350 0x557b3047d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 --2- 172.21.15.17:0/3160640325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046c190 0x557b30477720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 >> 172.21.15.17:0/3160640325 conn(0x557b30466e50 msgr2=0x557b30468ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f16257fa640 0 client.17187 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 shutdown_connections 2024-02-14T15:14:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 1 -- 172.21.15.17:0/3160640325 wait complete. 2024-02-14T15:14:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.117+0000 7f1646d6d740 20 client.17187 trim_cache size 0 max 16384 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 Processor -- start 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 -- start start 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b3047d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3047e350 0x557b3046f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b300d3600 con 0x557b3046cb40 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b304620b0 con 0x557b3046c190 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1646d6d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b30467ed0 con 0x557b3047e350 2024-02-14T15:14:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1641735640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b3047d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1641735640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b3047d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44068/0 (socket says 172.21.15.17:44068) 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1641735640 1 -- 172.21.15.17:0/1771678914 learned_addr learned my addr 172.21.15.17:0/1771678914 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1641f36640 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.132+0000 7f1642737640 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3047e350 0x557b3046f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f1641735640 1 -- 172.21.15.17:0/1771678914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3047e350 msgr2=0x557b3046f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f1641735640 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3047e350 0x557b3046f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f1641735640 1 -- 172.21.15.17:0/1771678914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 msgr2=0x557b304711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f1641735640 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f1641735640 1 -- 172.21.15.17:0/1771678914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b3036e820 con 0x557b3046cb40 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f1641735640 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b3047d3c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f163809b6b0 tx=0x7f1638010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8a2435dc777d14 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f16277fe640 1 -- 172.21.15.17:0/1771678914 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163809b080 con 0x557b3046cb40 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f16277fe640 1 -- 172.21.15.17:0/1771678914 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16380b9e90 con 0x557b3046cb40 2024-02-14T15:14:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.133+0000 7f16277fe640 1 -- 172.21.15.17:0/1771678914 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380ccbd0 con 0x557b3046cb40 2024-02-14T15:14:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 -- 172.21.15.17:0/1771678914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 msgr2=0x557b3047d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b3047d3c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f163809b6b0 tx=0x7f1638010c70 comp rx=0 tx=0).stop 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 -- 172.21.15.17:0/1771678914 shutdown_connections 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3046cb40 0x557b3047d3c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3047e350 0x557b3046f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 --2- 172.21.15.17:0/1771678914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 -- 172.21.15.17:0/1771678914 >> 172.21.15.17:0/1771678914 conn(0x557b30466e50 msgr2=0x557b3046d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 -- 172.21.15.17:0/1771678914 shutdown_connections 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 -- 172.21.15.17:0/1771678914 wait complete. 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 -- start start 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.134+0000 7f1646d6d740 1 Processor -- start 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304e1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 0x557b304e1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1646d6d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b304e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1646d6d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b304e6610 con 0x557b3047e350 2024-02-14T15:14:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1646d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b304e6780 con 0x557b3046c190 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1646d6d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b304e6940 con 0x557b3046cb40 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b304e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1646d6d740 10 client.0 _ll_register_callbacks cb 0x7f16380979c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b304e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44072/0 (socket says 172.21.15.17:44072) 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 -- 172.21.15.17:0/655061407 learned_addr learned my addr 172.21.15.17:0/655061407 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1641735640 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 0x557b304e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1641f36640 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 -- 172.21.15.17:0/655061407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 msgr2=0x557b304e1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 0x557b304e1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 -- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 msgr2=0x557b304e1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304e1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 -- 172.21.15.17:0/655061407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b303d1d70 con 0x557b3047e350 2024-02-14T15:14:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f1642737640 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b304e6050 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f163400af00 tx=0x7f1634003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158d66d2967dc81b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f16257fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.135+0000 7f16257fa640 1 -- 172.21.15.17:0/655061407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634003c40 con 0x557b3047e350 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f16257fa640 1 -- 172.21.15.17:0/655061407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f16340045d0 con 0x557b3047e350 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f163400a9f0 con 0x557b3047e350 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f16257fa640 1 -- 172.21.15.17:0/655061407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1634004790 con 0x557b3047e350 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 10 client.17193 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557b30394720 con 0x557b3047e350 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f16257fa640 1 -- 172.21.15.17:0/655061407 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f163405a080 con 0x557b3047e350 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f16257fa640 1 -- 172.21.15.17:0/655061407 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1634045040 con 0x557b3047e350 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 10 client.17193 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 20 client.17193 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304edf30 0x557b304f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 4 client.17193 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:14:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.136+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b30110550 con 0x557b304edf30 2024-02-14T15:14:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.137+0000 7f1641f36640 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304edf30 0x557b304f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.137+0000 7f1641f36640 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304edf30 0x557b304f0350 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bee0bdf42e4e689 server_cookie=99a11b2cd97c4b23 in_seq=0 out_seq=0 2024-02-14T15:14:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.137+0000 7f16257fa640 10 client.17193 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.148+0000 7f16257fa640 1 -- 172.21.15.17:0/655061407 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b30110550 con 0x557b304edf30 2024-02-14T15:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.148+0000 7f16257fa640 10 client.17193 handle_command_reply: tid=0 2024-02-14T15:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.148+0000 7f1646d6d740 1 client.17193 shutdown 2024-02-14T15:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 msgr2=0x557b304e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b304e6050 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f163400af00 tx=0x7f1634003e40 comp rx=0 tx=0).stop 2024-02-14T15:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 shutdown_connections 2024-02-14T15:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x557b304edf30 0x557b304f0350 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f16257fa640 0 client.17193 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557b3047e350 0x557b304e6050 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b3046cb40 0x557b304e1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 --2- 172.21.15.17:0/655061407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b3046c190 0x557b304e1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 >> 172.21.15.17:0/655061407 conn(0x557b30466e50 msgr2=0x557b30465630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 shutdown_connections 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 1 -- 172.21.15.17:0/655061407 wait complete. 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.149+0000 7f1646d6d740 20 client.17193 trim_cache size 0 max 16384 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:14:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:14:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16280a22f0 msgr2=0x7f16280a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16280a22f0 0x7f16280a47b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f162c006fd0 tx=0x7f162c008040 comp rx=0 tx=0).stop 2024-02-14T15:14:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f163c10cf60 msgr2=0x7f163c1a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f163c10cf60 0x7f163c1a0370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f163800d6a0 tx=0x7f163800db60 comp rx=0 tx=0).stop 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 shutdown_connections 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16280a22f0 0x7f16280a47b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c1a46f0 0x7f163c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f163c10cf60 0x7f163c1a0370 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 --2- 172.21.15.17:0/2504583493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f163c1084a0 0x7f163c19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 >> 172.21.15.17:0/2504583493 conn(0x7f163c06c2e0 msgr2=0x7f163c10b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 shutdown_connections 2024-02-14T15:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:22.150+0000 7f164419a640 1 -- 172.21.15.17:0/2504583493 wait complete. 2024-02-14T15:14:22.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1136727 10.69 MB/sec execute 444 sec latency 75.841 ms 2024-02-14T15:14:23.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:22 smithi017 ceph-mon[32549]: pgmap v344: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:22 smithi050 ceph-mon[36512]: pgmap v344: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:23.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:22 smithi076 ceph-mon[37057]: pgmap v344: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:23.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1138088 10.68 MB/sec execute 445 sec latency 19.627 ms 2024-02-14T15:14:24.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1140317 10.68 MB/sec execute 446 sec latency 14.803 ms 2024-02-14T15:14:25.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:24 smithi017 ceph-mon[32549]: pgmap v345: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:24 smithi050 ceph-mon[36512]: pgmap v345: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:24 smithi076 ceph-mon[37057]: pgmap v345: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:25.886 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1141850 10.67 MB/sec execute 447 sec latency 18.325 ms 2024-02-14T15:14:26.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1143884 10.68 MB/sec execute 448 sec latency 32.426 ms 2024-02-14T15:14:27.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:26 smithi017 ceph-mon[32549]: pgmap v346: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:27.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:26 smithi050 ceph-mon[36512]: pgmap v346: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:26 smithi076 ceph-mon[37057]: pgmap v346: 129 pgs: 129 active+clean; 582 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 199 op/s 2024-02-14T15:14:27.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1144815 10.66 MB/sec execute 449 sec latency 64.181 ms 2024-02-14T15:14:28.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1147317 10.67 MB/sec execute 450 sec latency 17.288 ms 2024-02-14T15:14:29.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:28 smithi017 ceph-mon[32549]: pgmap v347: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 270 op/s 2024-02-14T15:14:29.035 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:28 smithi076 ceph-mon[37057]: pgmap v347: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 270 op/s 2024-02-14T15:14:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:28 smithi050 ceph-mon[36512]: pgmap v347: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 270 op/s 2024-02-14T15:14:29.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1148677 10.66 MB/sec execute 451 sec latency 15.900 ms 2024-02-14T15:14:30.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1150920 10.67 MB/sec execute 452 sec latency 15.659 ms 2024-02-14T15:14:31.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:30 smithi017 ceph-mon[32549]: pgmap v348: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 171 op/s 2024-02-14T15:14:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:30 smithi050 ceph-mon[36512]: pgmap v348: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 171 op/s 2024-02-14T15:14:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:30 smithi076 ceph-mon[37057]: pgmap v348: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 171 op/s 2024-02-14T15:14:31.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1152288 10.66 MB/sec execute 453 sec latency 28.344 ms 2024-02-14T15:14:32.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1154454 10.66 MB/sec execute 454 sec latency 47.711 ms 2024-02-14T15:14:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:32 smithi050 ceph-mon[36512]: pgmap v349: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-02-14T15:14:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:32 smithi076 ceph-mon[37057]: pgmap v349: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-02-14T15:14:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:32 smithi017 ceph-mon[32549]: pgmap v349: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-02-14T15:14:33.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1155769 10.65 MB/sec execute 455 sec latency 28.080 ms 2024-02-14T15:14:34.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1158022 10.66 MB/sec execute 456 sec latency 15.454 ms 2024-02-14T15:14:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:34 smithi050 ceph-mon[36512]: pgmap v350: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 172 op/s 2024-02-14T15:14:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:34 smithi076 ceph-mon[37057]: pgmap v350: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 172 op/s 2024-02-14T15:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- 172.21.15.17:0/2146948279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084068d40 msgr2=0x7f1084069120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 --2- 172.21.15.17:0/2146948279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084068d40 0x7f1084069120 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1074009930 tx=0x7f107402bc30 comp rx=0 tx=0).stop 2024-02-14T15:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- 172.21.15.17:0/2146948279 shutdown_connections 2024-02-14T15:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 --2- 172.21.15.17:0/2146948279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1084106510 0x7f108410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 --2- 172.21.15.17:0/2146948279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10840696f0 0x7f1084105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 --2- 172.21.15.17:0/2146948279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084068d40 0x7f1084069120 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- 172.21.15.17:0/2146948279 >> 172.21.15.17:0/2146948279 conn(0x7f1084077220 msgr2=0x7f1084077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- 172.21.15.17:0/2146948279 shutdown_connections 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- 172.21.15.17:0/2146948279 wait complete. 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 Processor -- start 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- start start 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1084068d40 0x7f10841000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1084106510 0x7f1084100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084100b70 0x7f10841a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841a60c0 con 0x7f1084100b70 2024-02-14T15:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841a6230 con 0x7f1084068d40 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.239+0000 7f1089fbc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841a63f0 con 0x7f1084106510 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084100b70 0x7f10841a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f10837fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1084068d40 0x7f10841000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084100b70 0x7f10841a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34684/0 (socket says 172.21.15.17:34684) 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 -- 172.21.15.17:0/3818673783 learned_addr learned my addr 172.21.15.17:0/3818673783 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f107bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1084106510 0x7f1084100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 -- 172.21.15.17:0/3818673783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1084106510 msgr2=0x7f1084100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1084106510 0x7f1084100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 -- 172.21.15.17:0/3818673783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1084068d40 msgr2=0x7f10841000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1084068d40 0x7f10841000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 -- 172.21.15.17:0/3818673783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074009560 con 0x7f1084100b70 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f10837fe640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1084068d40 0x7f10841000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1083fff640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084100b70 0x7f10841a5af0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f107c00eb10 tx=0x7f107c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f10817fa640 1 -- 172.21.15.17:0/3818673783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c004b20 con 0x7f1084100b70 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f10817fa640 1 -- 172.21.15.17:0/3818673783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f107c004c80 con 0x7f1084100b70 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10841a66d0 con 0x7f1084100b70 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f10817fa640 1 -- 172.21.15.17:0/3818673783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c010650 con 0x7f1084100b70 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.240+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10841a6be0 con 0x7f1084100b70 2024-02-14T15:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.242+0000 7f10817fa640 1 -- 172.21.15.17:0/3818673783 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f107c005020 con 0x7f1084100b70 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 Processor -- start 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 -- start start 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10817fa640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f10700a2250 0x7f10700a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f107bfff640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f10700a2250 0x7f10700a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0011b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d001a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10817fa640 1 -- 172.21.15.17:0/3818673783 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f107c014070 con 0x7f1084100b70 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627cff7f9e0 con 0x5627d0012040 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627cff7cf10 con 0x5627d001ac20 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f108cb82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627cff786f0 con 0x5627d0011720 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0011b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f107bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d001a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0011b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:60942/0 (socket says 172.21.15.17:60942) 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 -- 172.21.15.17:0/1156815723 learned_addr learned my addr 172.21.15.17:0/1156815723 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f1083fff640 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 -- 172.21.15.17:0/1156815723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 msgr2=0x5627d001d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 -- 172.21.15.17:0/1156815723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 msgr2=0x5627d001a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d001a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.243+0000 7f10837fe640 1 -- 172.21.15.17:0/1156815723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627cfc89920 con 0x5627d0011720 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.244+0000 7f107bfff640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f10700a2250 0x7f10700a4710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1084101b80 tx=0x7f106c013830 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.244+0000 7f10837fe640 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0011b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1074036040 tx=0x7f107402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f64299c94464a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.244+0000 7f1079ffb640 1 -- 172.21.15.17:0/1156815723 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074004af0 con 0x5627d0011720 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.244+0000 7f1079ffb640 1 -- 172.21.15.17:0/1156815723 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1074004030 con 0x5627d0011720 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.244+0000 7f1079ffb640 1 -- 172.21.15.17:0/1156815723 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740053e0 con 0x5627d0011720 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.244+0000 7f108cb82740 1 -- 172.21.15.17:0/1156815723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 msgr2=0x5627d0011b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.244+0000 7f108cb82740 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0011b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1074036040 tx=0x7f107402c660 comp rx=0 tx=0).stop 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- 172.21.15.17:0/1156815723 shutdown_connections 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d001a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 --2- 172.21.15.17:0/1156815723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0011b00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- 172.21.15.17:0/1156815723 >> 172.21.15.17:0/1156815723 conn(0x5627d000be50 msgr2=0x5627d000c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- 172.21.15.17:0/1156815723 shutdown_connections 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- 172.21.15.17:0/1156815723 wait complete. 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- start start 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 Processor -- start 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d001a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d0017960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d0017f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627cff14fa0 con 0x5627d0012040 2024-02-14T15:14:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627cfc4c250 con 0x5627d001ac20 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627cfefe200 con 0x5627d0011720 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f108cb82740 10 client.0 _ll_register_callbacks cb 0x5627cff6db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f107bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d0017960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f107bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d0017960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34698/0 (socket says 172.21.15.17:34698) 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.245+0000 7f107bfff640 1 -- 172.21.15.17:0/1211316890 learned_addr learned my addr 172.21.15.17:0/1211316890 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f1083fff640 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d0017f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f10837fe640 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d001a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f107bfff640 1 -- 172.21.15.17:0/1211316890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 msgr2=0x5627d001a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f107bfff640 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d001a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f107bfff640 1 -- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 msgr2=0x5627d0017f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f107bfff640 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d0017f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f107bfff640 1 -- 172.21.15.17:0/1211316890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108410e8f0 con 0x5627d0012040 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f107bfff640 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d0017960 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f106c003820 tx=0x7f106c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadd86d6400c3ee8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f1062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f1062ffd640 1 -- 172.21.15.17:0/1211316890 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c018040 con 0x5627d0012040 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627cfc78600 con 0x5627d0012040 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f1062ffd640 1 -- 172.21.15.17:0/1211316890 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f106c012270 con 0x5627d0012040 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f1062ffd640 1 -- 172.21.15.17:0/1211316890 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f106c00fb20 con 0x5627d0012040 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f108cb82740 10 client.17205 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5627cff76d70 con 0x5627d0012040 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.246+0000 7f1062ffd640 1 -- 172.21.15.17:0/1211316890 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5627cfc78600 con 0x5627d0012040 2024-02-14T15:14:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f1062ffd640 1 -- 172.21.15.17:0/1211316890 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f106c026070 con 0x5627d0012040 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f108cb82740 10 client.17205 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f108cb82740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f108cb82740 20 client.17205 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f108cb82740 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d002a980 0x5627d002cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f108cb82740 4 client.17205 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627cfcc91b0 con 0x5627d002a980 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f10837fe640 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d002a980 0x5627d002cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f10837fe640 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d002a980 0x5627d002cda0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71989ef1ef77505 server_cookie=2f3090edfe346280 in_seq=0 out_seq=0 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.247+0000 7f1062ffd640 10 client.17205 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f1062ffd640 1 -- 172.21.15.17:0/1211316890 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627cfcc91b0 con 0x5627d002a980 2024-02-14T15:14:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f1062ffd640 10 client.17205 handle_command_reply: tid=0 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 client.17205 shutdown 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 msgr2=0x5627d0017960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d0017960 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f106c003820 tx=0x7f106c004730 comp rx=0 tx=0).stop 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 shutdown_connections 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d002a980 0x5627d002cda0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0012040 0x5627d0017960 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d0017f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f1062ffd640 0 client.17205 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 --2- 172.21.15.17:0/1211316890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d001a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 >> 172.21.15.17:0/1211316890 conn(0x5627d000be50 msgr2=0x5627d000dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 shutdown_connections 2024-02-14T15:14:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 1 -- 172.21.15.17:0/1211316890 wait complete. 2024-02-14T15:14:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.248+0000 7f108cb82740 20 client.17205 trim_cache size 0 max 16384 2024-02-14T15:14:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 Processor -- start 2024-02-14T15:14:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 -- start start 2024-02-14T15:14:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0011720 0x5627d0017900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0012040 0x5627d001a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d0027450 con 0x5627d0011720 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d00109e0 con 0x5627d001ac20 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f108cb82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d0013570 con 0x5627d0012040 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0011720 0x5627d0017900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0011720 0x5627d0017900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34704/0 (socket says 172.21.15.17:34704) 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 -- 172.21.15.17:0/2555562201 learned_addr learned my addr 172.21.15.17:0/2555562201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f107bfff640 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0012040 0x5627d001a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f1083fff640 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 -- 172.21.15.17:0/2555562201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0012040 msgr2=0x5627d001a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0012040 0x5627d001a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 -- 172.21.15.17:0/2555562201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 msgr2=0x5627d001a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 -- 172.21.15.17:0/2555562201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627cff13820 con 0x5627d0011720 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.262+0000 7f10837fe640 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0011720 0x5627d0017900 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1074005050 tx=0x7f1074041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a80fdb84b2034ad server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f10797fa640 1 -- 172.21.15.17:0/2555562201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074054070 con 0x5627d0011720 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f10797fa640 1 -- 172.21.15.17:0/2555562201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10740042a0 con 0x5627d0011720 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f10797fa640 1 -- 172.21.15.17:0/2555562201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074005350 con 0x5627d0011720 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 -- 172.21.15.17:0/2555562201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0011720 msgr2=0x5627d0017900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0011720 0x5627d0017900 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1074005050 tx=0x7f1074041040 comp rx=0 tx=0).stop 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 -- 172.21.15.17:0/2555562201 shutdown_connections 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d001ac20 0x5627d001a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0012040 0x5627d001a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 --2- 172.21.15.17:0/2555562201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d0011720 0x5627d0017900 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 -- 172.21.15.17:0/2555562201 >> 172.21.15.17:0/2555562201 conn(0x5627d000be50 msgr2=0x5627d00171f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 -- 172.21.15.17:0/2555562201 shutdown_connections 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.263+0000 7f108cb82740 1 -- 172.21.15.17:0/2555562201 wait complete. 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 -- start start 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 Processor -- start 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0021e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d0012040 0x5627d00202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d001ac20 0x5627d0020830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d0020f30 con 0x5627d001ac20 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d00210a0 con 0x5627d0012040 2024-02-14T15:14:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627d0080e90 con 0x5627d0011720 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f108cb82740 10 client.0 _ll_register_callbacks cb 0x7f107c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d001ac20 0x5627d0020830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d001ac20 0x5627d0020830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34710/0 (socket says 172.21.15.17:34710) 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 -- 172.21.15.17:0/3741821333 learned_addr learned my addr 172.21.15.17:0/3741821333 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f10837fe640 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0021e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f107bfff640 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d0012040 0x5627d00202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 -- 172.21.15.17:0/3741821333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 msgr2=0x5627d0021e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0021e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 -- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d0012040 msgr2=0x5627d00202f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d0012040 0x5627d00202f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.264+0000 7f1083fff640 1 -- 172.21.15.17:0/3741821333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627cfc89920 con 0x5627d001ac20 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f1083fff640 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d001ac20 0x5627d0020830 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f107c09aee0 tx=0x7f107c009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3801d742a1b03f1e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f1062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f1062ffd640 1 -- 172.21.15.17:0/3741821333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0a3260 con 0x5627d001ac20 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f1062ffd640 1 -- 172.21.15.17:0/3741821333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f107c0a33c0 con 0x5627d001ac20 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1074004030 con 0x5627d001ac20 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f1062ffd640 1 -- 172.21.15.17:0/3741821333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f107c0b9250 con 0x5627d001ac20 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 10 client.17211 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5627cff39720 con 0x5627d001ac20 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f1062ffd640 1 -- 172.21.15.17:0/3741821333 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1074004030 con 0x5627d001ac20 2024-02-14T15:14:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f1062ffd640 1 -- 172.21.15.17:0/3741821333 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f107c01f060 con 0x5627d001ac20 2024-02-14T15:14:35.274 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:34 smithi017 ceph-mon[32549]: pgmap v350: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 172 op/s 2024-02-14T15:14:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 10 client.17211 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 20 client.17211 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d00825a0 0x5627d00849c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 4 client.17211 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:14:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.265+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5627cfcb5550 con 0x5627d00825a0 2024-02-14T15:14:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.266+0000 7f10837fe640 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d00825a0 0x5627d00849c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.266+0000 7f1062ffd640 10 client.17211 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.266+0000 7f10837fe640 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d00825a0 0x5627d00849c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc87f64512558f2b server_cookie=c9272f2932e4c in_seq=0 out_seq=0 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f1062ffd640 1 -- 172.21.15.17:0/3741821333 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5627cfcb5550 con 0x5627d00825a0 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f1062ffd640 10 client.17211 handle_command_reply: tid=0 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 client.17211 shutdown 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d001ac20 msgr2=0x5627d0020830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d001ac20 0x5627d0020830 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f107c09aee0 tx=0x7f107c009a70 comp rx=0 tx=0).stop 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 shutdown_connections 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5627d00825a0 0x5627d00849c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5627d001ac20 0x5627d0020830 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f1062ffd640 0 client.17211 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627d0012040 0x5627d00202f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 --2- 172.21.15.17:0/3741821333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d0011720 0x5627d0021e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 >> 172.21.15.17:0/3741821333 conn(0x5627d000be50 msgr2=0x5627d0012560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 shutdown_connections 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.278+0000 7f108cb82740 1 -- 172.21.15.17:0/3741821333 wait complete. 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f108cb82740 20 client.17211 trim_cache size 0 max 16384 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:14:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:14:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:14:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f10700a2250 msgr2=0x7f10700a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f10700a2250 0x7f10700a4710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1084101b80 tx=0x7f106c013830 comp rx=0 tx=0).stop 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084100b70 msgr2=0x7f10841a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084100b70 0x7f10841a5af0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f107c00eb10 tx=0x7f107c00c2d0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 shutdown_connections 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f10700a2250 0x7f10700a4710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1084100b70 0x7f10841a5af0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1084106510 0x7f1084100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 --2- 172.21.15.17:0/3818673783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1084068d40 0x7f10841000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 >> 172.21.15.17:0/3818673783 conn(0x7f1084077220 msgr2=0x7f10840fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 shutdown_connections 2024-02-14T15:14:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:35.279+0000 7f1089fbc640 1 -- 172.21.15.17:0/3818673783 wait complete. 2024-02-14T15:14:35.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1159185 10.64 MB/sec execute 457 sec latency 23.243 ms 2024-02-14T15:14:36.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1161484 10.65 MB/sec execute 458 sec latency 15.953 ms 2024-02-14T15:14:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:36 smithi050 ceph-mon[36512]: pgmap v351: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 172 op/s 2024-02-14T15:14:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:36 smithi076 ceph-mon[37057]: pgmap v351: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 172 op/s 2024-02-14T15:14:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:36 smithi017 ceph-mon[32549]: pgmap v351: 129 pgs: 129 active+clean; 601 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 172 op/s 2024-02-14T15:14:37.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1162388 10.63 MB/sec execute 459 sec latency 42.931 ms 2024-02-14T15:14:38.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1164823 10.64 MB/sec execute 460 sec latency 35.174 ms 2024-02-14T15:14:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:38 smithi050 ceph-mon[36512]: pgmap v352: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-02-14T15:14:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:38 smithi076 ceph-mon[37057]: pgmap v352: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-02-14T15:14:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:38 smithi017 ceph-mon[32549]: pgmap v352: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-02-14T15:14:39.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1165825 10.63 MB/sec execute 461 sec latency 20.111 ms 2024-02-14T15:14:40.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1168391 10.64 MB/sec execute 462 sec latency 22.483 ms 2024-02-14T15:14:41.067 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:40 smithi017 ceph-mon[32549]: pgmap v353: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-02-14T15:14:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:40 smithi050 ceph-mon[36512]: pgmap v353: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-02-14T15:14:41.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:40 smithi076 ceph-mon[37057]: pgmap v353: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-02-14T15:14:41.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1169491 10.62 MB/sec execute 463 sec latency 16.213 ms 2024-02-14T15:14:42.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1171892 10.63 MB/sec execute 464 sec latency 55.220 ms 2024-02-14T15:14:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:42 smithi050 ceph-mon[36512]: pgmap v354: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-02-14T15:14:43.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:42 smithi076 ceph-mon[37057]: pgmap v354: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-02-14T15:14:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:42 smithi017 ceph-mon[32549]: pgmap v354: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-02-14T15:14:43.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1172705 10.62 MB/sec execute 465 sec latency 22.035 ms 2024-02-14T15:14:44.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:43 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:14:44.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:43 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:14:44.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:43 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:14:44.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1174088 10.61 MB/sec execute 466 sec latency 272.560 ms 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: pgmap v355: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 172 op/s 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:14:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:44 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: pgmap v355: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 172 op/s 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:14:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:14:45.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:44 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.308 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: pgmap v355: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 172 op/s 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:14:45.309 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:44 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:14:45.887 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1175918 10.61 MB/sec execute 467 sec latency 98.215 ms 2024-02-14T15:14:46.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1177857 10.60 MB/sec execute 468 sec latency 17.260 ms 2024-02-14T15:14:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:46 smithi050 ceph-mon[36512]: pgmap v356: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 172 op/s 2024-02-14T15:14:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:46 smithi017 ceph-mon[32549]: pgmap v356: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 172 op/s 2024-02-14T15:14:47.266 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:46 smithi076 ceph-mon[37057]: pgmap v356: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 172 op/s 2024-02-14T15:14:47.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1179463 10.61 MB/sec execute 469 sec latency 82.525 ms 2024-02-14T15:14:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2390778999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4068d40 msgr2=0x7f9ec4069120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2390778999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4068d40 0x7f9ec4069120 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009930 tx=0x7f9eb402bc30 comp rx=0 tx=0).stop 2024-02-14T15:14:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2390778999 shutdown_connections 2024-02-14T15:14:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2390778999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4106510 0x7f9ec410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2390778999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec40696f0 0x7f9ec4105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2390778999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4068d40 0x7f9ec4069120 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2390778999 >> 172.21.15.17:0/2390778999 conn(0x7f9ec4077220 msgr2=0x7f9ec4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2390778999 shutdown_connections 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2390778999 wait complete. 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 Processor -- start 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.375+0000 7f9ec9a96640 1 -- start start 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec9a96640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec4068d40 0x7f9ec419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec9a96640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4106510 0x7f9ec419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec9a96640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec41976d0 0x7f9ec4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec9a96640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec41981a0 con 0x7f9ec4106510 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec9a96640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec4198310 con 0x7f9ec41976d0 2024-02-14T15:14:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec9a96640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec41984d0 con 0x7f9ec4068d40 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4106510 0x7f9ec419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4106510 0x7f9ec419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42724/0 (socket says 172.21.15.17:42724) 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 -- 172.21.15.17:0/2846067332 learned_addr learned my addr 172.21.15.17:0/2846067332 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec4068d40 0x7f9ec419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec41976d0 0x7f9ec4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 -- 172.21.15.17:0/2846067332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec4068d40 msgr2=0x7f9ec419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec4068d40 0x7f9ec419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 -- 172.21.15.17:0/2846067332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec41976d0 msgr2=0x7f9ec4197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec41976d0 0x7f9ec4197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 -- 172.21.15.17:0/2846067332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4009560 con 0x7f9ec4106510 2024-02-14T15:14:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4106510 0x7f9ec419db50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800e9f0 tx=0x7f9eb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec0ff9640 1 -- 172.21.15.17:0/2846067332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80096c0 con 0x7f9ec4106510 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec0ff9640 1 -- 172.21.15.17:0/2846067332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9eb8005500 con 0x7f9ec4106510 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec41a4470 con 0x7f9ec4106510 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.376+0000 7f9ec0ff9640 1 -- 172.21.15.17:0/2846067332 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8010430 con 0x7f9ec4106510 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.377+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec41a4900 con 0x7f9ec4106510 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.379+0000 7f9ec0ff9640 1 -- 172.21.15.17:0/2846067332 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9eb8005020 con 0x7f9ec4106510 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.379+0000 7f9ecc65e740 1 Processor -- start 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.379+0000 7f9ec0ff9640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9eac0a2220 0x7f9eac0a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.379+0000 7f9ecc65e740 1 -- start start 2024-02-14T15:14:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.379+0000 7f9ec0ff9640 1 -- 172.21.15.17:0/2846067332 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9eb8014070 con 0x7f9ec4106510 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.379+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9eac0a2220 0x7f9eac0a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.379+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb053b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb05c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb05f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beafc19e0 con 0x555beb054040 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beafbef10 con 0x555beb053720 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beafba6f0 con 0x555beb05cc20 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb05c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb053b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb05c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42740/0 (socket says 172.21.15.17:42740) 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb053b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:59046/0 (socket says 172.21.15.17:59046) 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec2ffd640 1 -- 172.21.15.17:0/506790750 learned_addr learned my addr 172.21.15.17:0/506790750 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb05f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9eac0a2220 0x7f9eac0a46e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 -- 172.21.15.17:0/506790750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 msgr2=0x555beb05f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb05f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 -- 172.21.15.17:0/506790750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 msgr2=0x555beb05c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb05c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 -- 172.21.15.17:0/506790750 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555beaccb920 con 0x555beb053720 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb05c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb053b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009230 tx=0x7f9eb402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7095dcab3b6c69f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.380+0000 7f9ea9ffb640 1 -- 172.21.15.17:0/506790750 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4004800 con 0x555beb053720 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ea9ffb640 1 -- 172.21.15.17:0/506790750 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4004960 con 0x555beb053720 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ea9ffb640 1 -- 172.21.15.17:0/506790750 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4005050 con 0x555beb053720 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 -- 172.21.15.17:0/506790750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 msgr2=0x555beb053b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb053b00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009230 tx=0x7f9eb402c5d0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 -- 172.21.15.17:0/506790750 shutdown_connections 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb05c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb05f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/506790750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb053b00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 -- 172.21.15.17:0/506790750 >> 172.21.15.17:0/506790750 conn(0x555beb04de50 msgr2=0x555beb04e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 -- 172.21.15.17:0/506790750 shutdown_connections 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.381+0000 7f9ecc65e740 1 -- 172.21.15.17:0/506790750 wait complete. 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 -- start start 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 Processor -- start 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb05c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb059960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb059f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beaf56fa0 con 0x555beb054040 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beac8e250 con 0x555beb053720 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beaf40200 con 0x555beb05cc20 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ecc65e740 10 client.0 _ll_register_callbacks cb 0x555beafafb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ec2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb059960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ec3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb059f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb05c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ec3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb059f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:39984/0 (socket says 172.21.15.17:39984) 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ec2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb059960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42748/0 (socket says 172.21.15.17:42748) 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb05c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:59052/0 (socket says 172.21.15.17:59052) 2024-02-14T15:14:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.382+0000 7f9ec3fff640 1 -- 172.21.15.17:0/2541777142 learned_addr learned my addr 172.21.15.17:0/2541777142 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec3fff640 1 -- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 msgr2=0x555beb05c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb05c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec3fff640 1 -- 172.21.15.17:0/2541777142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 msgr2=0x555beb059960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb059960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec3fff640 1 -- 172.21.15.17:0/2541777142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec410e8f0 con 0x555beb05cc20 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb05c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb059f00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc004380 tx=0x7f9ebc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2652d849f8a10e00 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2541777142 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc021070 con 0x555beb05cc20 2024-02-14T15:14:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2541777142 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9ebc004b20 con 0x555beb05cc20 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555beacba600 con 0x555beb05cc20 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2541777142 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ebc01c6f0 con 0x555beb05cc20 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ecc65e740 10 client.26734 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555beafb8d70 con 0x555beb05cc20 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.383+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2541777142 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555beacba600 con 0x555beb05cc20 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2541777142 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ebc025960 con 0x555beb05cc20 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ecc65e740 10 client.26734 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ecc65e740 10 client.26734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ecc65e740 20 client.26734 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb06c9d0 0x555beb06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ecc65e740 4 client.26734 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:14:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bead0b1b0 con 0x555beb06c9d0 2024-02-14T15:14:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb06c9d0 0x555beb06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ea2ffd640 10 client.26734 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.384+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb06c9d0 0x555beb06edf0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60042a72e2ee8754 server_cookie=d4be75b2f54c098 in_seq=0 out_seq=0 2024-02-14T15:14:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2541777142 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bead0b1b0 con 0x555beb06c9d0 2024-02-14T15:14:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ea2ffd640 10 client.26734 handle_command_reply: tid=0 2024-02-14T15:14:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 client.26734 shutdown 2024-02-14T15:14:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 msgr2=0x555beb059f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb059f00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc004380 tx=0x7f9ebc004ab0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 shutdown_connections 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb06c9d0 0x555beb06edf0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb054040 0x555beb059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ea2ffd640 0 client.26734 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb05cc20 0x555beb059f00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2541777142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb053720 0x555beb05c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 >> 172.21.15.17:0/2541777142 conn(0x555beb04de50 msgr2=0x555beb04fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 shutdown_connections 2024-02-14T15:14:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2541777142 wait complete. 2024-02-14T15:14:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.385+0000 7f9ecc65e740 20 client.26734 trim_cache size 0 max 16384 2024-02-14T15:14:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 Processor -- start 2024-02-14T15:14:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 -- start start 2024-02-14T15:14:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb05c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb05c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beb04c300 con 0x555beb053720 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beb05f340 con 0x555beb05cc20 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beb04d640 con 0x555beb054040 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb05c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb05c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:59064/0 (socket says 172.21.15.17:59064) 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb05c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42764/0 (socket says 172.21.15.17:42764) 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 -- 172.21.15.17:0/2232617348 learned_addr learned my addr 172.21.15.17:0/2232617348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 -- 172.21.15.17:0/2232617348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 msgr2=0x555beb05c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/2232617348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb05c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 -- 172.21.15.17:0/2232617348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 msgr2=0x555beb05c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/2232617348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb05c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.400+0000 7f9ec37fe640 1 -- 172.21.15.17:0/2232617348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555beaf55820 con 0x555beb053720 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/2232617348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb073750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009930 tx=0x7f9eb4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2cdafb9708c013 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ea97fa640 1 -- 172.21.15.17:0/2232617348 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb40044e0 con 0x555beb053720 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ea97fa640 1 -- 172.21.15.17:0/2232617348 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4051810 con 0x555beb053720 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ea97fa640 1 -- 172.21.15.17:0/2232617348 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb40050e0 con 0x555beb053720 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2232617348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 msgr2=0x555beb073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2232617348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb073750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009930 tx=0x7f9eb4041040 comp rx=0 tx=0).stop 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2232617348 shutdown_connections 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2232617348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb05c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2232617348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb05c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/2232617348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb073750 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.401+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2232617348 >> 172.21.15.17:0/2232617348 conn(0x555beb04de50 msgr2=0x555beb068520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2232617348 shutdown_connections 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 -- 172.21.15.17:0/2232617348 wait complete. 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 -- start start 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 Processor -- start 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb061a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb061f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb062530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beb0c2070 con 0x555beb053720 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beb0c21e0 con 0x555beb05cc20 2024-02-14T15:14:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555beb0c23a0 con 0x555beb054040 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ecc65e740 10 client.0 _ll_register_callbacks cb 0x7f9eb8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ec37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb061a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42778/0 (socket says 172.21.15.17:42778) 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ec37fe640 1 -- 172.21.15.17:0/659158631 learned_addr learned my addr 172.21.15.17:0/659158631 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb062530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.402+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb061f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ec37fe640 1 -- 172.21.15.17:0/659158631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 msgr2=0x555beb061f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb061f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ec37fe640 1 -- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 msgr2=0x555beb062530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:14:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb062530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ec37fe640 1 -- 172.21.15.17:0/659158631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555beaccb920 con 0x555beb053720 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ec3fff640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb062530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ec37fe640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb061a50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9eb402c240 tx=0x7f9eb402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d407cee34988ee server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/659158631 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4051530 con 0x555beb053720 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/659158631 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9eb404f070 con 0x555beb053720 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9eb4004960 con 0x555beb053720 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/659158631 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eb4051c80 con 0x555beb053720 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ecc65e740 10 client.17229 fetch_fsmap learned FSMap version 33 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555beaf7b720 con 0x555beb053720 2024-02-14T15:14:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.403+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/659158631 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4004960 con 0x555beb053720 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/659158631 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9eb4062070 con 0x555beb053720 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ecc65e740 10 client.17229 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ecc65e740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ecc65e740 20 client.17229 populate_metadata read hostname 'smithi017' 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb0c7b50 0x555beb0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ecc65e740 4 client.17229 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555beacf7550 con 0x555beb0c7b50 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb0c7b50 0x555beb0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ea2ffd640 10 client.17229 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.404+0000 7f9ec2ffd640 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb0c7b50 0x555beb0c9f40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8bd6f7a56e809a server_cookie=413d188b1f3982b2 in_seq=0 out_seq=0 2024-02-14T15:14:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.416+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/659158631 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555beacf7550 con 0x555beb0c7b50 2024-02-14T15:14:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.416+0000 7f9ea2ffd640 10 client.17229 handle_command_reply: tid=0 2024-02-14T15:14:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.416+0000 7f9ecc65e740 1 client.17229 shutdown 2024-02-14T15:14:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 msgr2=0x555beb061a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb061a50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9eb402c240 tx=0x7f9eb402c5d0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 shutdown_connections 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x555beb0c7b50 0x555beb0c9f40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb05cc20 0x555beb062530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ea2ffd640 0 client.17229 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555beb054040 0x555beb061f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 --2- 172.21.15.17:0/659158631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555beb053720 0x555beb061a50 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 >> 172.21.15.17:0/659158631 conn(0x555beb04de50 msgr2=0x555beb068520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 shutdown_connections 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 1 -- 172.21.15.17:0/659158631 wait complete. 2024-02-14T15:14:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ecc65e740 20 client.17229 trim_cache size 0 max 16384 2024-02-14T15:14:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:14:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:14:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:14:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9eac0a2220 msgr2=0x7f9eac0a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9eac0a2220 0x7f9eac0a46e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc009500 comp rx=0 tx=0).stop 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4106510 msgr2=0x7f9ec419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4106510 0x7f9ec419db50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800e9f0 tx=0x7f9eb800eeb0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 shutdown_connections 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9eac0a2220 0x7f9eac0a46e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ec4106510 0x7f9ec419db50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec41976d0 0x7f9ec4197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 --2- 172.21.15.17:0/2846067332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec4068d40 0x7f9ec419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:14:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.417+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 >> 172.21.15.17:0/2846067332 conn(0x7f9ec4077220 msgr2=0x7f9ec40fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:14:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.418+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 shutdown_connections 2024-02-14T15:14:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:14:48.418+0000 7f9ec9a96640 1 -- 172.21.15.17:0/2846067332 wait complete. 2024-02-14T15:14:48.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1181026 10.60 MB/sec execute 470 sec latency 22.929 ms 2024-02-14T15:14:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:48 smithi050 ceph-mon[36512]: pgmap v357: 129 pgs: 129 active+clean; 571 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-02-14T15:14:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:48 smithi076 ceph-mon[37057]: pgmap v357: 129 pgs: 129 active+clean; 571 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-02-14T15:14:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:48 smithi017 ceph-mon[32549]: pgmap v357: 129 pgs: 129 active+clean; 571 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-02-14T15:14:49.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1183040 10.60 MB/sec execute 471 sec latency 57.430 ms 2024-02-14T15:14:50.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1184997 10.59 MB/sec execute 472 sec latency 19.024 ms 2024-02-14T15:14:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:50 smithi050 ceph-mon[36512]: pgmap v358: 129 pgs: 129 active+clean; 571 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-02-14T15:14:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:50 smithi076 ceph-mon[37057]: pgmap v358: 129 pgs: 129 active+clean; 571 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-02-14T15:14:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:50 smithi017 ceph-mon[32549]: pgmap v358: 129 pgs: 129 active+clean; 571 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 169 op/s 2024-02-14T15:14:51.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1186612 10.60 MB/sec execute 473 sec latency 20.736 ms 2024-02-14T15:14:52.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1187949 10.59 MB/sec execute 474 sec latency 126.663 ms 2024-02-14T15:14:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:52 smithi050 ceph-mon[36512]: pgmap v359: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-02-14T15:14:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:52 smithi076 ceph-mon[37057]: pgmap v359: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-02-14T15:14:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:52 smithi017 ceph-mon[32549]: pgmap v359: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-02-14T15:14:53.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1189957 10.59 MB/sec execute 475 sec latency 35.484 ms 2024-02-14T15:14:54.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1191398 10.58 MB/sec execute 476 sec latency 32.225 ms 2024-02-14T15:14:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:54 smithi050 ceph-mon[36512]: pgmap v360: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 166 op/s 2024-02-14T15:14:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:54 smithi076 ceph-mon[37057]: pgmap v360: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 166 op/s 2024-02-14T15:14:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:54 smithi017 ceph-mon[32549]: pgmap v360: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 166 op/s 2024-02-14T15:14:55.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1193390 10.59 MB/sec execute 477 sec latency 137.216 ms 2024-02-14T15:14:56.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1194454 10.57 MB/sec execute 478 sec latency 16.159 ms 2024-02-14T15:14:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:56 smithi050 ceph-mon[36512]: pgmap v361: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 166 op/s 2024-02-14T15:14:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:56 smithi076 ceph-mon[37057]: pgmap v361: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 166 op/s 2024-02-14T15:14:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:56 smithi017 ceph-mon[32549]: pgmap v361: 129 pgs: 129 active+clean; 582 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 166 op/s 2024-02-14T15:14:57.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1196925 10.58 MB/sec execute 479 sec latency 17.109 ms 2024-02-14T15:14:58.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1197607 10.57 MB/sec execute 480 sec latency 107.969 ms 2024-02-14T15:14:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:14:58 smithi050 ceph-mon[36512]: pgmap v362: 129 pgs: 129 active+clean; 592 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 248 op/s 2024-02-14T15:14:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:14:58 smithi076 ceph-mon[37057]: pgmap v362: 129 pgs: 129 active+clean; 592 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 248 op/s 2024-02-14T15:14:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:14:58 smithi017 ceph-mon[32549]: pgmap v362: 129 pgs: 129 active+clean; 592 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 248 op/s 2024-02-14T15:14:59.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1200231 10.57 MB/sec execute 481 sec latency 12.164 ms 2024-02-14T15:15:00.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1201280 10.56 MB/sec execute 482 sec latency 14.370 ms 2024-02-14T15:15:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:00 smithi050 ceph-mon[36512]: pgmap v363: 129 pgs: 129 active+clean; 592 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 166 op/s 2024-02-14T15:15:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:00 smithi076 ceph-mon[37057]: pgmap v363: 129 pgs: 129 active+clean; 592 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 166 op/s 2024-02-14T15:15:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:00 smithi017 ceph-mon[32549]: pgmap v363: 129 pgs: 129 active+clean; 592 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 166 op/s 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.541+0000 7f896a15c640 1 -- 172.21.15.17:0/3525283489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964068d40 msgr2=0x7f8964069120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.541+0000 7f896a15c640 1 --2- 172.21.15.17:0/3525283489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964068d40 0x7f8964069120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8954009930 tx=0x7f895402bc30 comp rx=0 tx=0).stop 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.542+0000 7f896a15c640 1 -- 172.21.15.17:0/3525283489 shutdown_connections 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.542+0000 7f896a15c640 1 --2- 172.21.15.17:0/3525283489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8964106510 0x7f896410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.542+0000 7f896a15c640 1 --2- 172.21.15.17:0/3525283489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89640696f0 0x7f8964105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.542+0000 7f896a15c640 1 --2- 172.21.15.17:0/3525283489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964068d40 0x7f8964069120 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.542+0000 7f896a15c640 1 -- 172.21.15.17:0/3525283489 >> 172.21.15.17:0/3525283489 conn(0x7f8964077220 msgr2=0x7f8964077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.542+0000 7f896a15c640 1 -- 172.21.15.17:0/3525283489 shutdown_connections 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.542+0000 7f896a15c640 1 -- 172.21.15.17:0/3525283489 wait complete. 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 Processor -- start 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 -- start start 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964068d40 0x7f89641000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964106510 0x7f8964100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8964100b70 0x7f89641a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a6050 con 0x7f8964106510 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a61c0 con 0x7f8964068d40 2024-02-14T15:15:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f896a15c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a6380 con 0x7f8964100b70 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964106510 0x7f8964100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964106510 0x7f8964100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36262/0 (socket says 172.21.15.17:36262) 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f8962ffd640 1 -- 172.21.15.17:0/575224779 learned_addr learned my addr 172.21.15.17:0/575224779 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f8963fff640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8964100b70 0x7f89641a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.543+0000 7f89637fe640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964068d40 0x7f89641000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f89637fe640 1 -- 172.21.15.17:0/575224779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8964100b70 msgr2=0x7f89641a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f89637fe640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8964100b70 0x7f89641a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f89637fe640 1 -- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964106510 msgr2=0x7f8964100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f89637fe640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964106510 0x7f8964100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f89637fe640 1 -- 172.21.15.17:0/575224779 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954009560 con 0x7f8964068d40 2024-02-14T15:15:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f89637fe640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964068d40 0x7f89641000f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8954036040 tx=0x7f895402c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f8960ff9640 1 -- 172.21.15.17:0/575224779 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540042d0 con 0x7f8964068d40 2024-02-14T15:15:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89641a6600 con 0x7f8964068d40 2024-02-14T15:15:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f8960ff9640 1 -- 172.21.15.17:0/575224779 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8954004430 con 0x7f8964068d40 2024-02-14T15:15:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f8960ff9640 1 -- 172.21.15.17:0/575224779 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954003bf0 con 0x7f8964068d40 2024-02-14T15:15:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.544+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89641a6b30 con 0x7f8964068d40 2024-02-14T15:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 Processor -- start 2024-02-14T15:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 -- start start 2024-02-14T15:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f8960ff9640 1 -- 172.21.15.17:0/575224779 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8954003d50 con 0x7f8964068d40 2024-02-14T15:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6463b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe646c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe646f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe63d19e0 con 0x556fe646cc20 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe63cef10 con 0x556fe6463720 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f896cdfe740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe63ca6f0 con 0x556fe6464040 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe646f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f8963fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6463b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f89637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe646c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f8963fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6463b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:55012/0 (socket says 172.21.15.17:55012) 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f8963fff640 1 -- 172.21.15.17:0/1668917385 learned_addr learned my addr 172.21.15.17:0/1668917385 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.547+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe646f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36268/0 (socket says 172.21.15.17:36268) 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8960ff9640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f894c0a2250 0x7f894c0a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8963fff640 1 -- 172.21.15.17:0/1668917385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 msgr2=0x556fe646c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8963fff640 1 --2- 172.21.15.17:0/1668917385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe646c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8963fff640 1 -- 172.21.15.17:0/1668917385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 msgr2=0x556fe646f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8963fff640 1 --2- 172.21.15.17:0/1668917385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe646f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8963fff640 1 -- 172.21.15.17:0/1668917385 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fe60db920 con 0x556fe6463720 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8960ff9640 1 -- 172.21.15.17:0/575224779 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f89540f0890 con 0x7f8964068d40 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8963fff640 1 --2- 172.21.15.17:0/1668917385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6463b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f895c009f50 tx=0x7f895c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f86ed956894f22d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8963fff640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f894c0a2250 0x7f894c0a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8949ffb640 1 -- 172.21.15.17:0/1668917385 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c020030 con 0x556fe6463720 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8949ffb640 1 -- 172.21.15.17:0/1668917385 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895c003e70 con 0x556fe6463720 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.548+0000 7f8949ffb640 1 -- 172.21.15.17:0/1668917385 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c01b430 con 0x556fe6463720 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.549+0000 7f896cdfe740 1 -- 172.21.15.17:0/1668917385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 msgr2=0x556fe6463b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.549+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1668917385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6463b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f895c009f50 tx=0x7f895c0118d0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.549+0000 7f8963fff640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f894c0a2250 0x7f894c0a4710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f895c012fd0 tx=0x7f895c01ba20 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.549+0000 7f896cdfe740 1 -- 172.21.15.17:0/1668917385 shutdown_connections 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1668917385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe646f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1668917385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe646c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1668917385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6463b00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 -- 172.21.15.17:0/1668917385 >> 172.21.15.17:0/1668917385 conn(0x556fe645de50 msgr2=0x556fe645e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 -- 172.21.15.17:0/1668917385 shutdown_connections 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 -- 172.21.15.17:0/1668917385 wait complete. 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 -- start start 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 Processor -- start 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe646c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.550+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe64698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe6469e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f896cdfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe6366fa0 con 0x556fe6463720 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f89637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe646c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f89637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe646c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36272/0 (socket says 172.21.15.17:36272) 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe609e250 con 0x556fe6464040 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f89637fe640 1 -- 172.21.15.17:0/1785295085 learned_addr learned my addr 172.21.15.17:0/1785295085 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe6350200 con 0x556fe646cc20 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f8962ffd640 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe64698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f8963fff640 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe6469e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f896cdfe740 10 client.0 _ll_register_callbacks cb 0x556fe63bfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f8962ffd640 1 -- 172.21.15.17:0/1785295085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 msgr2=0x556fe6469e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f8962ffd640 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe6469e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f8962ffd640 1 -- 172.21.15.17:0/1785295085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 msgr2=0x556fe646c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f8962ffd640 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe646c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.551+0000 7f8962ffd640 1 -- 172.21.15.17:0/1785295085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c009c00 con 0x556fe6464040 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f8962ffd640 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe64698f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8958009600 tx=0x7f8958012b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cd92f9e1a63575 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f8942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f8942ffd640 1 -- 172.21.15.17:0/1785295085 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958017070 con 0x556fe6464040 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f8942ffd640 1 -- 172.21.15.17:0/1785295085 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f8958004050 con 0x556fe6464040 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fe60ca600 con 0x556fe6464040 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f8942ffd640 1 -- 172.21.15.17:0/1785295085 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8958004210 con 0x556fe6464040 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 10 client.26579 fetch_fsmap learned FSMap version 33 2024-02-14T15:15:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556fe63c8d70 con 0x556fe6464040 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f8942ffd640 1 -- 172.21.15.17:0/1785295085 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556fe60ca600 con 0x556fe6464040 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f8942ffd640 1 -- 172.21.15.17:0/1785295085 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f89580039c0 con 0x556fe6464040 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 10 client.26579 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 10 client.26579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 20 client.26579 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe647c9a0 0x556fe647edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 4 client.26579 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.552+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fe611b1b0 con 0x556fe647c9a0 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.553+0000 7f89637fe640 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe647c9a0 0x556fe647edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.553+0000 7f8942ffd640 10 client.26579 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.553+0000 7f89637fe640 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe647c9a0 0x556fe647edc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902bec4e26a097f9 server_cookie=b75efb5b9c17443b in_seq=0 out_seq=0 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f8942ffd640 1 -- 172.21.15.17:0/1785295085 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556fe611b1b0 con 0x556fe647c9a0 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f8942ffd640 10 client.26579 handle_command_reply: tid=0 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 client.26579 shutdown 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 msgr2=0x556fe64698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe64698f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8958009600 tx=0x7f8958012b30 comp rx=0 tx=0).stop 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 shutdown_connections 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe647c9a0 0x556fe647edc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe6469e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe64698f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.566 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f8942ffd640 0 client.26579 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 --2- 172.21.15.17:0/1785295085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe646c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 >> 172.21.15.17:0/1785295085 conn(0x556fe645de50 msgr2=0x556fe645fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 shutdown_connections 2024-02-14T15:15:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 1 -- 172.21.15.17:0/1785295085 wait complete. 2024-02-14T15:15:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.554+0000 7f896cdfe740 20 client.26579 trim_cache size 0 max 16384 2024-02-14T15:15:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 Processor -- start 2024-02-14T15:15:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 -- start start 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe645f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe646c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe646c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe645ee50 con 0x556fe6463720 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe64630f0 con 0x556fe6464040 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.569+0000 7f896cdfe740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe6462aa0 con 0x556fe646cc20 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe646c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f89637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe645f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe646c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe646c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37158/0 (socket says 172.21.15.17:37158) 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe646c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:55018/0 (socket says 172.21.15.17:55018) 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 -- 172.21.15.17:0/3442364079 learned_addr learned my addr 172.21.15.17:0/3442364079 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f89637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe645f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36288/0 (socket says 172.21.15.17:36288) 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 -- 172.21.15.17:0/3442364079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 msgr2=0x556fe646c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 --2- 172.21.15.17:0/3442364079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe646c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 -- 172.21.15.17:0/3442364079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 msgr2=0x556fe645f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 --2- 172.21.15.17:0/3442364079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe645f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 -- 172.21.15.17:0/3442364079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fe6365820 con 0x556fe646cc20 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f8963fff640 1 --2- 172.21.15.17:0/3442364079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe646c560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f895c011dd0 tx=0x7f895c028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f1a91fa166abd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.570+0000 7f89497fa640 1 -- 172.21.15.17:0/3442364079 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c020070 con 0x556fe646cc20 2024-02-14T15:15:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f89497fa640 1 -- 172.21.15.17:0/3442364079 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895c003970 con 0x556fe646cc20 2024-02-14T15:15:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f89497fa640 1 -- 172.21.15.17:0/3442364079 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c004510 con 0x556fe646cc20 2024-02-14T15:15:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 -- 172.21.15.17:0/3442364079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 msgr2=0x556fe646c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3442364079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe646c560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f895c011dd0 tx=0x7f895c028000 comp rx=0 tx=0).stop 2024-02-14T15:15:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 -- 172.21.15.17:0/3442364079 shutdown_connections 2024-02-14T15:15:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3442364079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe646cc20 0x556fe646c560 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3442364079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6464040 0x556fe646c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3442364079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe6463720 0x556fe645f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 -- 172.21.15.17:0/3442364079 >> 172.21.15.17:0/3442364079 conn(0x556fe645de50 msgr2=0x556fe645f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 -- 172.21.15.17:0/3442364079 shutdown_connections 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.571+0000 7f896cdfe740 1 -- 172.21.15.17:0/3442364079 wait complete. 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 -- start start 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 Processor -- start 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6472810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe6472d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe6473290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe64d2f90 con 0x556fe646cc20 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe64d3100 con 0x556fe6463720 2024-02-14T15:15:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.572+0000 7f896cdfe740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fe64d32c0 con 0x556fe6464040 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f89637fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6472810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe6472d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe6472d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37162/0 (socket says 172.21.15.17:37162) 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f89637fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6472810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:55032/0 (socket says 172.21.15.17:55032) 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 -- 172.21.15.17:0/3776538647 learned_addr learned my addr 172.21.15.17:0/3776538647 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f896cdfe740 10 client.0 _ll_register_callbacks cb 0x7f895c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8963fff640 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe6473290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 -- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 msgr2=0x556fe6472810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6472810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 -- 172.21.15.17:0/3776538647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 msgr2=0x556fe6473290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe6473290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 -- 172.21.15.17:0/3776538647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fe60db920 con 0x556fe6464040 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f89637fe640 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6472810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8962ffd640 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe6472d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x556fe646aec0 tx=0x7f89580104b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c404c0a7181b5f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8942ffd640 1 -- 172.21.15.17:0/3776538647 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958017040 con 0x556fe6464040 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f895c003e70 con 0x556fe6464040 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.573+0000 7f8942ffd640 1 -- 172.21.15.17:0/3776538647 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f8958003800 con 0x556fe6464040 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f8942ffd640 1 -- 172.21.15.17:0/3776538647 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89580039c0 con 0x556fe6464040 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 10 client.26758 fetch_fsmap learned FSMap version 33 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556fe638b720 con 0x556fe6464040 2024-02-14T15:15:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f8942ffd640 1 -- 172.21.15.17:0/3776538647 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895c003e70 con 0x556fe6464040 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f8942ffd640 1 -- 172.21.15.17:0/3776538647 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f89580107b0 con 0x556fe6464040 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 10 client.26758 fetch_fsmap finished waiting for FSMap version 33 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 10 client.26758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 20 client.26758 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe64d4a20 0x556fe64d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 4 client.26758 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556fe6107550 con 0x556fe64d4a20 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.574+0000 7f89637fe640 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe64d4a20 0x556fe64d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.575+0000 7f8942ffd640 10 client.26758 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:01.575+0000 7f89637fe640 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe64d4a20 0x556fe64d6e40 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2ff84325ea69c2 server_cookie=4c3a1be3c512cf0c in_seq=0 out_seq=0 2024-02-14T15:15:01.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1201897 10.55 MB/sec execute 483 sec latency 95.490 ms 2024-02-14T15:15:02.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1204045 10.55 MB/sec execute 484 sec latency 18.140 ms 2024-02-14T15:15:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f8942ffd640 1 -- 172.21.15.17:0/3776538647 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556fe6107550 con 0x556fe64d4a20 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f8942ffd640 10 client.26758 handle_command_reply: tid=0 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 client.26758 shutdown 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 msgr2=0x556fe6472d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe6472d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x556fe646aec0 tx=0x7f89580104b0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 shutdown_connections 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556fe64d4a20 0x556fe64d6e40 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556fe646cc20 0x556fe6473290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556fe6464040 0x556fe6472d50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f8942ffd640 0 client.26758 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 --2- 172.21.15.17:0/3776538647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556fe6463720 0x556fe6472810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 >> 172.21.15.17:0/3776538647 conn(0x556fe645de50 msgr2=0x556fe645fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 shutdown_connections 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 1 -- 172.21.15.17:0/3776538647 wait complete. 2024-02-14T15:15:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.181+0000 7f896cdfe740 20 client.26758 trim_cache size 0 max 16384 2024-02-14T15:15:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:15:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:15:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:15:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f894c0a2250 msgr2=0x7f894c0a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f894c0a2250 0x7f894c0a4710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f895c012fd0 tx=0x7f895c01ba20 comp rx=0 tx=0).stop 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964068d40 msgr2=0x7f89641000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964068d40 0x7f89641000f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8954036040 tx=0x7f895402c640 comp rx=0 tx=0).stop 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 shutdown_connections 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f894c0a2250 0x7f894c0a4710 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8964106510 0x7f8964100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8964100b70 0x7f89641a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 --2- 172.21.15.17:0/575224779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964068d40 0x7f89641000f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 >> 172.21.15.17:0/575224779 conn(0x7f8964077220 msgr2=0x7f89640fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 shutdown_connections 2024-02-14T15:15:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:03.182+0000 7f896a15c640 1 -- 172.21.15.17:0/575224779 wait complete. 2024-02-14T15:15:03.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:02 smithi076 ceph-mon[37057]: pgmap v364: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 270 op/s 2024-02-14T15:15:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:02 smithi017 ceph-mon[32549]: pgmap v364: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 270 op/s 2024-02-14T15:15:03.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:02 smithi050 ceph-mon[36512]: pgmap v364: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 270 op/s 2024-02-14T15:15:03.888 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1204888 10.54 MB/sec execute 485 sec latency 97.288 ms 2024-02-14T15:15:04.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: *** Caught signal (Segmentation fault) ** 2024-02-14T15:15:04.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: in thread 7fa84b13c700 thread_name:md_log_replay 2024-02-14T15:15:04.393 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa85c4fdd20] 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2: /usr/bin/ceph-mds(+0x503d42) [0x55e1a1f8ad42] 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e1a1f8cf93] 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 4: (MDLog::_replay_thread()+0x647) [0x55e1a1fe97b7] 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e1a1ff3651] 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa85c4f31ca] 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 7: clone() 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2024-02-14T15:15:03.956+0000 7fa84b13c700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: in thread 7fa84b13c700 thread_name:md_log_replay 2024-02-14T15:15:04.394 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa85c4fdd20] 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2: /usr/bin/ceph-mds(+0x503d42) [0x55e1a1f8ad42] 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e1a1f8cf93] 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 4: (MDLog::_replay_thread()+0x647) [0x55e1a1fe97b7] 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e1a1ff3651] 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa85c4f31ca] 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 7: clone() 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:15:04.395 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 0> 2024-02-14T15:15:03.956+0000 7fa84b13c700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: in thread 7fa84b13c700 thread_name:md_log_replay 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa85c4fdd20] 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2: /usr/bin/ceph-mds(+0x503d42) [0x55e1a1f8ad42] 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e1a1f8cf93] 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 4: (MDLog::_replay_thread()+0x647) [0x55e1a1fe97b7] 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e1a1ff3651] 2024-02-14T15:15:04.396 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa85c4f31ca] 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 7: clone() 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 0> 2024-02-14T15:15:03.956+0000 7fa84b13c700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: in thread 7fa84b13c700 thread_name:md_log_replay 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa85c4fdd20] 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2: /usr/bin/ceph-mds(+0x503d42) [0x55e1a1f8ad42] 2024-02-14T15:15:04.397 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55e1a1f8cf93] 2024-02-14T15:15:04.398 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 4: (MDLog::_replay_thread()+0x647) [0x55e1a1fe97b7] 2024-02-14T15:15:04.398 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55e1a1ff3651] 2024-02-14T15:15:04.398 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fa85c4f31ca] 2024-02-14T15:15:04.398 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 7: clone() 2024-02-14T15:15:04.398 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:15:04.398 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[72368]: 2024-02-14T15:15:04.398 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:04 smithi050 podman[73227]: 2024-02-14 15:15:04.022728865 +0000 UTC m=+0.021466942 container died ae581fba06637b172763cd7b23ff86e4779013ed504bd322540049940ac59ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-14T15:15:04.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1207369 10.54 MB/sec execute 486 sec latency 38.282 ms 2024-02-14T15:15:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:04 smithi050 ceph-mon[36512]: pgmap v365: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 185 op/s 2024-02-14T15:15:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:04 smithi050 ceph-mon[36512]: osdmap e108: 12 total, 12 up, 12 in 2024-02-14T15:15:05.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:04 smithi050 podman[73227]: 2024-02-14 15:15:04.860572142 +0000 UTC m=+0.859310204 container remove ae581fba06637b172763cd7b23ff86e4779013ed504bd322540049940ac59ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True) 2024-02-14T15:15:05.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:04 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-02-14T15:15:05.143 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:05 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-02-14T15:15:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:04 smithi076 ceph-mon[37057]: pgmap v365: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 185 op/s 2024-02-14T15:15:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:04 smithi076 ceph-mon[37057]: osdmap e108: 12 total, 12 up, 12 in 2024-02-14T15:15:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:04 smithi017 ceph-mon[32549]: pgmap v365: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 185 op/s 2024-02-14T15:15:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:04 smithi017 ceph-mon[32549]: osdmap e108: 12 total, 12 up, 12 in 2024-02-14T15:15:05.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1208390 10.53 MB/sec execute 487 sec latency 25.967 ms 2024-02-14T15:15:06.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1210053 10.53 MB/sec execute 488 sec latency 207.461 ms 2024-02-14T15:15:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:06 smithi076 ceph-mon[37057]: pgmap v367: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2024-02-14T15:15:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:06 smithi017 ceph-mon[32549]: pgmap v367: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2024-02-14T15:15:07.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:06 smithi050 ceph-mon[36512]: pgmap v367: 129 pgs: 129 active+clean; 603 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 223 op/s 2024-02-14T15:15:07.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1211585 10.53 MB/sec execute 489 sec latency 26.675 ms 2024-02-14T15:15:08.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1213028 10.52 MB/sec execute 490 sec latency 54.972 ms 2024-02-14T15:15:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:08 smithi076 ceph-mon[37057]: pgmap v368: 129 pgs: 129 active+clean; 609 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-02-14T15:15:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:08 smithi017 ceph-mon[32549]: pgmap v368: 129 pgs: 129 active+clean; 609 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-02-14T15:15:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:08 smithi050 ceph-mon[36512]: pgmap v368: 129 pgs: 129 active+clean; 609 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-02-14T15:15:09.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1215028 10.52 MB/sec execute 491 sec latency 12.777 ms 2024-02-14T15:15:10.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1216803 10.51 MB/sec execute 492 sec latency 32.022 ms 2024-02-14T15:15:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:10 smithi076 ceph-mon[37057]: pgmap v369: 129 pgs: 129 active+clean; 609 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-02-14T15:15:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:10 smithi076 ceph-mon[37057]: osdmap e109: 12 total, 12 up, 12 in 2024-02-14T15:15:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:10 smithi017 ceph-mon[32549]: pgmap v369: 129 pgs: 129 active+clean; 609 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-02-14T15:15:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:10 smithi017 ceph-mon[32549]: osdmap e109: 12 total, 12 up, 12 in 2024-02-14T15:15:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:10 smithi050 ceph-mon[36512]: pgmap v369: 129 pgs: 129 active+clean; 609 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-02-14T15:15:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:10 smithi050 ceph-mon[36512]: osdmap e109: 12 total, 12 up, 12 in 2024-02-14T15:15:11.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1218478 10.52 MB/sec execute 493 sec latency 65.464 ms 2024-02-14T15:15:12.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1220125 10.51 MB/sec execute 494 sec latency 15.674 ms 2024-02-14T15:15:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:12 smithi076 ceph-mon[37057]: pgmap v371: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 267 op/s 2024-02-14T15:15:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:12 smithi017 ceph-mon[32549]: pgmap v371: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 267 op/s 2024-02-14T15:15:13.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:12 smithi050 ceph-mon[36512]: pgmap v371: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.3 MiB/s wr, 267 op/s 2024-02-14T15:15:13.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1221843 10.51 MB/sec execute 495 sec latency 65.348 ms 2024-02-14T15:15:14.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1223178 10.50 MB/sec execute 496 sec latency 52.097 ms 2024-02-14T15:15:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:14 smithi076 ceph-mon[37057]: pgmap v372: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 232 op/s 2024-02-14T15:15:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:14 smithi017 ceph-mon[32549]: pgmap v372: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 232 op/s 2024-02-14T15:15:15.371 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 4. 2024-02-14T15:15:15.371 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 systemd[1]: Stopped Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:15:15.371 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 systemd[1]: Starting Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:15:15.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:14 smithi050 ceph-mon[36512]: pgmap v372: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 232 op/s 2024-02-14T15:15:15.642 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 podman[73336]: 2024-02-14 15:15:15.274005911 +0000 UTC m=+0.018912049 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:15:15.642 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 podman[73336]: 2024-02-14 15:15:15.386743675 +0000 UTC m=+0.131649790 container create 9da18a661f5c618383223b23593080b8effb14b38d1a670d3850aa84e3012096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD) 2024-02-14T15:15:15.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1225369 10.51 MB/sec execute 497 sec latency 19.276 ms 2024-02-14T15:15:15.913 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 podman[73336]: 2024-02-14 15:15:15.756950928 +0000 UTC m=+0.501857056 container init 9da18a661f5c618383223b23593080b8effb14b38d1a670d3850aa84e3012096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-14T15:15:15.913 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 podman[73336]: 2024-02-14 15:15:15.76003602 +0000 UTC m=+0.504942144 container start 9da18a661f5c618383223b23593080b8effb14b38d1a670d3850aa84e3012096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131) 2024-02-14T15:15:15.913 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 ceph-mds[73356]: -- 172.21.15.50:0/3196428778 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621c479b520 con 0x5621c4770c00 2024-02-14T15:15:15.913 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[73351]: starting mds.h at 2024-02-14T15:15:15.913 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 bash[73336]: 9da18a661f5c618383223b23593080b8effb14b38d1a670d3850aa84e3012096 2024-02-14T15:15:15.913 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:15:15 smithi050 systemd[1]: Started Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:15:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:15 smithi076 ceph-mon[37057]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-02-14T15:15:16.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:15 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-02-14T15:15:16.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:15 smithi017 ceph-mon[32549]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-02-14T15:15:16.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:15 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-02-14T15:15:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/3036728555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 msgr2=0x7fd9b0101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/3036728555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b0101840 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009930 tx=0x7fd9a402bc30 comp rx=0 tx=0).stop 2024-02-14T15:15:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/3036728555 shutdown_connections 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/3036728555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9b0106530 0x7fd9b010e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/3036728555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9b0101e10 0x7fd9b0105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/3036728555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b0101840 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/3036728555 >> 172.21.15.17:0/3036728555 conn(0x7fd9b00fccc0 msgr2=0x7fd9b00ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/3036728555 shutdown_connections 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/3036728555 wait complete. 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 Processor -- start 2024-02-14T15:15:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- start start 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9b0106530 0x7fd9b019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9b01977a0 0x7fd9b0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0198270 con 0x7fd9b0101460 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b01983e0 con 0x7fd9b01977a0 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.270+0000 7fd9b6a6f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b01985a0 con 0x7fd9b0106530 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37494/0 (socket says 172.21.15.17:37494) 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 -- 172.21.15.17:0/1998008975 learned_addr learned my addr 172.21.15.17:0/1998008975 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9affff640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9b0106530 0x7fd9b019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b500c640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9b01977a0 0x7fd9b0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 -- 172.21.15.17:0/1998008975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9b0106530 msgr2=0x7fd9b019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9b0106530 0x7fd9b019dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 -- 172.21.15.17:0/1998008975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9b01977a0 msgr2=0x7fd9b0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9b01977a0 0x7fd9b0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 -- 172.21.15.17:0/1998008975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a4009560 con 0x7fd9b0101460 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b480b640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b019d690 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009230 tx=0x7fd9a402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9adffb640 1 -- 172.21.15.17:0/1998008975 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4004350 con 0x7fd9b0101460 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9adffb640 1 -- 172.21.15.17:0/1998008975 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9a40044b0 con 0x7fd9b0101460 2024-02-14T15:15:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9adffb640 1 -- 172.21.15.17:0/1998008975 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4003d00 con 0x7fd9b0101460 2024-02-14T15:15:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b01a4490 con 0x7fd9b0101460 2024-02-14T15:15:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.271+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b01a4920 con 0x7fd9b0101460 2024-02-14T15:15:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.274+0000 7fd9adffb640 1 -- 172.21.15.17:0/1998008975 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd9a4004df0 con 0x7fd9b0101460 2024-02-14T15:15:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.274+0000 7fd9b9641740 1 Processor -- start 2024-02-14T15:15:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.274+0000 7fd9b9641740 1 -- start start 2024-02-14T15:15:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f313450 0x55929f313830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f320af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f321030 0x55929f323420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b9641740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f2819e0 con 0x55929f321030 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b9641740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f27ef10 con 0x55929f313d70 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b9641740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f27a6f0 con 0x55929f313450 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9adffb640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9a00a2220 0x7fd9a00a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f321030 0x55929f323420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f321030 0x55929f323420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37504/0 (socket says 172.21.15.17:37504) 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 -- 172.21.15.17:0/610839099 learned_addr learned my addr 172.21.15.17:0/610839099 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b500c640 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f313450 0x55929f313830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b480b640 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f320af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b500c640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9a00a2220 0x7fd9a00a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9adffb640 1 -- 172.21.15.17:0/1998008975 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd9a40f03a0 con 0x7fd9b0101460 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 -- 172.21.15.17:0/610839099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f313450 msgr2=0x55929f313830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f313450 0x55929f313830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 -- 172.21.15.17:0/610839099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 msgr2=0x55929f320af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f320af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 -- 172.21.15.17:0/610839099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55929ef8b920 con 0x55929f321030 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9b500c640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9a00a2220 0x7fd9a00a46e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005fd0 tx=0x7fd9a8009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd9affff640 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f321030 0x55929f323420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009c60 tx=0x7fd99c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac58b849b7f59fb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd99affd640 1 -- 172.21.15.17:0/610839099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c004230 con 0x55929f321030 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd99affd640 1 -- 172.21.15.17:0/610839099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd99c009e10 con 0x55929f321030 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.275+0000 7fd99affd640 1 -- 172.21.15.17:0/610839099 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c004c30 con 0x55929f321030 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 -- 172.21.15.17:0/610839099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f321030 msgr2=0x55929f323420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f321030 0x55929f323420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009c60 tx=0x7fd99c011810 comp rx=0 tx=0).stop 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 -- 172.21.15.17:0/610839099 shutdown_connections 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f321030 0x55929f323420 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f320af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 --2- 172.21.15.17:0/610839099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f313450 0x55929f313830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 -- 172.21.15.17:0/610839099 >> 172.21.15.17:0/610839099 conn(0x55929f30de50 msgr2=0x55929f30e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 -- 172.21.15.17:0/610839099 shutdown_connections 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 -- 172.21.15.17:0/610839099 wait complete. 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.276+0000 7fd9b9641740 1 -- start start 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 1 Processor -- start 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f320920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f31dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f31e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f216fa0 con 0x55929f313d70 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929ef4e250 con 0x55929f313450 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f200200 con 0x55929f321030 2024-02-14T15:15:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b9641740 10 client.0 _ll_register_callbacks cb 0x55929f26fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f31dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b480b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f320920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f31dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37514/0 (socket says 172.21.15.17:37514) 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9affff640 1 -- 172.21.15.17:0/974127529 learned_addr learned my addr 172.21.15.17:0/974127529 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b500c640 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f31e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b480b640 1 -- 172.21.15.17:0/974127529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 msgr2=0x55929f31e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b480b640 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f31e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b480b640 1 -- 172.21.15.17:0/974127529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 msgr2=0x55929f31dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b480b640 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f31dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9b480b640 1 -- 172.21.15.17:0/974127529 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c009890 con 0x55929f313450 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.277+0000 7fd9affff640 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f31dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b480b640 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f320920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40ed7e0 tx=0x7fd9a408c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c01bec57d22a128 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd98ffff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd98ffff640 1 -- 172.21.15.17:0/974127529 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40edb50 con 0x55929f313450 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55929ef7a600 con 0x55929f313450 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd98ffff640 1 -- 172.21.15.17:0/974127529 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd9a40035f0 con 0x55929f313450 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd98ffff640 1 -- 172.21.15.17:0/974127529 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9a40f0180 con 0x55929f313450 2024-02-14T15:15:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 10 client.26603 fetch_fsmap learned FSMap version 34 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55929f278d70 con 0x55929f313450 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd98ffff640 1 -- 172.21.15.17:0/974127529 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55929ef7a600 con 0x55929f313450 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd98ffff640 1 -- 172.21.15.17:0/974127529 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 34) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fd9a403c070 con 0x55929f313450 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 10 client.26603 fetch_fsmap finished waiting for FSMap version 34 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 20 client.26603 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f330c20 0x55929f333030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 4 client.26603 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55929efcb1b0 con 0x55929f330c20 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.278+0000 7fd9affff640 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f330c20 0x55929f333030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.279+0000 7fd9affff640 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f330c20 0x55929f333030 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3cf9e93182ccb4e server_cookie=a2a74e742184bd90 in_seq=0 out_seq=0 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.279+0000 7fd98ffff640 10 client.26603 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.279+0000 7fd98ffff640 1 -- 172.21.15.17:0/974127529 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55929efcb1b0 con 0x55929f330c20 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.279+0000 7fd98ffff640 10 client.26603 handle_command_reply: tid=0 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.279+0000 7fd9b9641740 1 client.26603 shutdown 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 msgr2=0x55929f320920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f320920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40ed7e0 tx=0x7fd9a408c8e0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 shutdown_connections 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f330c20 0x55929f333030 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f31dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f31e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd98ffff640 0 client.26603 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 --2- 172.21.15.17:0/974127529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f320920 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 >> 172.21.15.17:0/974127529 conn(0x55929f30de50 msgr2=0x55929f30fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 shutdown_connections 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 1 -- 172.21.15.17:0/974127529 wait complete. 2024-02-14T15:15:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.280+0000 7fd9b9641740 20 client.26603 trim_cache size 0 max 16384 2024-02-14T15:15:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 Processor -- start 2024-02-14T15:15:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 -- start start 2024-02-14T15:15:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313450 0x55929f31cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f3203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f320920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f309080 con 0x55929f313450 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f338250 con 0x55929f313d70 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b9641740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f30c2d0 con 0x55929f321030 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313450 0x55929f31cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313450 0x55929f31cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37524/0 (socket says 172.21.15.17:37524) 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 -- 172.21.15.17:0/4027529657 learned_addr learned my addr 172.21.15.17:0/4027529657 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9affff640 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f3203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b500c640 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f320920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 -- 172.21.15.17:0/4027529657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 msgr2=0x55929f320920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f320920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 -- 172.21.15.17:0/4027529657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 msgr2=0x55929f3203e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f3203e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 -- 172.21.15.17:0/4027529657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55929f215820 con 0x55929f313450 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.293+0000 7fd9b480b640 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313450 0x55929f31cbf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40bca10 tx=0x7fd9a402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7325afe791ea0dc6 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd99a7fc640 1 -- 172.21.15.17:0/4027529657 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40c4f90 con 0x55929f313450 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd99a7fc640 1 -- 172.21.15.17:0/4027529657 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9a40bc3e0 con 0x55929f313450 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd99a7fc640 1 -- 172.21.15.17:0/4027529657 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40edb20 con 0x55929f313450 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 -- 172.21.15.17:0/4027529657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313450 msgr2=0x55929f31cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313450 0x55929f31cbf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40bca10 tx=0x7fd9a402bc30 comp rx=0 tx=0).stop 2024-02-14T15:15:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 -- 172.21.15.17:0/4027529657 shutdown_connections 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f320920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313d70 0x55929f3203e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 --2- 172.21.15.17:0/4027529657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313450 0x55929f31cbf0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 -- 172.21.15.17:0/4027529657 >> 172.21.15.17:0/4027529657 conn(0x55929f30de50 msgr2=0x55929f319320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 -- 172.21.15.17:0/4027529657 shutdown_connections 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 -- 172.21.15.17:0/4027529657 wait complete. 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 -- start start 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.294+0000 7fd9b9641740 1 Processor -- start 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f328170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f3265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b9641740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f326af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b9641740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f327100 con 0x55929f313d70 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b9641740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f327270 con 0x55929f313450 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b9641740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929f327430 con 0x55929f321030 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b9641740 10 client.0 _ll_register_callbacks cb 0x7fd99c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f3265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f3265b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37540/0 (socket says 172.21.15.17:37540) 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 -- 172.21.15.17:0/3196604944 learned_addr learned my addr 172.21.15.17:0/3196604944 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b480b640 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f328170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9b500c640 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f326af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 -- 172.21.15.17:0/3196604944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 msgr2=0x55929f326af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f326af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 -- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 msgr2=0x55929f328170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f328170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 -- 172.21.15.17:0/3196604944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55929f278d70 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd9affff640 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f3265b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd99c013010 tx=0x7fd99c004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7269688402f57fc2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.295+0000 7fd98ffff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd98ffff640 1 -- 172.21.15.17:0/3196604944 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c02a070 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd99c009e10 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd98ffff640 1 -- 172.21.15.17:0/3196604944 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd99c019d50 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd98ffff640 1 -- 172.21.15.17:0/3196604944 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd99c019820 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 10 client.17271 fetch_fsmap learned FSMap version 34 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55929f23b720 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd98ffff640 1 -- 172.21.15.17:0/3196604944 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd99c009e10 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd98ffff640 1 -- 172.21.15.17:0/3196604944 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 34) v1 ==== 7315+0+0 (secure 0 0 0) 0x7fd99c033420 con 0x55929f313d70 2024-02-14T15:15:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 10 client.17271 fetch_fsmap finished waiting for FSMap version 34 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 20 client.17271 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f3887e0 0x55929f38ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 4 client.17271 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55929efb7550 con 0x55929f3887e0 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.296+0000 7fd9b480b640 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f3887e0 0x55929f38ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.297+0000 7fd98ffff640 10 client.17271 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.297+0000 7fd9b480b640 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f3887e0 0x55929f38ac00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=191120c702b9a9e3 server_cookie=aaafacf5bc4c5bbc in_seq=0 out_seq=0 2024-02-14T15:15:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd98ffff640 1 -- 172.21.15.17:0/3196604944 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55929efb7550 con 0x55929f3887e0 2024-02-14T15:15:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd98ffff640 10 client.17271 handle_command_reply: tid=0 2024-02-14T15:15:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 client.17271 shutdown 2024-02-14T15:15:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 msgr2=0x55929f3265b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f3265b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd99c013010 tx=0x7fd99c004650 comp rx=0 tx=0).stop 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 shutdown_connections 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55929f3887e0 0x55929f38ac00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55929f313d70 0x55929f3265b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55929f321030 0x55929f326af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 --2- 172.21.15.17:0/3196604944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55929f313450 0x55929f328170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd98ffff640 0 client.17271 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.310+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 >> 172.21.15.17:0/3196604944 conn(0x55929f30de50 msgr2=0x55929f30c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 shutdown_connections 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b9641740 1 -- 172.21.15.17:0/3196604944 wait complete. 2024-02-14T15:15:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b9641740 20 client.17271 trim_cache size 0 max 16384 2024-02-14T15:15:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:15:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:15:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:15:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:15:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9a00a2220 msgr2=0x7fd9a00a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9a00a2220 0x7fd9a00a46e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005fd0 tx=0x7fd9a8009500 comp rx=0 tx=0).stop 2024-02-14T15:15:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 msgr2=0x7fd9b019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b019d690 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009230 tx=0x7fd9a402c650 comp rx=0 tx=0).stop 2024-02-14T15:15:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 shutdown_connections 2024-02-14T15:15:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd9a00a2220 0x7fd9a00a46e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9b01977a0 0x7fd9b0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9b0106530 0x7fd9b019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 --2- 172.21.15.17:0/1998008975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9b0101460 0x7fd9b019d690 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 >> 172.21.15.17:0/1998008975 conn(0x7fd9b00fccc0 msgr2=0x7fd9b00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 shutdown_connections 2024-02-14T15:15:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:16.311+0000 7fd9b6a6f640 1 -- 172.21.15.17:0/1998008975 wait complete. 2024-02-14T15:15:16.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:15 smithi050 ceph-mon[36512]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-02-14T15:15:16.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:15 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-02-14T15:15:16.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1226522 10.50 MB/sec execute 498 sec latency 22.714 ms 2024-02-14T15:15:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:16 smithi076 ceph-mon[37057]: pgmap v373: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 214 op/s 2024-02-14T15:15:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:16 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6836/349944697,v1:172.21.15.50:6837/349944697] up:boot 2024-02-14T15:15:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:16 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:15:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:16 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:15:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:16 smithi017 ceph-mon[32549]: pgmap v373: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 214 op/s 2024-02-14T15:15:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:16 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6836/349944697,v1:172.21.15.50:6837/349944697] up:boot 2024-02-14T15:15:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:16 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:15:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:16 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:15:17.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:16 smithi050 ceph-mon[36512]: pgmap v373: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 214 op/s 2024-02-14T15:15:17.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:16 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6836/349944697,v1:172.21.15.50:6837/349944697] up:boot 2024-02-14T15:15:17.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:16 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:15:17.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:16 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-14T15:15:17.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1228857 10.50 MB/sec execute 499 sec latency 15.852 ms 2024-02-14T15:15:18.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1229746 10.49 MB/sec execute 500 sec latency 43.757 ms 2024-02-14T15:15:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:18 smithi076 ceph-mon[37057]: pgmap v374: 129 pgs: 129 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 199 op/s 2024-02-14T15:15:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:18 smithi017 ceph-mon[32549]: pgmap v374: 129 pgs: 129 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 199 op/s 2024-02-14T15:15:19.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:18 smithi050 ceph-mon[36512]: pgmap v374: 129 pgs: 129 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 199 op/s 2024-02-14T15:15:19.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1232350 10.50 MB/sec execute 501 sec latency 15.865 ms 2024-02-14T15:15:20.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1233369 10.49 MB/sec execute 502 sec latency 19.976 ms 2024-02-14T15:15:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:20 smithi076 ceph-mon[37057]: pgmap v375: 129 pgs: 129 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 199 op/s 2024-02-14T15:15:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:20 smithi017 ceph-mon[32549]: pgmap v375: 129 pgs: 129 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 199 op/s 2024-02-14T15:15:21.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:20 smithi050 ceph-mon[36512]: pgmap v375: 129 pgs: 129 active+clean; 568 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 199 op/s 2024-02-14T15:15:21.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1235927 10.50 MB/sec execute 503 sec latency 16.473 ms 2024-02-14T15:15:22.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1236984 10.48 MB/sec execute 504 sec latency 17.444 ms 2024-02-14T15:15:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:22 smithi076 ceph-mon[37057]: pgmap v376: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 291 op/s 2024-02-14T15:15:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:22 smithi017 ceph-mon[32549]: pgmap v376: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 291 op/s 2024-02-14T15:15:23.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:22 smithi050 ceph-mon[36512]: pgmap v376: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 291 op/s 2024-02-14T15:15:23.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1238134 10.47 MB/sec execute 505 sec latency 77.865 ms 2024-02-14T15:15:24.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1239625 10.47 MB/sec execute 506 sec latency 101.585 ms 2024-02-14T15:15:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:24 smithi076 ceph-mon[37057]: pgmap v377: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2024-02-14T15:15:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:24 smithi017 ceph-mon[32549]: pgmap v377: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2024-02-14T15:15:25.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:24 smithi050 ceph-mon[36512]: pgmap v377: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2024-02-14T15:15:25.889 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1241132 10.47 MB/sec execute 507 sec latency 17.441 ms 2024-02-14T15:15:26.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1243248 10.47 MB/sec execute 508 sec latency 17.164 ms 2024-02-14T15:15:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:26 smithi076 ceph-mon[37057]: pgmap v378: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2024-02-14T15:15:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:26 smithi017 ceph-mon[32549]: pgmap v378: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2024-02-14T15:15:27.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:26 smithi050 ceph-mon[36512]: pgmap v378: 129 pgs: 129 active+clean; 580 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 171 op/s 2024-02-14T15:15:27.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1244834 10.46 MB/sec execute 509 sec latency 18.384 ms 2024-02-14T15:15:28.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1246633 10.47 MB/sec execute 510 sec latency 301.150 ms 2024-02-14T15:15:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:28 smithi076 ceph-mon[37057]: pgmap v379: 129 pgs: 129 active+clean; 591 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 260 op/s 2024-02-14T15:15:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:28 smithi017 ceph-mon[32549]: pgmap v379: 129 pgs: 129 active+clean; 591 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 260 op/s 2024-02-14T15:15:29.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:28 smithi050 ceph-mon[36512]: pgmap v379: 129 pgs: 129 active+clean; 591 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 260 op/s 2024-02-14T15:15:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.399+0000 7f9dbaef4640 1 -- 172.21.15.17:0/663877049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db40696f0 msgr2=0x7f9db4105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.399+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/663877049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db40696f0 0x7f9db4105e30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009930 tx=0x7f9da802bc30 comp rx=0 tx=0).stop 2024-02-14T15:15:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 -- 172.21.15.17:0/663877049 shutdown_connections 2024-02-14T15:15:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/663877049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db40696f0 0x7f9db4105e30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/663877049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9db4106510 0x7f9db410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/663877049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db4068d40 0x7f9db4069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 -- 172.21.15.17:0/663877049 >> 172.21.15.17:0/663877049 conn(0x7f9db4077220 msgr2=0x7f9db4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 -- 172.21.15.17:0/663877049 shutdown_connections 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 -- 172.21.15.17:0/663877049 wait complete. 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 Processor -- start 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.400+0000 7f9dbaef4640 1 -- start start 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9dbaef4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9db4068d40 0x7f9db419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9dbaef4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4106510 0x7f9db419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9dbaef4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db41976d0 0x7f9db4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9dbaef4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41981a0 con 0x7f9db4106510 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9dbaef4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4198310 con 0x7f9db4068d40 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9dbaef4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db41984d0 con 0x7f9db41976d0 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4106510 0x7f9db419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4106510 0x7f9db419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49988/0 (socket says 172.21.15.17:49988) 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 -- 172.21.15.17:0/3665937055 learned_addr learned my addr 172.21.15.17:0/3665937055 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db9491640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db41976d0 0x7f9db4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db8c90640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9db4068d40 0x7f9db419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 -- 172.21.15.17:0/3665937055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db41976d0 msgr2=0x7f9db4197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db41976d0 0x7f9db4197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 -- 172.21.15.17:0/3665937055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9db4068d40 msgr2=0x7f9db419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9db4068d40 0x7f9db419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 -- 172.21.15.17:0/3665937055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da8009560 con 0x7f9db4106510 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db3fff640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4106510 0x7f9db419db50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009230 tx=0x7f9da8004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db1ffb640 1 -- 172.21.15.17:0/3665937055 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da80043d0 con 0x7f9db4106510 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db41a4410 con 0x7f9db4106510 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db1ffb640 1 -- 172.21.15.17:0/3665937055 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9da80052d0 con 0x7f9db4106510 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.401+0000 7f9db1ffb640 1 -- 172.21.15.17:0/3665937055 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8003d80 con 0x7f9db4106510 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.402+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db41a4940 con 0x7f9db4106510 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db1ffb640 1 -- 172.21.15.17:0/3665937055 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9da8005440 con 0x7f9db4106510 2024-02-14T15:15:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 Processor -- start 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 -- start start 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d891b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d892040 0x55c39d89a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d89d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d7ff9e0 con 0x55c39d892040 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d7fcf10 con 0x55c39d89ac20 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d7f86f0 con 0x55c39d891720 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db1ffb640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d9c0a2220 0x7f9d9c0a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db8c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d892040 0x55c39d89a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db8c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d892040 0x55c39d89a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49994/0 (socket says 172.21.15.17:49994) 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db8c90640 1 -- 172.21.15.17:0/2042797766 learned_addr learned my addr 172.21.15.17:0/2042797766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db9491640 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d891b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db9491640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d9c0a2220 0x7f9d9c0a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.404+0000 7f9db3fff640 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d89d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db1ffb640 1 -- 172.21.15.17:0/3665937055 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9da80f04c0 con 0x7f9db4106510 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db8c90640 1 -- 172.21.15.17:0/2042797766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 msgr2=0x55c39d891b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db8c90640 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d891b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db8c90640 1 -- 172.21.15.17:0/2042797766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 msgr2=0x55c39d89d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db8c90640 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d89d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db8c90640 1 -- 172.21.15.17:0/2042797766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c39d509920 con 0x55c39d892040 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db8c90640 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d892040 0x55c39d89a6e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009c90 tx=0x7f9da40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9cb2a788ba0ace server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9db9491640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d9c0a2220 0x7f9d9c0a46e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009990 tx=0x7f9dac0096a0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9d9affd640 1 -- 172.21.15.17:0/2042797766 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4009760 con 0x55c39d892040 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9d9affd640 1 -- 172.21.15.17:0/2042797766 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9da4012930 con 0x55c39d892040 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.405+0000 7f9d9affd640 1 -- 172.21.15.17:0/2042797766 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400fab0 con 0x55c39d892040 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2042797766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d892040 msgr2=0x55c39d89a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d892040 0x55c39d89a6e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009c90 tx=0x7f9da40117b0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2042797766 shutdown_connections 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d892040 0x55c39d89a6e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d89d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2042797766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d891b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2042797766 >> 172.21.15.17:0/2042797766 conn(0x55c39d88be50 msgr2=0x55c39d88c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2042797766 shutdown_connections 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2042797766 wait complete. 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 -- start start 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.406+0000 7f9dbdb95740 1 Processor -- start 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d89a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d897960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d89ac20 0x55c39d897f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d794fa0 con 0x55c39d89ac20 2024-02-14T15:15:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d4cc250 con 0x55c39d892040 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d77e200 con 0x55c39d891720 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9dbdb95740 10 client.0 _ll_register_callbacks cb 0x55c39d7edb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d897960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d89ac20 0x55c39d897f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db8c90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d89a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d89ac20 0x55c39d897f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50008/0 (socket says 172.21.15.17:50008) 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d897960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:54386/0 (socket says 172.21.15.17:54386) 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 -- 172.21.15.17:0/1388377068 learned_addr learned my addr 172.21.15.17:0/1388377068 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 -- 172.21.15.17:0/1388377068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 msgr2=0x55c39d89a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d89a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 -- 172.21.15.17:0/1388377068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 msgr2=0x55c39d897960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d897960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 -- 172.21.15.17:0/1388377068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da40098c0 con 0x55c39d89ac20 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db8c90640 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d89a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9db9491640 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d89ac20 0x55c39d897f00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9dac003cf0 tx=0x7f9dac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d0df273a9480a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.407+0000 7f9d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9d93fff640 1 -- 172.21.15.17:0/1388377068 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac003d20 con 0x55c39d89ac20 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9d93fff640 1 -- 172.21.15.17:0/1388377068 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9dac004040 con 0x55c39d89ac20 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c39d4f8600 con 0x55c39d89ac20 2024-02-14T15:15:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9d93fff640 1 -- 172.21.15.17:0/1388377068 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dac003e80 con 0x55c39d89ac20 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 10 client.17283 fetch_fsmap learned FSMap version 35 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c39d7f6d70 con 0x55c39d89ac20 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9d93fff640 1 -- 172.21.15.17:0/1388377068 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9dac032020 con 0x55c39d89ac20 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9d93fff640 1 -- 172.21.15.17:0/1388377068 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9dac0107b0 con 0x55c39d89ac20 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 10 client.17283 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 20 client.17283 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d8aaa90 0x55c39d8aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 4 client.17283 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:15:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.408+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c39d5491b0 con 0x55c39d8aaa90 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.409+0000 7f9db8c90640 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d8aaa90 0x55c39d8aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.409+0000 7f9d93fff640 10 client.17283 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.409+0000 7f9db8c90640 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d8aaa90 0x55c39d8aceb0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd7e66cdb53253e server_cookie=2e7470df036b7069 in_seq=0 out_seq=0 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9d93fff640 1 -- 172.21.15.17:0/1388377068 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c39d5491b0 con 0x55c39d8aaa90 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9d93fff640 10 client.17283 handle_command_reply: tid=0 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 client.17283 shutdown 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d89ac20 msgr2=0x55c39d897f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d89ac20 0x55c39d897f00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9dac003cf0 tx=0x7f9dac004a40 comp rx=0 tx=0).stop 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 shutdown_connections 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d8aaa90 0x55c39d8aceb0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d89ac20 0x55c39d897f00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d897960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9d93fff640 0 client.17283 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/1388377068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d891720 0x55c39d89a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 >> 172.21.15.17:0/1388377068 conn(0x55c39d88be50 msgr2=0x55c39d88dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 shutdown_connections 2024-02-14T15:15:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 1 -- 172.21.15.17:0/1388377068 wait complete. 2024-02-14T15:15:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.410+0000 7f9dbdb95740 20 client.17283 trim_cache size 0 max 16384 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 Processor -- start 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 -- start start 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d89a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d89a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d89ac20 0x55c39d896230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d890ec0 con 0x55c39d891720 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d8a6e10 con 0x55c39d892040 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.423+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d898500 con 0x55c39d89ac20 2024-02-14T15:15:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d89a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d89a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50010/0 (socket says 172.21.15.17:50010) 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 -- 172.21.15.17:0/2659191073 learned_addr learned my addr 172.21.15.17:0/2659191073 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db9491640 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d89ac20 0x55c39d896230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db3fff640 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d89a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 -- 172.21.15.17:0/2659191073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d89ac20 msgr2=0x55c39d896230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d89ac20 0x55c39d896230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 -- 172.21.15.17:0/2659191073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 msgr2=0x55c39d89a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d89a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 -- 172.21.15.17:0/2659191073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c39d793820 con 0x55c39d891720 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9db8c90640 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d89a090 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009e60 tx=0x7f9da4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7043c23f2038d82 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9d9a7fc640 1 -- 172.21.15.17:0/2659191073 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4012760 con 0x55c39d891720 2024-02-14T15:15:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9d9a7fc640 1 -- 172.21.15.17:0/2659191073 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9da4029070 con 0x55c39d891720 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.424+0000 7f9d9a7fc640 1 -- 172.21.15.17:0/2659191073 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400f9c0 con 0x55c39d891720 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2659191073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 msgr2=0x55c39d89a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d89a090 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009e60 tx=0x7f9da4011d60 comp rx=0 tx=0).stop 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2659191073 shutdown_connections 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d89ac20 0x55c39d896230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d892040 0x55c39d89a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/2659191073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d89a090 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2659191073 >> 172.21.15.17:0/2659191073 conn(0x55c39d88be50 msgr2=0x55c39d8b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2659191073 shutdown_connections 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 -- 172.21.15.17:0/2659191073 wait complete. 2024-02-14T15:15:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 -- start start 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.425+0000 7f9dbdb95740 1 Processor -- start 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d8a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d892040 0x55c39d8a2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d8a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d8a0770 con 0x55c39d891720 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d8a08e0 con 0x55c39d89ac20 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d8a0aa0 con 0x55c39d892040 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 10 client.0 _ll_register_callbacks cb 0x7f9da40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d8a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d8a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50012/0 (socket says 172.21.15.17:50012) 2024-02-14T15:15:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 -- 172.21.15.17:0/4158103665 learned_addr learned my addr 172.21.15.17:0/4158103665 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db3fff640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d892040 0x55c39d8a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db9491640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d8a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 -- 172.21.15.17:0/4158103665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d892040 msgr2=0x55c39d8a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d892040 0x55c39d8a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 -- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 msgr2=0x55c39d8a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d8a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 -- 172.21.15.17:0/4158103665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c39d509920 con 0x55c39d891720 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db3fff640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d892040 0x55c39d8a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db9491640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d8a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9db8c90640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d8a1ac0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9da401c040 tx=0x7f9da40299f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc18a39efadc7717 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9d93fff640 1 -- 172.21.15.17:0/4158103665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4012760 con 0x55c39d891720 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9d93fff640 1 -- 172.21.15.17:0/4158103665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9da402d070 con 0x55c39d891720 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.426+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9da4012930 con 0x55c39d891720 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9d93fff640 1 -- 172.21.15.17:0/4158103665 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9da40291e0 con 0x55c39d891720 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 10 client.17289 fetch_fsmap learned FSMap version 35 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c39d7b9720 con 0x55c39d891720 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9d93fff640 1 -- 172.21.15.17:0/4158103665 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9da4012930 con 0x55c39d891720 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9d93fff640 1 -- 172.21.15.17:0/4158103665 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9da4022040 con 0x55c39d891720 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 10 client.17289 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 20 client.17289 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d902060 0x55c39d904480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 4 client.17289 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c39d535550 con 0x55c39d902060 2024-02-14T15:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.427+0000 7f9db3fff640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d902060 0x55c39d904480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.428+0000 7f9db3fff640 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d902060 0x55c39d904480 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3325988f85f03fbe server_cookie=bf6af6a891269964 in_seq=0 out_seq=0 2024-02-14T15:15:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.428+0000 7f9d93fff640 10 client.17289 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9d93fff640 1 -- 172.21.15.17:0/4158103665 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c39d535550 con 0x55c39d902060 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9d93fff640 10 client.17289 handle_command_reply: tid=0 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 client.17289 shutdown 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 msgr2=0x55c39d8a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d8a1ac0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9da401c040 tx=0x7f9da40299f0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 shutdown_connections 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c39d902060 0x55c39d904480 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c39d89ac20 0x55c39d8a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c39d892040 0x55c39d8a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9d93fff640 0 client.17289 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 --2- 172.21.15.17:0/4158103665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c39d891720 0x55c39d8a1ac0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 >> 172.21.15.17:0/4158103665 conn(0x55c39d88be50 msgr2=0x55c39d8961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 shutdown_connections 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 1 -- 172.21.15.17:0/4158103665 wait complete. 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.440+0000 7f9dbdb95740 20 client.17289 trim_cache size 0 max 16384 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:15:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:15:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:15:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d9c0a2220 msgr2=0x7f9d9c0a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d9c0a2220 0x7f9d9c0a46e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009990 tx=0x7f9dac0096a0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4106510 msgr2=0x7f9db419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4106510 0x7f9db419db50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009230 tx=0x7f9da8004c00 comp rx=0 tx=0).stop 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 shutdown_connections 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9d9c0a2220 0x7f9d9c0a46e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4106510 0x7f9db419db50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9db41976d0 0x7f9db4197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 --2- 172.21.15.17:0/3665937055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9db4068d40 0x7f9db419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 >> 172.21.15.17:0/3665937055 conn(0x7f9db4077220 msgr2=0x7f9db40fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 shutdown_connections 2024-02-14T15:15:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:29.441+0000 7f9dbaef4640 1 -- 172.21.15.17:0/3665937055 wait complete. 2024-02-14T15:15:29.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1247460 10.45 MB/sec execute 511 sec latency 395.648 ms 2024-02-14T15:15:30.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1249935 10.46 MB/sec execute 512 sec latency 16.314 ms 2024-02-14T15:15:31.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:30 smithi076 ceph-mon[37057]: pgmap v380: 129 pgs: 129 active+clean; 591 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 173 op/s 2024-02-14T15:15:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:30 smithi017 ceph-mon[32549]: pgmap v380: 129 pgs: 129 active+clean; 591 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 173 op/s 2024-02-14T15:15:31.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:30 smithi050 ceph-mon[36512]: pgmap v380: 129 pgs: 129 active+clean; 591 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 173 op/s 2024-02-14T15:15:31.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1250877 10.45 MB/sec execute 513 sec latency 15.858 ms 2024-02-14T15:15:32.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1253625 10.46 MB/sec execute 514 sec latency 13.047 ms 2024-02-14T15:15:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:32 smithi076 ceph-mon[37057]: pgmap v381: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 255 op/s 2024-02-14T15:15:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:32 smithi017 ceph-mon[32549]: pgmap v381: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 255 op/s 2024-02-14T15:15:33.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:32 smithi050 ceph-mon[36512]: pgmap v381: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 255 op/s 2024-02-14T15:15:33.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1254424 10.44 MB/sec execute 515 sec latency 100.555 ms 2024-02-14T15:15:34.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1256564 10.44 MB/sec execute 516 sec latency 63.466 ms 2024-02-14T15:15:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:34 smithi076 ceph-mon[37057]: pgmap v382: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 170 op/s 2024-02-14T15:15:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:34 smithi017 ceph-mon[32549]: pgmap v382: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 170 op/s 2024-02-14T15:15:35.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:34 smithi050 ceph-mon[36512]: pgmap v382: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 170 op/s 2024-02-14T15:15:35.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1257895 10.44 MB/sec execute 517 sec latency 23.541 ms 2024-02-14T15:15:36.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1260180 10.44 MB/sec execute 518 sec latency 15.590 ms 2024-02-14T15:15:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:36 smithi076 ceph-mon[37057]: pgmap v383: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 170 op/s 2024-02-14T15:15:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:36 smithi017 ceph-mon[32549]: pgmap v383: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 170 op/s 2024-02-14T15:15:37.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:36 smithi050 ceph-mon[36512]: pgmap v383: 129 pgs: 129 active+clean; 604 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 170 op/s 2024-02-14T15:15:37.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1261545 10.44 MB/sec execute 519 sec latency 20.512 ms 2024-02-14T15:15:38.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1264110 10.44 MB/sec execute 520 sec latency 51.211 ms 2024-02-14T15:15:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:38 smithi076 ceph-mon[37057]: pgmap v384: 129 pgs: 129 active+clean; 616 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-02-14T15:15:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:38 smithi017 ceph-mon[32549]: pgmap v384: 129 pgs: 129 active+clean; 616 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-02-14T15:15:39.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:38 smithi050 ceph-mon[36512]: pgmap v384: 129 pgs: 129 active+clean; 616 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 245 op/s 2024-02-14T15:15:39.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1264856 10.43 MB/sec execute 521 sec latency 62.171 ms 2024-02-14T15:15:40.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1266881 10.43 MB/sec execute 522 sec latency 32.497 ms 2024-02-14T15:15:41.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:40 smithi076 ceph-mon[37057]: pgmap v385: 129 pgs: 129 active+clean; 616 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 156 op/s 2024-02-14T15:15:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:40 smithi017 ceph-mon[32549]: pgmap v385: 129 pgs: 129 active+clean; 616 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 156 op/s 2024-02-14T15:15:41.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:40 smithi050 ceph-mon[36512]: pgmap v385: 129 pgs: 129 active+clean; 616 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 156 op/s 2024-02-14T15:15:41.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1268406 10.43 MB/sec execute 523 sec latency 21.849 ms 2024-02-14T15:15:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 -- 172.21.15.17:0/1704107155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300101e30 msgr2=0x7fd300105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 --2- 172.21.15.17:0/1704107155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300101e30 0x7fd300105e70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009930 tx=0x7fd2f002bc30 comp rx=0 tx=0).stop 2024-02-14T15:15:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 -- 172.21.15.17:0/1704107155 shutdown_connections 2024-02-14T15:15:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 --2- 172.21.15.17:0/1704107155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300101e30 0x7fd300105e70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 --2- 172.21.15.17:0/1704107155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd300106550 0x7fd30010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 --2- 172.21.15.17:0/1704107155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd300101480 0x7fd300101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 -- 172.21.15.17:0/1704107155 >> 172.21.15.17:0/1704107155 conn(0x7fd3000fcd20 msgr2=0x7fd3000ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 -- 172.21.15.17:0/1704107155 shutdown_connections 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.538+0000 7fd30845b640 1 -- 172.21.15.17:0/1704107155 wait complete. 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 Processor -- start 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 -- start start 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd300101480 0x7fd30019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300106550 0x7fd30019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3001976e0 0x7fd300197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001981b0 con 0x7fd300106550 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300198320 con 0x7fd300101480 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd30845b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001984e0 con 0x7fd3001976e0 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd3059f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300106550 0x7fd30019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd3059f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300106550 0x7fd30019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35800/0 (socket says 172.21.15.17:35800) 2024-02-14T15:15:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd3059f6640 1 -- 172.21.15.17:0/3429483894 learned_addr learned my addr 172.21.15.17:0/3429483894 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd3061f7640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd300101480 0x7fd30019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.539+0000 7fd3069f8640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3001976e0 0x7fd300197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd3059f6640 1 -- 172.21.15.17:0/3429483894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3001976e0 msgr2=0x7fd300197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd3059f6640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3001976e0 0x7fd300197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd3059f6640 1 -- 172.21.15.17:0/3429483894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd300101480 msgr2=0x7fd30019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd3059f6640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd300101480 0x7fd30019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd3059f6640 1 -- 172.21.15.17:0/3429483894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f0009560 con 0x7fd300106550 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd3061f7640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd300101480 0x7fd30019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd3059f6640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300106550 0x7fd30019db60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0005ab0 tx=0x7fd2f002c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd2f77fe640 1 -- 172.21.15.17:0/3429483894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00042d0 con 0x7fd300106550 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd2f77fe640 1 -- 172.21.15.17:0/3429483894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2f0004430 con 0x7fd300106550 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3001a4420 con 0x7fd300106550 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd2f77fe640 1 -- 172.21.15.17:0/3429483894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0003bf0 con 0x7fd300106550 2024-02-14T15:15:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.540+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3001a4950 con 0x7fd300106550 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.542+0000 7fd2f77fe640 1 -- 172.21.15.17:0/3429483894 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd2f0003d50 con 0x7fd300106550 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 Processor -- start 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd2f77fe640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd2ec0a22a0 0x7fd2ec0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd2f77fe640 1 -- 172.21.15.17:0/3429483894 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd2f00f0920 con 0x7fd300106550 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 -- start start 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd3069f8640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd2ec0a22a0 0x7fd2ec0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d331b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d33a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d33d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d29f9e0 con 0x559a7d331720 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d29cf10 con 0x559a7d33ac20 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.543+0000 7fd30b020740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d2986f0 con 0x559a7d332040 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d331b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3059f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d33a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d331b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35812/0 (socket says 172.21.15.17:35812) 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3061f7640 1 -- 172.21.15.17:0/158501793 learned_addr learned my addr 172.21.15.17:0/158501793 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3069f8640 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d33d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3069f8640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd2ec0a22a0 0x7fd2ec0a4760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005fd0 tx=0x7fd2f8009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3059f6640 1 -- 172.21.15.17:0/158501793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 msgr2=0x559a7d33d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3059f6640 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d33d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3059f6640 1 -- 172.21.15.17:0/158501793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 msgr2=0x559a7d331b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3059f6640 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d331b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3059f6640 1 -- 172.21.15.17:0/158501793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a7cfa9920 con 0x559a7d332040 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3061f7640 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d331b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd3059f6640 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d33a6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00eddc0 tx=0x7fd2f0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8879b6583b8014f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd2f4bfe640 1 -- 172.21.15.17:0/158501793 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00c12d0 con 0x559a7d332040 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd2f4bfe640 1 -- 172.21.15.17:0/158501793 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2f00c5920 con 0x559a7d332040 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.544+0000 7fd2f4bfe640 1 -- 172.21.15.17:0/158501793 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00ee080 con 0x559a7d332040 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 -- 172.21.15.17:0/158501793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 msgr2=0x559a7d33a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d33a6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00eddc0 tx=0x7fd2f0003680 comp rx=0 tx=0).stop 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 -- 172.21.15.17:0/158501793 shutdown_connections 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d33d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d33a6e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 --2- 172.21.15.17:0/158501793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d331b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 -- 172.21.15.17:0/158501793 >> 172.21.15.17:0/158501793 conn(0x559a7d32be50 msgr2=0x559a7d32c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 -- 172.21.15.17:0/158501793 shutdown_connections 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 -- 172.21.15.17:0/158501793 wait complete. 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.545+0000 7fd30b020740 1 -- start start 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 1 Processor -- start 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d331720 0x559a7d33a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d332040 0x559a7d337990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d337fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d234fa0 con 0x559a7d332040 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7cf6c250 con 0x559a7d331720 2024-02-14T15:15:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d21e200 con 0x559a7d33ac20 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd30b020740 10 client.0 _ll_register_callbacks cb 0x559a7d28db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d332040 0x559a7d337990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d331720 0x559a7d33a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3069f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d337fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d332040 0x559a7d337990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35814/0 (socket says 172.21.15.17:35814) 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 -- 172.21.15.17:0/3987973393 learned_addr learned my addr 172.21.15.17:0/3987973393 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d331720 0x559a7d33a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:43078/0 (socket says 172.21.15.17:43078) 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 -- 172.21.15.17:0/3987973393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 msgr2=0x559a7d337fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d337fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 -- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d331720 msgr2=0x559a7d33a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d331720 0x559a7d33a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 -- 172.21.15.17:0/3987973393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3001a4950 con 0x559a7d332040 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3069f8640 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d337fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3061f7640 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d331720 0x559a7d33a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd3059f6640 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d332040 0x559a7d337990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a675156207e3a3f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.546+0000 7fd2e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/3987973393 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00ee100 con 0x559a7d332040 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/3987973393 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd2f00c12d0 con 0x559a7d332040 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a7cf98600 con 0x559a7d332040 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/3987973393 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f00b97f0 con 0x559a7d332040 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 10 client.17301 fetch_fsmap learned FSMap version 35 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559a7d296d70 con 0x559a7d332040 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/3987973393 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2f00c9080 con 0x559a7d332040 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/3987973393 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2f003c070 con 0x559a7d332040 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 10 client.17301 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 20 client.17301 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d34a3c0 0x559a7d34c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 4 client.17301 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a7cfe91b0 con 0x559a7d34a3c0 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.547+0000 7fd3061f7640 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d34a3c0 0x559a7d34c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd2e9ffb640 10 client.17301 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd3061f7640 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d34a3c0 0x559a7d34c7b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f599a1549c0e2b5 server_cookie=65b53b0da39caaf9 in_seq=0 out_seq=0 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/3987973393 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a7cfe91b0 con 0x559a7d34a3c0 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd2e9ffb640 10 client.17301 handle_command_reply: tid=0 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd30b020740 1 client.17301 shutdown 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d332040 msgr2=0x559a7d337990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd30b020740 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d332040 0x559a7d337990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f0009260 comp rx=0 tx=0).stop 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.548+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 shutdown_connections 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d34a3c0 0x559a7d34c7b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d332040 0x559a7d337990 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd2e9ffb640 0 client.17301 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d337fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 1 --2- 172.21.15.17:0/3987973393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d331720 0x559a7d33a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 >> 172.21.15.17:0/3987973393 conn(0x559a7d32be50 msgr2=0x559a7d32dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 shutdown_connections 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 1 -- 172.21.15.17:0/3987973393 wait complete. 2024-02-14T15:15:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.549+0000 7fd30b020740 20 client.17301 trim_cache size 0 max 16384 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 Processor -- start 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 -- start start 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d337930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d332040 0x559a7d33a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d33a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d3270b0 con 0x559a7d331720 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d32ce50 con 0x559a7d332040 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd30b020740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d351960 con 0x559a7d33ac20 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d337930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d337930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35824/0 (socket says 172.21.15.17:35824) 2024-02-14T15:15:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd3061f7640 1 -- 172.21.15.17:0/2553154121 learned_addr learned my addr 172.21.15.17:0/2553154121 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd3069f8640 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d33a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.562+0000 7fd3059f6640 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d332040 0x559a7d33a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd3061f7640 1 -- 172.21.15.17:0/2553154121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 msgr2=0x559a7d33a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd3061f7640 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d33a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd3061f7640 1 -- 172.21.15.17:0/2553154121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d332040 msgr2=0x559a7d33a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd3061f7640 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d332040 0x559a7d33a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd3061f7640 1 -- 172.21.15.17:0/2553154121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a7d233820 con 0x559a7d331720 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd3061f7640 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d337930 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x559a7d34d060 tx=0x7fd2fc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7214b1c890f6bab8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd2ebfff640 1 -- 172.21.15.17:0/2553154121 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc017040 con 0x559a7d331720 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd2ebfff640 1 -- 172.21.15.17:0/2553154121 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2fc012e90 con 0x559a7d331720 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.563+0000 7fd2ebfff640 1 -- 172.21.15.17:0/2553154121 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc004bc0 con 0x559a7d331720 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- 172.21.15.17:0/2553154121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 msgr2=0x559a7d337930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d337930 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x559a7d34d060 tx=0x7fd2fc005f90 comp rx=0 tx=0).stop 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- 172.21.15.17:0/2553154121 shutdown_connections 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d33ac20 0x559a7d33a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d332040 0x559a7d33a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 --2- 172.21.15.17:0/2553154121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d337930 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- 172.21.15.17:0/2553154121 >> 172.21.15.17:0/2553154121 conn(0x559a7d32be50 msgr2=0x559a7d337220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- 172.21.15.17:0/2553154121 shutdown_connections 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- 172.21.15.17:0/2553154121 wait complete. 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- start start 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 Processor -- start 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d340520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d340a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d340fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d3a4c50 con 0x559a7d331720 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d3a4d90 con 0x559a7d33ac20 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7d3a4f50 con 0x559a7d332040 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd30b020740 10 client.0 _ll_register_callbacks cb 0x7fd2f00b9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d340520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd3061f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d340520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35840/0 (socket says 172.21.15.17:35840) 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd3059f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d340a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd3061f7640 1 -- 172.21.15.17:0/4176782444 learned_addr learned my addr 172.21.15.17:0/4176782444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.564+0000 7fd3069f8640 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d340fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd3059f6640 1 -- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 msgr2=0x559a7d340fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd3059f6640 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d340fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd3059f6640 1 -- 172.21.15.17:0/4176782444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 msgr2=0x559a7d340520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd3059f6640 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d340520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd3059f6640 1 -- 172.21.15.17:0/4176782444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc012b00 con 0x559a7d332040 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd3059f6640 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d340a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x559a7d338f60 tx=0x7fd2f0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8e2e4a82c9dffa server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd2e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/4176782444 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00c12d0 con 0x559a7d332040 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/4176782444 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd2f00c5b70 con 0x559a7d332040 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2f00c5920 con 0x559a7d332040 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/4176782444 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f00bc620 con 0x559a7d332040 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd30b020740 10 client.26818 fetch_fsmap learned FSMap version 35 2024-02-14T15:15:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559a7d259720 con 0x559a7d332040 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/4176782444 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2f00d0030 con 0x559a7d332040 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.565+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/4176782444 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2f003c040 con 0x559a7d332040 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd30b020740 10 client.26818 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd30b020740 10 client.26818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd30b020740 20 client.26818 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd30b020740 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d3a6250 0x559a7d3a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd30b020740 4 client.26818 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559a7cfd5550 con 0x559a7d3a6250 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd3061f7640 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d3a6250 0x559a7d3a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd3061f7640 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d3a6250 0x559a7d3a8640 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ae11ace560098d server_cookie=23c59ed877695c13 in_seq=0 out_seq=0 2024-02-14T15:15:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.566+0000 7fd2e9ffb640 10 client.26818 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd2e9ffb640 1 -- 172.21.15.17:0/4176782444 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559a7cfd5550 con 0x559a7d3a6250 2024-02-14T15:15:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd2e9ffb640 10 client.26818 handle_command_reply: tid=0 2024-02-14T15:15:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 client.26818 shutdown 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 msgr2=0x559a7d340a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d340a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x559a7d338f60 tx=0x7fd2f0003680 comp rx=0 tx=0).stop 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 shutdown_connections 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x559a7d3a6250 0x559a7d3a8640 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559a7d33ac20 0x559a7d340fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd2e9ffb640 0 client.26818 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a7d332040 0x559a7d340a60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 --2- 172.21.15.17:0/4176782444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a7d331720 0x559a7d340520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.578+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 >> 172.21.15.17:0/4176782444 conn(0x559a7d32be50 msgr2=0x559a7d32d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 shutdown_connections 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30b020740 1 -- 172.21.15.17:0/4176782444 wait complete. 2024-02-14T15:15:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30b020740 20 client.26818 trim_cache size 0 max 16384 2024-02-14T15:15:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:15:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:15:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:15:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd2ec0a22a0 msgr2=0x7fd2ec0a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd2ec0a22a0 0x7fd2ec0a4760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005fd0 tx=0x7fd2f8009500 comp rx=0 tx=0).stop 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300106550 msgr2=0x7fd30019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300106550 0x7fd30019db60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0005ab0 tx=0x7fd2f002c5f0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 shutdown_connections 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd2ec0a22a0 0x7fd2ec0a4760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd300106550 0x7fd30019db60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3001976e0 0x7fd300197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 --2- 172.21.15.17:0/3429483894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd300101480 0x7fd30019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 >> 172.21.15.17:0/3429483894 conn(0x7fd3000fcd20 msgr2=0x7fd3000ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 shutdown_connections 2024-02-14T15:15:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:42.579+0000 7fd30845b640 1 -- 172.21.15.17:0/3429483894 wait complete. 2024-02-14T15:15:42.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1270450 10.42 MB/sec execute 524 sec latency 30.809 ms 2024-02-14T15:15:43.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:42 smithi076 ceph-mon[37057]: pgmap v386: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 242 op/s 2024-02-14T15:15:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:42 smithi017 ceph-mon[32549]: pgmap v386: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 242 op/s 2024-02-14T15:15:43.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:42 smithi050 ceph-mon[36512]: pgmap v386: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 242 op/s 2024-02-14T15:15:43.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1271946 10.42 MB/sec execute 525 sec latency 22.337 ms 2024-02-14T15:15:44.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1273479 10.42 MB/sec execute 526 sec latency 53.917 ms 2024-02-14T15:15:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:44 smithi076 ceph-mon[37057]: pgmap v387: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 161 op/s 2024-02-14T15:15:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:44 smithi017 ceph-mon[32549]: pgmap v387: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 161 op/s 2024-02-14T15:15:45.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:44 smithi050 ceph-mon[36512]: pgmap v387: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 161 op/s 2024-02-14T15:15:45.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1275074 10.42 MB/sec execute 527 sec latency 208.800 ms 2024-02-14T15:15:46.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:45 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:15:46.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:45 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:15:46.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:45 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:15:46.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:45 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:15:46.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:45 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:15:46.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:45 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:15:46.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:45 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:15:46.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:45 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:15:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:45 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:15:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:45 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:15:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:45 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:15:46.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:45 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:15:46.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1276586 10.41 MB/sec execute 528 sec latency 27.055 ms 2024-02-14T15:15:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:46 smithi076 ceph-mon[37057]: pgmap v388: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 161 op/s 2024-02-14T15:15:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:46 smithi017 ceph-mon[32549]: pgmap v388: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 161 op/s 2024-02-14T15:15:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:46 smithi050 ceph-mon[36512]: pgmap v388: 129 pgs: 129 active+clean; 627 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 161 op/s 2024-02-14T15:15:47.890 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1278720 10.42 MB/sec execute 529 sec latency 18.727 ms 2024-02-14T15:15:48.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1280340 10.41 MB/sec execute 530 sec latency 37.151 ms 2024-02-14T15:15:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:48 smithi076 ceph-mon[37057]: pgmap v389: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 244 op/s 2024-02-14T15:15:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:48 smithi017 ceph-mon[32549]: pgmap v389: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 244 op/s 2024-02-14T15:15:49.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:48 smithi050 ceph-mon[36512]: pgmap v389: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.0 MiB/s wr, 244 op/s 2024-02-14T15:15:49.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1282098 10.41 MB/sec execute 531 sec latency 122.577 ms 2024-02-14T15:15:50.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1283287 10.40 MB/sec execute 532 sec latency 34.963 ms 2024-02-14T15:15:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:50 smithi076 ceph-mon[37057]: pgmap v390: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 169 op/s 2024-02-14T15:15:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:50 smithi017 ceph-mon[32549]: pgmap v390: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 169 op/s 2024-02-14T15:15:51.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:50 smithi050 ceph-mon[36512]: pgmap v390: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 169 op/s 2024-02-14T15:15:51.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1285634 10.41 MB/sec execute 533 sec latency 14.142 ms 2024-02-14T15:15:52.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1286783 10.40 MB/sec execute 534 sec latency 17.094 ms 2024-02-14T15:15:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:52 smithi076 ceph-mon[37057]: pgmap v391: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-02-14T15:15:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:52 smithi017 ceph-mon[32549]: pgmap v391: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-02-14T15:15:53.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:52 smithi050 ceph-mon[36512]: pgmap v391: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 253 op/s 2024-02-14T15:15:53.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1289214 10.41 MB/sec execute 535 sec latency 12.346 ms 2024-02-14T15:15:54.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1290006 10.39 MB/sec execute 536 sec latency 49.755 ms 2024-02-14T15:15:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:54 smithi076 ceph-mon[37057]: pgmap v392: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 167 op/s 2024-02-14T15:15:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:54 smithi017 ceph-mon[32549]: pgmap v392: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 167 op/s 2024-02-14T15:15:55.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:54 smithi050 ceph-mon[36512]: pgmap v392: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 167 op/s 2024-02-14T15:15:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 -- 172.21.15.17:0/1106155981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100cf0 msgr2=0x7f7a38104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 --2- 172.21.15.17:0/1106155981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100cf0 0x7f7a38104d30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009930 tx=0x7f7a2802bc30 comp rx=0 tx=0).stop 2024-02-14T15:15:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 -- 172.21.15.17:0/1106155981 shutdown_connections 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 --2- 172.21.15.17:0/1106155981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100cf0 0x7f7a38104d30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 --2- 172.21.15.17:0/1106155981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38105410 0x7f7a3810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 --2- 172.21.15.17:0/1106155981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a38100340 0x7f7a38100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 -- 172.21.15.17:0/1106155981 >> 172.21.15.17:0/1106155981 conn(0x7f7a38077220 msgr2=0x7f7a38077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 -- 172.21.15.17:0/1106155981 shutdown_connections 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.674+0000 7f7a40702640 1 -- 172.21.15.17:0/1106155981 wait complete. 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 Processor -- start 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 -- start start 2024-02-14T15:15:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100340 0x7f7a3819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38105410 0x7f7a3819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a381965b0 0x7f7a38196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38196fa0 con 0x7f7a38100340 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38197110 con 0x7f7a38105410 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a40702640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a381972d0 con 0x7f7a381965b0 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100340 0x7f7a3819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100340 0x7f7a3819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36952/0 (socket says 172.21.15.17:36952) 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 -- 172.21.15.17:0/3432964935 learned_addr learned my addr 172.21.15.17:0/3432964935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a381965b0 0x7f7a38196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3dc9d640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38105410 0x7f7a3819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 -- 172.21.15.17:0/3432964935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a381965b0 msgr2=0x7f7a38196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a381965b0 0x7f7a38196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 -- 172.21.15.17:0/3432964935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38105410 msgr2=0x7f7a3819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38105410 0x7f7a3819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.675+0000 7f7a3e49e640 1 -- 172.21.15.17:0/3432964935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28009560 con 0x7f7a38100340 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.676+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100340 0x7f7a3819c400 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400e9f0 tx=0x7f7a3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.676+0000 7f7a2f7fe640 1 -- 172.21.15.17:0/3432964935 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34009700 con 0x7f7a38100340 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.676+0000 7f7a2f7fe640 1 -- 172.21.15.17:0/3432964935 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a3400cd80 con 0x7f7a38100340 2024-02-14T15:15:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.676+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a381a32c0 con 0x7f7a38100340 2024-02-14T15:15:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.676+0000 7f7a2f7fe640 1 -- 172.21.15.17:0/3432964935 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34018920 con 0x7f7a38100340 2024-02-14T15:15:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.676+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a381a37a0 con 0x7f7a38100340 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.678+0000 7f7a433af740 1 Processor -- start 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a433af740 1 -- start start 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a2f7fe640 1 -- 172.21.15.17:0/3432964935 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7a34018a80 con 0x7f7a38100340 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a433af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ea410 0x56057c3ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a433af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a433af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3f5e30 0x56057c3f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a433af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c3589e0 con 0x56057c3ea410 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a433af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c355f10 con 0x56057c3f5e30 2024-02-14T15:15:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a433af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c3516f0 con 0x56057c3ead30 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3ec9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ea410 0x56057c3ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:56696/0 (socket says 172.21.15.17:56696) 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 -- 172.21.15.17:0/2962295400 learned_addr learned my addr 172.21.15.17:0/2962295400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3ec9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ea410 0x56057c3ea7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36960/0 (socket says 172.21.15.17:36960) 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3dc9d640 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3f5e30 0x56057c3f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 -- 172.21.15.17:0/2962295400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3f5e30 msgr2=0x56057c3f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3f5e30 0x56057c3f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 -- 172.21.15.17:0/2962295400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ea410 msgr2=0x56057c3ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ea410 0x56057c3ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3e49e640 1 -- 172.21.15.17:0/2962295400 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56057c062920 con 0x56057c3ead30 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.679+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ea410 0x56057c3ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.680+0000 7f7a2f7fe640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7a240a2220 0x7f7a240a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.680+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3f58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a340cbba0 tx=0x7f7a340d4f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70b61cfe457c58ed server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.680+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7a240a2220 0x7f7a240a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.680+0000 7f7a2f7fe640 1 -- 172.21.15.17:0/3432964935 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f7a34014070 con 0x7f7a38100340 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.680+0000 7f7a2cff9640 1 -- 172.21.15.17:0/2962295400 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34097170 con 0x56057c3ead30 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.680+0000 7f7a2cff9640 1 -- 172.21.15.17:0/2962295400 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a340d3030 con 0x56057c3ead30 2024-02-14T15:15:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.680+0000 7f7a2cff9640 1 -- 172.21.15.17:0/2962295400 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3409b5d0 con 0x56057c3ead30 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 -- 172.21.15.17:0/2962295400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 msgr2=0x56057c3f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7a240a2220 0x7f7a240a46e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009fd0 tx=0x7f7a30009500 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3f58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a340cbba0 tx=0x7f7a340d4f10 comp rx=0 tx=0).stop 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 -- 172.21.15.17:0/2962295400 shutdown_connections 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3f5e30 0x56057c3f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3f58f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 --2- 172.21.15.17:0/2962295400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ea410 0x56057c3ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 -- 172.21.15.17:0/2962295400 >> 172.21.15.17:0/2962295400 conn(0x56057c3e4e50 msgr2=0x56057c3e5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 -- 172.21.15.17:0/2962295400 shutdown_connections 2024-02-14T15:15:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 -- 172.21.15.17:0/2962295400 wait complete. 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 -- start start 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 Processor -- start 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ead30 0x56057c3f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3f5e30 0x56057c3f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.681+0000 7f7a433af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c2edfa0 con 0x56057c3ead30 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a433af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c025250 con 0x56057c3ea410 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a433af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c2d7200 con 0x56057c3f5e30 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a433af740 10 client.0 _ll_register_callbacks cb 0x56057c346b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3dc9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ead30 0x56057c3f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3ec9f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3f5e30 0x56057c3f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3dc9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ead30 0x56057c3f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36976/0 (socket says 172.21.15.17:36976) 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3dc9d640 1 -- 172.21.15.17:0/2429756521 learned_addr learned my addr 172.21.15.17:0/2429756521 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 msgr2=0x56057c3f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/2429756521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ead30 msgr2=0x56057c3f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ead30 0x56057c3f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/2429756521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a381a37a0 con 0x56057c3f5e30 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3f5e30 0x56057c3f3070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009d20 tx=0x7f7a30004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7de2799d49ac2f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a21ffb640 1 -- 172.21.15.17:0/2429756521 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30016070 con 0x56057c3f5e30 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56057c051600 con 0x56057c3f5e30 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a21ffb640 1 -- 172.21.15.17:0/2429756521 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f7a30003c40 con 0x56057c3f5e30 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a21ffb640 1 -- 172.21.15.17:0/2429756521 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a30003e00 con 0x56057c3f5e30 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a433af740 10 client.26830 fetch_fsmap learned FSMap version 35 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.682+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56057c34fd70 con 0x56057c3f5e30 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a21ffb640 1 -- 172.21.15.17:0/2429756521 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56057c051600 con 0x56057c3f5e30 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a21ffb640 1 -- 172.21.15.17:0/2429756521 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a3001f990 con 0x56057c3f5e30 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a433af740 10 client.26830 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a433af740 10 client.26830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a433af740 20 client.26830 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a433af740 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c405ca0 0x56057c4080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a433af740 4 client.26830 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56057c0a21b0 con 0x56057c405ca0 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c405ca0 0x56057c4080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a21ffb640 10 client.26830 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.683+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c405ca0 0x56057c4080c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7438795ffe70f535 server_cookie=51bfd4554b1c7bab in_seq=0 out_seq=0 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a21ffb640 1 -- 172.21.15.17:0/2429756521 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56057c0a21b0 con 0x56057c405ca0 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a21ffb640 10 client.26830 handle_command_reply: tid=0 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 client.26830 shutdown 2024-02-14T15:15:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3f5e30 msgr2=0x56057c3f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3f5e30 0x56057c3f3070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009d20 tx=0x7f7a30004ab0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 shutdown_connections 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c405ca0 0x56057c4080c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3ead30 0x56057c3f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a21ffb640 0 client.26830 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3f5e30 0x56057c3f3070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 --2- 172.21.15.17:0/2429756521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 >> 172.21.15.17:0/2429756521 conn(0x56057c3e4e50 msgr2=0x56057c3e6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 shutdown_connections 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 1 -- 172.21.15.17:0/2429756521 wait complete. 2024-02-14T15:15:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.684+0000 7f7a433af740 20 client.26830 trim_cache size 0 max 16384 2024-02-14T15:15:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.697+0000 7f7a433af740 1 Processor -- start 2024-02-14T15:15:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a433af740 1 -- start start 2024-02-14T15:15:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a433af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a433af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a433af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a433af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c3e05a0 con 0x56057c3f5e30 2024-02-14T15:15:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a433af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c3f8220 con 0x56057c3ea410 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a433af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c3e32d0 con 0x56057c3ead30 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3efdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36980/0 (socket says 172.21.15.17:36980) 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/3375002883 learned_addr learned my addr 172.21.15.17:0/3375002883 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3dc9d640 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/3375002883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 msgr2=0x56057c3fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/3375002883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 msgr2=0x56057c3fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/3375002883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56057c2ec820 con 0x56057c3f5e30 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3efdb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005c40 tx=0x7f7a30009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52c1cc3f17b63af server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a23fff640 1 -- 172.21.15.17:0/3375002883 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000fe70 con 0x56057c3f5e30 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a23fff640 1 -- 172.21.15.17:0/3375002883 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a3001fa50 con 0x56057c3f5e30 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.698+0000 7f7a23fff640 1 -- 172.21.15.17:0/3375002883 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30036e20 con 0x56057c3f5e30 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 -- 172.21.15.17:0/3375002883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 msgr2=0x56057c3efdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3efdb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005c40 tx=0x7f7a30009be0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 -- 172.21.15.17:0/3375002883 shutdown_connections 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3efdb0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 --2- 172.21.15.17:0/3375002883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 -- 172.21.15.17:0/3375002883 >> 172.21.15.17:0/3375002883 conn(0x56057c3e4e50 msgr2=0x56057c3f1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 -- 172.21.15.17:0/3375002883 shutdown_connections 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.699+0000 7f7a433af740 1 -- 172.21.15.17:0/3375002883 wait complete. 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 -- start start 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 Processor -- start 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c3fd5b0 con 0x56057c3f5e30 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c45c1b0 con 0x56057c3ea410 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057c45c2f0 con 0x56057c3ead30 2024-02-14T15:15:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a433af740 10 client.0 _ll_register_callbacks cb 0x7f7a340d4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3ec9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3dc9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3e49e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3dc9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:56732/0 (socket says 172.21.15.17:56732) 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3dc9d640 1 -- 172.21.15.17:0/3283811260 learned_addr learned my addr 172.21.15.17:0/3283811260 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/3283811260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 msgr2=0x56057c3fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/3283811260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 msgr2=0x56057c3fb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.700+0000 7f7a3ec9f640 1 -- 172.21.15.17:0/3283811260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56057c34fd70 con 0x56057c3f5e30 2024-02-14T15:15:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a3ec9f640 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3fc1b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7a3001ff80 tx=0x7f7a30036670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ad68d19ec68d54 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a21ffb640 1 -- 172.21.15.17:0/3283811260 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30004e50 con 0x56057c3f5e30 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a21ffb640 1 -- 172.21.15.17:0/3283811260 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f7a30036760 con 0x56057c3f5e30 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a340d3030 con 0x56057c3f5e30 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a21ffb640 1 -- 172.21.15.17:0/3283811260 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a30036a20 con 0x56057c3f5e30 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 10 client.17325 fetch_fsmap learned FSMap version 35 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56057c312720 con 0x56057c3f5e30 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a21ffb640 1 -- 172.21.15.17:0/3283811260 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a340d3030 con 0x56057c3f5e30 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a21ffb640 1 -- 172.21.15.17:0/3283811260 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a30005720 con 0x56057c3f5e30 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 10 client.17325 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:15:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 10 client.17325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:15:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 20 client.17325 populate_metadata read hostname 'smithi017' 2024-02-14T15:15:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c45d7d0 0x56057c45fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:15:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 4 client.17325 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:15:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56057c08e550 con 0x56057c45d7d0 2024-02-14T15:15:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.701+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c45d7d0 0x56057c45fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:15:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.702+0000 7f7a3e49e640 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c45d7d0 0x56057c45fbc0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278319f5022e8ba6 server_cookie=c697760ce93422f4 in_seq=0 out_seq=0 2024-02-14T15:15:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.702+0000 7f7a21ffb640 10 client.17325 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.715+0000 7f7a21ffb640 1 -- 172.21.15.17:0/3283811260 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56057c08e550 con 0x56057c45d7d0 2024-02-14T15:15:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.715+0000 7f7a21ffb640 10 client.17325 handle_command_reply: tid=0 2024-02-14T15:15:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.715+0000 7f7a433af740 1 client.17325 shutdown 2024-02-14T15:15:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 msgr2=0x56057c3fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3fc1b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7a3001ff80 tx=0x7f7a30036670 comp rx=0 tx=0).stop 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 shutdown_connections 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56057c45d7d0 0x56057c45fbc0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56057c3f5e30 0x56057c3fc1b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56057c3ead30 0x56057c3fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 --2- 172.21.15.17:0/3283811260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56057c3ea410 0x56057c3fb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a21ffb640 0 client.17325 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 >> 172.21.15.17:0/3283811260 conn(0x56057c3e4e50 msgr2=0x56057c3e3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 shutdown_connections 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 1 -- 172.21.15.17:0/3283811260 wait complete. 2024-02-14T15:15:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a433af740 20 client.17325 trim_cache size 0 max 16384 2024-02-14T15:15:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:15:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:15:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:15:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:15:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7a240a2220 msgr2=0x7f7a240a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7a240a2220 0x7f7a240a46e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009fd0 tx=0x7f7a30009500 comp rx=0 tx=0).stop 2024-02-14T15:15:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100340 msgr2=0x7f7a3819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:15:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100340 0x7f7a3819c400 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400e9f0 tx=0x7f7a3400eeb0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 shutdown_connections 2024-02-14T15:15:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7a240a2220 0x7f7a240a46e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a381965b0 0x7f7a38196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38105410 0x7f7a3819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 --2- 172.21.15.17:0/3432964935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a38100340 0x7f7a3819c400 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:15:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.716+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 >> 172.21.15.17:0/3432964935 conn(0x7f7a38077220 msgr2=0x7f7a380fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:15:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.717+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 shutdown_connections 2024-02-14T15:15:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:15:55.717+0000 7f7a40702640 1 -- 172.21.15.17:0/3432964935 wait complete. 2024-02-14T15:15:55.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1292110 10.39 MB/sec execute 537 sec latency 84.270 ms 2024-02-14T15:15:56.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1293379 10.39 MB/sec execute 538 sec latency 29.658 ms 2024-02-14T15:15:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:56 smithi076 ceph-mon[37057]: pgmap v393: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 167 op/s 2024-02-14T15:15:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:56 smithi017 ceph-mon[32549]: pgmap v393: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 167 op/s 2024-02-14T15:15:57.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:56 smithi050 ceph-mon[36512]: pgmap v393: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 167 op/s 2024-02-14T15:15:57.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1295699 10.39 MB/sec execute 539 sec latency 21.724 ms 2024-02-14T15:15:58.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1296936 10.38 MB/sec execute 540 sec latency 18.271 ms 2024-02-14T15:15:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:15:58 smithi076 ceph-mon[37057]: pgmap v394: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.4 MiB/s wr, 252 op/s 2024-02-14T15:15:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:15:58 smithi017 ceph-mon[32549]: pgmap v394: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.4 MiB/s wr, 252 op/s 2024-02-14T15:15:59.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:15:58 smithi050 ceph-mon[36512]: pgmap v394: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.4 MiB/s wr, 252 op/s 2024-02-14T15:15:59.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1298702 10.38 MB/sec execute 541 sec latency 112.760 ms 2024-02-14T15:16:00.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1299974 10.38 MB/sec execute 542 sec latency 59.480 ms 2024-02-14T15:16:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:00 smithi076 ceph-mon[37057]: pgmap v395: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 168 op/s 2024-02-14T15:16:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:00 smithi017 ceph-mon[32549]: pgmap v395: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 168 op/s 2024-02-14T15:16:01.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:00 smithi050 ceph-mon[36512]: pgmap v395: 129 pgs: 129 active+clean; 591 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 168 op/s 2024-02-14T15:16:01.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1300679 10.36 MB/sec execute 543 sec latency 51.940 ms 2024-02-14T15:16:02.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1303004 10.37 MB/sec execute 544 sec latency 69.366 ms 2024-02-14T15:16:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:02 smithi076 ceph-mon[37057]: pgmap v396: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 275 op/s 2024-02-14T15:16:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:02 smithi017 ceph-mon[32549]: pgmap v396: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 275 op/s 2024-02-14T15:16:03.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:02 smithi050 ceph-mon[36512]: pgmap v396: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 275 op/s 2024-02-14T15:16:03.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1304104 10.36 MB/sec execute 545 sec latency 20.075 ms 2024-02-14T15:16:04.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1305861 10.35 MB/sec execute 546 sec latency 87.899 ms 2024-02-14T15:16:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:04 smithi076 ceph-mon[37057]: pgmap v397: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 191 op/s 2024-02-14T15:16:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:04 smithi017 ceph-mon[32549]: pgmap v397: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 191 op/s 2024-02-14T15:16:05.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:04 smithi050 ceph-mon[36512]: pgmap v397: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 191 op/s 2024-02-14T15:16:05.891 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1307393 10.36 MB/sec execute 547 sec latency 14.370 ms 2024-02-14T15:16:06.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1309509 10.35 MB/sec execute 548 sec latency 34.991 ms 2024-02-14T15:16:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:06 smithi076 ceph-mon[37057]: pgmap v398: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 191 op/s 2024-02-14T15:16:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:06 smithi017 ceph-mon[32549]: pgmap v398: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 191 op/s 2024-02-14T15:16:07.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:06 smithi050 ceph-mon[36512]: pgmap v398: 129 pgs: 129 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 191 op/s 2024-02-14T15:16:07.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1310766 10.35 MB/sec execute 549 sec latency 132.176 ms 2024-02-14T15:16:08.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:08 smithi050 ceph-mon[36512]: pgmap v399: 129 pgs: 129 active+clean; 613 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-02-14T15:16:08.483 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:08 smithi076 ceph-mon[37057]: pgmap v399: 129 pgs: 129 active+clean; 613 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-02-14T15:16:08.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:08 smithi017 ceph-mon[32549]: pgmap v399: 129 pgs: 129 active+clean; 613 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.806+0000 7f7683476640 1 -- 172.21.15.17:0/1518755343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c068d40 msgr2=0x7f767c069120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.806+0000 7f7683476640 1 --2- 172.21.15.17:0/1518755343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c068d40 0x7f767c069120 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7678009930 tx=0x7f767802bc30 comp rx=0 tx=0).stop 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 -- 172.21.15.17:0/1518755343 shutdown_connections 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 --2- 172.21.15.17:0/1518755343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 0x7f767c10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 --2- 172.21.15.17:0/1518755343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767c0696f0 0x7f767c105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 --2- 172.21.15.17:0/1518755343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c068d40 0x7f767c069120 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 -- 172.21.15.17:0/1518755343 >> 172.21.15.17:0/1518755343 conn(0x7f767c077220 msgr2=0x7f767c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 -- 172.21.15.17:0/1518755343 shutdown_connections 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 -- 172.21.15.17:0/1518755343 wait complete. 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 Processor -- start 2024-02-14T15:16:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.807+0000 7f7683476640 1 -- start start 2024-02-14T15:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7683476640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767c068d40 0x7f767c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7683476640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 0x7f767c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7683476640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c197710 0x7f767c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7683476640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c1981e0 con 0x7f767c197710 2024-02-14T15:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7683476640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c198350 con 0x7f767c068d40 2024-02-14T15:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7683476640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c198510 con 0x7f767c106510 2024-02-14T15:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 0x7f767c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7681212640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767c068d40 0x7f767c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 0x7f767c19db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:48236/0 (socket says 172.21.15.17:48236) 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7681a13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c197710 0x7f767c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 -- 172.21.15.17:0/1512834997 learned_addr learned my addr 172.21.15.17:0/1512834997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 -- 172.21.15.17:0/1512834997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767c068d40 msgr2=0x7f767c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767c068d40 0x7f767c19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 -- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c197710 msgr2=0x7f767c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c197710 0x7f767c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 -- 172.21.15.17:0/1512834997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7678009560 con 0x7f767c106510 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7681a13640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c197710 0x7f767c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.808+0000 7f7680a11640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 0x7f767c19db90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f766c00bb00 tx=0x7f766c009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.809+0000 7f76727fc640 1 -- 172.21.15.17:0/1512834997 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00f460 con 0x7f767c106510 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.809+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c100080 con 0x7f767c106510 2024-02-14T15:16:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.809+0000 7f76727fc640 1 -- 172.21.15.17:0/1512834997 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f766c009e90 con 0x7f767c106510 2024-02-14T15:16:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.809+0000 7f76727fc640 1 -- 172.21.15.17:0/1512834997 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c0195c0 con 0x7f767c106510 2024-02-14T15:16:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.809+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c1005e0 con 0x7f767c106510 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.811+0000 7f7686046740 1 Processor -- start 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7686046740 1 -- start start 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f76727fc640 1 -- 172.21.15.17:0/1512834997 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f766c00e750 con 0x7f767c106510 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7686046740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf0720 0x5587fadf0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7686046740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf1040 0x5587fadf96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7686046740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadfc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7686046740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fad5e9e0 con 0x5587fadf1040 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7686046740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fad5bf10 con 0x5587fadf0720 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7686046740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fad576f0 con 0x5587fadf9c20 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681a13640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf0720 0x5587fadf0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf1040 0x5587fadf96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681a13640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf0720 0x5587fadf0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:47254/0 (socket says 172.21.15.17:47254) 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf1040 0x5587fadf96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33294/0 (socket says 172.21.15.17:33294) 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681a13640 1 -- 172.21.15.17:0/1227437003 learned_addr learned my addr 172.21.15.17:0/1227437003 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7680a11640 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadfc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681a13640 1 -- 172.21.15.17:0/1227437003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 msgr2=0x5587fadfc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681a13640 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadfc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681a13640 1 -- 172.21.15.17:0/1227437003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf1040 msgr2=0x5587fadf96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.812+0000 7f7681a13640 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf1040 0x5587fadf96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f7681a13640 1 -- 172.21.15.17:0/1227437003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587faa68920 con 0x5587fadf0720 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f7681212640 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf1040 0x5587fadf96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f76727fc640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f76680a2220 0x7f76680a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f7681a13640 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf0720 0x5587fadf0b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7674011790 tx=0x7f7674011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c63a82c53fb4f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f7681212640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f76680a2220 0x7f76680a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f76727fc640 1 -- 172.21.15.17:0/1512834997 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f766c0cddb0 con 0x7f767c106510 2024-02-14T15:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f76677fe640 1 -- 172.21.15.17:0/1227437003 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674011ea0 con 0x5587fadf0720 2024-02-14T15:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f76677fe640 1 -- 172.21.15.17:0/1227437003 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f767400fe80 con 0x5587fadf0720 2024-02-14T15:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.813+0000 7f76677fe640 1 -- 172.21.15.17:0/1227437003 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674004840 con 0x5587fadf0720 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 -- 172.21.15.17:0/1227437003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf0720 msgr2=0x5587fadf0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf0720 0x5587fadf0b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7674011790 tx=0x7f7674011c50 comp rx=0 tx=0).stop 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7681212640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f76680a2220 0x7f76680a46e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7678005ab0 tx=0x7f7678042040 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 -- 172.21.15.17:0/1227437003 shutdown_connections 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf1040 0x5587fadf96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadfc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 --2- 172.21.15.17:0/1227437003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf0720 0x5587fadf0b00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 -- 172.21.15.17:0/1227437003 >> 172.21.15.17:0/1227437003 conn(0x5587fadeae50 msgr2=0x5587fadeb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 -- 172.21.15.17:0/1227437003 shutdown_connections 2024-02-14T15:16:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 -- 172.21.15.17:0/1227437003 wait complete. 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 -- start start 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.814+0000 7f7686046740 1 Processor -- start 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7686046740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7686046740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 0x5587fadf6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7686046740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7686046740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587facf3fa0 con 0x5587fadf0720 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7686046740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587faa2b250 con 0x5587fadf1040 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7686046740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587facdd200 con 0x5587fadf9c20 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7686046740 10 client.0 _ll_register_callbacks cb 0x5587fad4cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7680a11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 0x5587fadf6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681a13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33304/0 (socket says 172.21.15.17:33304) 2024-02-14T15:16:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 -- 172.21.15.17:0/365437604 learned_addr learned my addr 172.21.15.17:0/365437604 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 -- 172.21.15.17:0/365437604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 msgr2=0x5587fadf6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 -- 172.21.15.17:0/365437604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 msgr2=0x5587fadf6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 0x5587fadf6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.815+0000 7f7681212640 1 -- 172.21.15.17:0/365437604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7674011440 con 0x5587fadf0720 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7681212640 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf9590 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7678035000 tx=0x7f767802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2837c94e2c60cf7 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7664ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7664ff9640 1 -- 172.21.15.17:0/365437604 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678004250 con 0x5587fadf0720 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7664ff9640 1 -- 172.21.15.17:0/365437604 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f76780043b0 con 0x5587fadf0720 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587faa57600 con 0x5587fadf0720 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7664ff9640 1 -- 172.21.15.17:0/365437604 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7678004df0 con 0x5587fadf0720 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 10 client.17337 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5587fad55d70 con 0x5587fadf0720 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7664ff9640 1 -- 172.21.15.17:0/365437604 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5587faa57600 con 0x5587fadf0720 2024-02-14T15:16:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7664ff9640 1 -- 172.21.15.17:0/365437604 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f76780039c0 con 0x5587fadf0720 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 10 client.17337 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 20 client.17337 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae099c0 0x5587fae0bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 4 client.17337 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.816+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587faaa81b0 con 0x5587fae099c0 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.817+0000 7f7680a11640 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae099c0 0x5587fae0bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.817+0000 7f7680a11640 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae099c0 0x5587fae0bdb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c5fe84aa7cc670 server_cookie=d6a3be80d5d480f2 in_seq=0 out_seq=0 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.817+0000 7f7664ff9640 10 client.17337 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7664ff9640 1 -- 172.21.15.17:0/365437604 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587faaa81b0 con 0x5587fae099c0 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7664ff9640 10 client.17337 handle_command_reply: tid=0 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 client.17337 shutdown 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 msgr2=0x5587fadf9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf9590 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7678035000 tx=0x7f767802bc30 comp rx=0 tx=0).stop 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 shutdown_connections 2024-02-14T15:16:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae099c0 0x5587fae0bdb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 0x5587fadf6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 --2- 172.21.15.17:0/365437604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf9590 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 >> 172.21.15.17:0/365437604 conn(0x5587fadeae50 msgr2=0x5587fadecba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7664ff9640 0 client.17337 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 shutdown_connections 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 1 -- 172.21.15.17:0/365437604 wait complete. 2024-02-14T15:16:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.818+0000 7f7686046740 20 client.17337 trim_cache size 0 max 16384 2024-02-14T15:16:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.831+0000 7f7686046740 1 Processor -- start 2024-02-14T15:16:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.831+0000 7f7686046740 1 -- start start 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.831+0000 7f7686046740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.831+0000 7f7686046740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 0x5587fadf9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7686046740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7686046740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fadebc50 con 0x5587fadf0720 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7686046740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fade65a0 con 0x5587fadf1040 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7686046740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fae06470 con 0x5587fadf9c20 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33314/0 (socket says 172.21.15.17:33314) 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 -- 172.21.15.17:0/2985697 learned_addr learned my addr 172.21.15.17:0/2985697 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681a13640 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 -- 172.21.15.17:0/2985697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 msgr2=0x5587fadf9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 -- 172.21.15.17:0/2985697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 msgr2=0x5587fadf9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 0x5587fadf9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 -- 172.21.15.17:0/2985697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587facf2820 con 0x5587fadf0720 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7681212640 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf68c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7678036040 tx=0x7f7678003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358da9a178c1676a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7666ffd640 1 -- 172.21.15.17:0/2985697 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678004760 con 0x5587fadf0720 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7666ffd640 1 -- 172.21.15.17:0/2985697 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7678046040 con 0x5587fadf0720 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.832+0000 7f7666ffd640 1 -- 172.21.15.17:0/2985697 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678005150 con 0x5587fadf0720 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 -- 172.21.15.17:0/2985697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 msgr2=0x5587fadf68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf68c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7678036040 tx=0x7f7678003f60 comp rx=0 tx=0).stop 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 -- 172.21.15.17:0/2985697 shutdown_connections 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf9c20 0x5587fadf9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf1040 0x5587fadf9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 --2- 172.21.15.17:0/2985697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadf68c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 -- 172.21.15.17:0/2985697 >> 172.21.15.17:0/2985697 conn(0x5587fadeae50 msgr2=0x5587fadf61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 -- 172.21.15.17:0/2985697 shutdown_connections 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 -- 172.21.15.17:0/2985697 wait complete. 2024-02-14T15:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 -- start start 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.833+0000 7f7686046740 1 Processor -- start 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf1040 0x5587fadff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf9c20 0x5587fadfff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fae013e0 con 0x5587fadf0720 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fae5fea0 con 0x5587fadf9c20 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fae5ffe0 con 0x5587fadf1040 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 10 client.0 _ll_register_callbacks cb 0x7f7674009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33322/0 (socket says 172.21.15.17:33322) 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 -- 172.21.15.17:0/2120873154 learned_addr learned my addr 172.21.15.17:0/2120873154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7680a11640 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf1040 0x5587fadff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681a13640 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf9c20 0x5587fadfff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 -- 172.21.15.17:0/2120873154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf1040 msgr2=0x5587fadff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf1040 0x5587fadff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 -- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf9c20 msgr2=0x5587fadfff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf9c20 0x5587fadfff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 -- 172.21.15.17:0/2120873154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587fad55d70 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7681212640 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadff490 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7678003f30 tx=0x7f7678004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973c874f30bf3228 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7664ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7664ff9640 1 -- 172.21.15.17:0/2120873154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767802c870 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7664ff9640 1 -- 172.21.15.17:0/2120873154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f7678003800 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.834+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f767400fe80 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7664ff9640 1 -- 172.21.15.17:0/2120873154 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76780039c0 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 10 client.17343 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5587fad18720 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7664ff9640 1 -- 172.21.15.17:0/2120873154 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f767400fe80 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7664ff9640 1 -- 172.21.15.17:0/2120873154 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7678071750 con 0x5587fadf0720 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 10 client.17343 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 20 client.17343 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae61330 0x5587fae63720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 4 client.17343 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:16:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5587faa94550 con 0x5587fae61330 2024-02-14T15:16:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.835+0000 7f7680a11640 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae61330 0x5587fae63720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.836+0000 7f7680a11640 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae61330 0x5587fae63720 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61414caf871fa53c server_cookie=c8c756678ad94edb in_seq=0 out_seq=0 2024-02-14T15:16:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.836+0000 7f7664ff9640 10 client.17343 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7664ff9640 1 -- 172.21.15.17:0/2120873154 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5587faa94550 con 0x5587fae61330 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7664ff9640 10 client.17343 handle_command_reply: tid=0 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 client.17343 shutdown 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 msgr2=0x5587fadff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadff490 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7678003f30 tx=0x7f7678004500 comp rx=0 tx=0).stop 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 shutdown_connections 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5587fae61330 0x5587fae63720 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5587fadf9c20 0x5587fadfff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587fadf1040 0x5587fadff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7664ff9640 0 client.17343 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 --2- 172.21.15.17:0/2120873154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587fadf0720 0x5587fadff490 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 >> 172.21.15.17:0/2120873154 conn(0x5587fadeae50 msgr2=0x5587fadec280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 shutdown_connections 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 1 -- 172.21.15.17:0/2120873154 wait complete. 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.849+0000 7f7686046740 20 client.17343 trim_cache size 0 max 16384 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:16:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:16:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f76680a2220 msgr2=0x7f76680a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f76680a2220 0x7f76680a46e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7678005ab0 tx=0x7f7678042040 comp rx=0 tx=0).stop 2024-02-14T15:16:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 msgr2=0x7f767c19db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 0x7f767c19db90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f766c00bb00 tx=0x7f766c009240 comp rx=0 tx=0).stop 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 shutdown_connections 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f76680a2220 0x7f76680a46e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c197710 0x7f767c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767c106510 0x7f767c19db90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 --2- 172.21.15.17:0/1512834997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767c068d40 0x7f767c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 >> 172.21.15.17:0/1512834997 conn(0x7f767c077220 msgr2=0x7f767c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 shutdown_connections 2024-02-14T15:16:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:08.850+0000 7f7683476640 1 -- 172.21.15.17:0/1512834997 wait complete. 2024-02-14T15:16:08.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1312752 10.35 MB/sec execute 550 sec latency 45.451 ms 2024-02-14T15:16:09.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1314149 10.35 MB/sec execute 551 sec latency 162.732 ms 2024-02-14T15:16:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:10 smithi050 ceph-mon[36512]: osdmap e110: 12 total, 12 up, 12 in 2024-02-14T15:16:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:10 smithi050 ceph-mon[36512]: pgmap v401: 129 pgs: 129 active+clean; 613 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-02-14T15:16:10.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:10 smithi076 ceph-mon[37057]: osdmap e110: 12 total, 12 up, 12 in 2024-02-14T15:16:10.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:10 smithi076 ceph-mon[37057]: pgmap v401: 129 pgs: 129 active+clean; 613 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-02-14T15:16:10.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:10 smithi017 ceph-mon[32549]: osdmap e110: 12 total, 12 up, 12 in 2024-02-14T15:16:10.516 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:10 smithi017 ceph-mon[32549]: pgmap v401: 129 pgs: 129 active+clean; 613 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-02-14T15:16:10.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1315646 10.34 MB/sec execute 552 sec latency 28.916 ms 2024-02-14T15:16:11.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1317693 10.34 MB/sec execute 553 sec latency 44.520 ms 2024-02-14T15:16:12.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1318981 10.34 MB/sec execute 554 sec latency 90.399 ms 2024-02-14T15:16:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:12 smithi050 ceph-mon[36512]: pgmap v402: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:13.218 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:12 smithi017 ceph-mon[32549]: pgmap v402: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:12 smithi076 ceph-mon[37057]: pgmap v402: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:13.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1321198 10.34 MB/sec execute 555 sec latency 27.181 ms 2024-02-14T15:16:14.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1322243 10.33 MB/sec execute 556 sec latency 83.875 ms 2024-02-14T15:16:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:14 smithi050 ceph-mon[36512]: pgmap v403: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:14 smithi076 ceph-mon[37057]: pgmap v403: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:14 smithi017 ceph-mon[32549]: pgmap v403: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:15.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1324608 10.34 MB/sec execute 557 sec latency 21.165 ms 2024-02-14T15:16:16.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1325550 10.32 MB/sec execute 558 sec latency 18.337 ms 2024-02-14T15:16:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:16 smithi050 ceph-mon[36512]: pgmap v404: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:16 smithi050 ceph-mon[36512]: osdmap e111: 12 total, 12 up, 12 in 2024-02-14T15:16:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:16 smithi076 ceph-mon[37057]: pgmap v404: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:16 smithi076 ceph-mon[37057]: osdmap e111: 12 total, 12 up, 12 in 2024-02-14T15:16:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:16 smithi017 ceph-mon[32549]: pgmap v404: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-02-14T15:16:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:16 smithi017 ceph-mon[32549]: osdmap e111: 12 total, 12 up, 12 in 2024-02-14T15:16:17.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1327938 10.33 MB/sec execute 559 sec latency 51.308 ms 2024-02-14T15:16:18.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1328901 10.32 MB/sec execute 560 sec latency 24.247 ms 2024-02-14T15:16:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:18 smithi050 ceph-mon[36512]: pgmap v406: 129 pgs: 129 active+clean; 561 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 278 op/s 2024-02-14T15:16:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:18 smithi076 ceph-mon[37057]: pgmap v406: 129 pgs: 129 active+clean; 561 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 278 op/s 2024-02-14T15:16:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:18 smithi017 ceph-mon[32549]: pgmap v406: 129 pgs: 129 active+clean; 561 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 278 op/s 2024-02-14T15:16:19.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1330785 10.32 MB/sec execute 561 sec latency 78.939 ms 2024-02-14T15:16:20.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1332117 10.32 MB/sec execute 562 sec latency 53.448 ms 2024-02-14T15:16:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:20 smithi050 ceph-mon[36512]: pgmap v407: 129 pgs: 129 active+clean; 561 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2024-02-14T15:16:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:20 smithi076 ceph-mon[37057]: pgmap v407: 129 pgs: 129 active+clean; 561 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2024-02-14T15:16:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:20 smithi017 ceph-mon[32549]: pgmap v407: 129 pgs: 129 active+clean; 561 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2024-02-14T15:16:21.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1334075 10.31 MB/sec execute 563 sec latency 33.095 ms 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.943+0000 7fee44876640 1 -- 172.21.15.17:0/61376681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 msgr2=0x7fee4010cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.943+0000 7fee44876640 1 --2- 172.21.15.17:0/61376681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee4010cfe0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fee38009cd0 tx=0x7fee3802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 -- 172.21.15.17:0/61376681 shutdown_connections 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 --2- 172.21.15.17:0/61376681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee4010cfe0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 --2- 172.21.15.17:0/61376681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40100cf0 0x7fee40104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 --2- 172.21.15.17:0/61376681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee40100340 0x7fee40100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 -- 172.21.15.17:0/61376681 >> 172.21.15.17:0/61376681 conn(0x7fee40077220 msgr2=0x7fee40077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 -- 172.21.15.17:0/61376681 shutdown_connections 2024-02-14T15:16:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 -- 172.21.15.17:0/61376681 wait complete. 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 Processor -- start 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.944+0000 7fee44876640 1 -- start start 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee44876640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40100340 0x7fee40101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee44876640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee40102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee44876640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee40102850 0x7fee401a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee44876640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401a6070 con 0x7fee40105410 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee44876640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401a61e0 con 0x7fee40100340 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee44876640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401a63a0 con 0x7fee40102850 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee40102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee40102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38954/0 (socket says 172.21.15.17:38954) 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3dd9b640 1 -- 172.21.15.17:0/2719172912 learned_addr learned my addr 172.21.15.17:0/2719172912 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3ed9d640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee40102850 0x7fee401a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3e59c640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40100340 0x7fee40101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3ed9d640 1 -- 172.21.15.17:0/2719172912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40100340 msgr2=0x7fee40101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3ed9d640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40100340 0x7fee40101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3ed9d640 1 -- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 msgr2=0x7fee40102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3ed9d640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee40102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3ed9d640 1 -- 172.21.15.17:0/2719172912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38009900 con 0x7fee40102850 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee40102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.945+0000 7fee3ed9d640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee40102850 0x7fee401a5aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee3802c630 tx=0x7fee38034a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.946+0000 7fee2f7fe640 1 -- 172.21.15.17:0/2719172912 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380328b0 con 0x7fee40102850 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.946+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee401a6620 con 0x7fee40102850 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.946+0000 7fee2f7fe640 1 -- 172.21.15.17:0/2719172912 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee38003e50 con 0x7fee40102850 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.946+0000 7fee2f7fe640 1 -- 172.21.15.17:0/2719172912 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38004860 con 0x7fee40102850 2024-02-14T15:16:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.946+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee401a6af0 con 0x7fee40102850 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.948+0000 7fee2f7fe640 1 -- 172.21.15.17:0/2719172912 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fee3800a040 con 0x7fee40102850 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.948+0000 7fee47449740 1 Processor -- start 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.948+0000 7fee47449740 1 -- start start 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee47449740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd7db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee47449740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd866e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee47449740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd89010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee47449740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedceb9e0 con 0x55deedd7e040 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee47449740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedce8f10 con 0x55deedd7d720 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee47449740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedce46f0 con 0x55deedd86c20 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee2f7fe640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fee280a2250 0x7fee280a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd866e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd866e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38964/0 (socket says 172.21.15.17:38964) 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 -- 172.21.15.17:0/4250301057 learned_addr learned my addr 172.21.15.17:0/4250301057 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd7db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd89010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee2f7fe640 1 -- 172.21.15.17:0/2719172912 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fee380f0fe0 con 0x7fee40102850 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3ed9d640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fee280a2250 0x7fee280a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 -- 172.21.15.17:0/4250301057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 msgr2=0x55deedd89010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd89010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 -- 172.21.15.17:0/4250301057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 msgr2=0x55deedd7db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd7db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 -- 172.21.15.17:0/4250301057 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deed9f5920 con 0x55deedd7e040 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3e59c640 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd866e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fee30009d20 tx=0x7fee30011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898202a5c6f1ccee server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.949+0000 7fee3ed9d640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fee280a2250 0x7fee280a4710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee380ba0b0 tx=0x7fee3802c0f0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee2cff9640 1 -- 172.21.15.17:0/4250301057 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30009e10 con 0x55deedd7e040 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee2cff9640 1 -- 172.21.15.17:0/4250301057 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee300198f0 con 0x55deedd7e040 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee2cff9640 1 -- 172.21.15.17:0/4250301057 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30018990 con 0x55deedd7e040 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 -- 172.21.15.17:0/4250301057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 msgr2=0x55deedd866e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd866e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fee30009d20 tx=0x7fee30011810 comp rx=0 tx=0).stop 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 -- 172.21.15.17:0/4250301057 shutdown_connections 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd866e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd89010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 --2- 172.21.15.17:0/4250301057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd7db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 -- 172.21.15.17:0/4250301057 >> 172.21.15.17:0/4250301057 conn(0x55deedd77e50 msgr2=0x55deedd78230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 -- 172.21.15.17:0/4250301057 shutdown_connections 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.950+0000 7fee47449740 1 -- 172.21.15.17:0/4250301057 wait complete. 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 -- start start 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 Processor -- start 2024-02-14T15:16:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7d720 0x55deedd86560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7e040 0x55deedd838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd83e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedc80fa0 con 0x55deedd86c20 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deed9b8250 con 0x55deedd7e040 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedc6a200 con 0x55deedd7d720 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7d720 0x55deedd86560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7d720 0x55deedd86560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:45288/0 (socket says 172.21.15.17:45288) 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee3dd9b640 1 -- 172.21.15.17:0/3340481068 learned_addr learned my addr 172.21.15.17:0/3340481068 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd83e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.951+0000 7fee47449740 10 client.0 _ll_register_callbacks cb 0x55deedcd9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3e59c640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7e040 0x55deedd838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3dd9b640 1 -- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7e040 msgr2=0x55deedd838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7e040 0x55deedd838f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3dd9b640 1 -- 172.21.15.17:0/3340481068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 msgr2=0x55deedd83e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd83e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3dd9b640 1 -- 172.21.15.17:0/3340481068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee30009950 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3e59c640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7e040 0x55deedd838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7d720 0x55deedd86560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee34011dc0 tx=0x7fee34009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e362c0cf6408 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee25ffb640 1 -- 172.21.15.17:0/3340481068 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3400fee0 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55deed9e4600 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee25ffb640 1 -- 172.21.15.17:0/3340481068 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fee34004250 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee25ffb640 1 -- 172.21.15.17:0/3340481068 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee34004410 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee47449740 10 client.26866 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.952+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55deedce2d70 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee25ffb640 1 -- 172.21.15.17:0/3340481068 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55deed9e4600 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee25ffb640 1 -- 172.21.15.17:0/3340481068 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fee340037e0 con 0x55deedd7d720 2024-02-14T15:16:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee47449740 10 client.26866 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee47449740 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee47449740 20 client.26866 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee47449740 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deedd96960 0x55deedd98d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee47449740 4 client.26866 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55deeda351b0 con 0x55deedd96960 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee3e59c640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deedd96960 0x55deedd98d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee3e59c640 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deedd96960 0x55deedd98d50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b3a0cf45370472 server_cookie=6e4e44e56fe3143b in_seq=0 out_seq=0 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.953+0000 7fee25ffb640 10 client.26866 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee25ffb640 1 -- 172.21.15.17:0/3340481068 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55deeda351b0 con 0x55deedd96960 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee25ffb640 10 client.26866 handle_command_reply: tid=0 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 client.26866 shutdown 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7d720 msgr2=0x55deedd86560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7d720 0x55deedd86560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee34011dc0 tx=0x7fee34009ed0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 shutdown_connections 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deedd96960 0x55deedd98d50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd83e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7e040 0x55deedd838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee25ffb640 0 client.26866 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 --2- 172.21.15.17:0/3340481068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7d720 0x55deedd86560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 >> 172.21.15.17:0/3340481068 conn(0x55deedd77e50 msgr2=0x55deedd79ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 shutdown_connections 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 1 -- 172.21.15.17:0/3340481068 wait complete. 2024-02-14T15:16:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.954+0000 7fee47449740 20 client.26866 trim_cache size 0 max 16384 2024-02-14T15:16:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 Processor -- start 2024-02-14T15:16:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 -- start start 2024-02-14T15:16:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd86020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7e040 0x55deedd86560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedd836e0 con 0x55deedd86c20 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedd89240 con 0x55deedd7d720 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee47449740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedd73060 con 0x55deedd7e040 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd838b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38988/0 (socket says 172.21.15.17:38988) 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 -- 172.21.15.17:0/2874340017 learned_addr learned my addr 172.21.15.17:0/2874340017 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3e59c640 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7e040 0x55deedd86560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd86020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 -- 172.21.15.17:0/2874340017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7e040 msgr2=0x55deedd86560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7e040 0x55deedd86560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 -- 172.21.15.17:0/2874340017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 msgr2=0x55deedd86020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd86020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 -- 172.21.15.17:0/2874340017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deedc7f820 con 0x55deedd86c20 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.968+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd838b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fee34001eb0 tx=0x7fee34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92addc8f5e5c8d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee27fff640 1 -- 172.21.15.17:0/2874340017 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34003db0 con 0x55deedd86c20 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee27fff640 1 -- 172.21.15.17:0/2874340017 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee34011760 con 0x55deedd86c20 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee27fff640 1 -- 172.21.15.17:0/2874340017 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3400ea10 con 0x55deedd86c20 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 -- 172.21.15.17:0/2874340017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 msgr2=0x55deedd838b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd838b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fee34001eb0 tx=0x7fee34005f90 comp rx=0 tx=0).stop 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 -- 172.21.15.17:0/2874340017 shutdown_connections 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd86c20 0x55deedd838b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd7e040 0x55deedd86560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 --2- 172.21.15.17:0/2874340017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd86020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 -- 172.21.15.17:0/2874340017 >> 172.21.15.17:0/2874340017 conn(0x55deedd77e50 msgr2=0x55deedd9d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 -- 172.21.15.17:0/2874340017 shutdown_connections 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.969+0000 7fee47449740 1 -- 172.21.15.17:0/2874340017 wait complete. 2024-02-14T15:16:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 -- start start 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 Processor -- start 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd8c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd8ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd8cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deeddece80 con 0x55deedd7e040 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deeddecfc0 con 0x55deedd7d720 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deedded100 con 0x55deedd86c20 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee47449740 10 client.0 _ll_register_callbacks cb 0x7fee30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd8ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd8c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd8ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38990/0 (socket says 172.21.15.17:38990) 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3e59c640 1 -- 172.21.15.17:0/664179007 learned_addr learned my addr 172.21.15.17:0/664179007 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd8cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3e59c640 1 -- 172.21.15.17:0/664179007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 msgr2=0x55deedd8cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3e59c640 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd8cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3e59c640 1 -- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 msgr2=0x55deedd8c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.970+0000 7fee3e59c640 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd8c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee3e59c640 1 -- 172.21.15.17:0/664179007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55deedce2d70 con 0x55deedd7e040 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd8c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee3e59c640 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd8ca40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x55deedd7d1f0 tx=0x7fee30009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cc8edd043d1e64 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee25ffb640 1 -- 172.21.15.17:0/664179007 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30019410 con 0x55deedd7e040 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee25ffb640 1 -- 172.21.15.17:0/664179007 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fee30019570 con 0x55deedd7e040 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee300198f0 con 0x55deedd7e040 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee25ffb640 1 -- 172.21.15.17:0/664179007 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee30019730 con 0x55deedd7e040 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 10 client.17361 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55deedca5720 con 0x55deedd7e040 2024-02-14T15:16:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee25ffb640 1 -- 172.21.15.17:0/664179007 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee300198f0 con 0x55deedd7e040 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee25ffb640 1 -- 172.21.15.17:0/664179007 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fee3002c020 con 0x55deedd7e040 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 10 client.17361 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 20 client.17361 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deeddee950 0x55deeddf0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 4 client.17361 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.971+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55deeda21550 con 0x55deeddee950 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.972+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deeddee950 0x55deeddf0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.972+0000 7fee3dd9b640 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deeddee950 0x55deeddf0d70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c460203df52452 server_cookie=c9fb491bf90fd5c4 in_seq=0 out_seq=0 2024-02-14T15:16:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.972+0000 7fee25ffb640 10 client.17361 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.984+0000 7fee25ffb640 1 -- 172.21.15.17:0/664179007 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55deeda21550 con 0x55deeddee950 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.984+0000 7fee25ffb640 10 client.17361 handle_command_reply: tid=0 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.984+0000 7fee47449740 1 client.17361 shutdown 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 msgr2=0x55deedd8ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd8ca40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x55deedd7d1f0 tx=0x7fee30009f30 comp rx=0 tx=0).stop 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 shutdown_connections 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55deeddee950 0x55deeddf0d70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55deedd7e040 0x55deedd8ca40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deedd86c20 0x55deedd8cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee25ffb640 0 client.17361 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 --2- 172.21.15.17:0/664179007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55deedd7d720 0x55deedd8c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 >> 172.21.15.17:0/664179007 conn(0x55deedd77e50 msgr2=0x55deedd83890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 shutdown_connections 2024-02-14T15:16:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 1 -- 172.21.15.17:0/664179007 wait complete. 2024-02-14T15:16:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.985+0000 7fee47449740 20 client.17361 trim_cache size 0 max 16384 2024-02-14T15:16:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:16:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:16:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:16:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fee280a2250 msgr2=0x7fee280a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fee280a2250 0x7fee280a4710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee380ba0b0 tx=0x7fee3802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee40102850 msgr2=0x7fee401a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee40102850 0x7fee401a5aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee3802c630 tx=0x7fee38034a90 comp rx=0 tx=0).stop 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 shutdown_connections 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fee280a2250 0x7fee280a4710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee40105410 0x7fee40102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee40102850 0x7fee401a5aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 --2- 172.21.15.17:0/2719172912 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40100340 0x7fee40101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 >> 172.21.15.17:0/2719172912 conn(0x7fee40077220 msgr2=0x7fee40068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 shutdown_connections 2024-02-14T15:16:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:21.986+0000 7fee44876640 1 -- 172.21.15.17:0/2719172912 wait complete. 2024-02-14T15:16:22.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1335615 10.31 MB/sec execute 564 sec latency 27.394 ms 2024-02-14T15:16:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:22 smithi050 ceph-mon[36512]: pgmap v408: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:22 smithi076 ceph-mon[37057]: pgmap v408: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:22 smithi017 ceph-mon[32549]: pgmap v408: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:23.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1337599 10.31 MB/sec execute 565 sec latency 17.184 ms 2024-02-14T15:16:24.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1339207 10.31 MB/sec execute 566 sec latency 17.985 ms 2024-02-14T15:16:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:24 smithi050 ceph-mon[36512]: pgmap v409: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:24 smithi076 ceph-mon[37057]: pgmap v409: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:24 smithi017 ceph-mon[32549]: pgmap v409: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:25.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1340597 10.30 MB/sec execute 567 sec latency 59.780 ms 2024-02-14T15:16:26.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1342633 10.31 MB/sec execute 568 sec latency 15.503 ms 2024-02-14T15:16:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:26 smithi050 ceph-mon[36512]: pgmap v410: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:26 smithi076 ceph-mon[37057]: pgmap v410: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:26 smithi017 ceph-mon[32549]: pgmap v410: 129 pgs: 129 active+clean; 572 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 197 op/s 2024-02-14T15:16:27.892 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1344340 10.30 MB/sec execute 569 sec latency 23.826 ms 2024-02-14T15:16:28.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1346266 10.31 MB/sec execute 570 sec latency 20.813 ms 2024-02-14T15:16:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:28 smithi050 ceph-mon[36512]: pgmap v411: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 267 op/s 2024-02-14T15:16:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:28 smithi076 ceph-mon[37057]: pgmap v411: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 267 op/s 2024-02-14T15:16:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:28 smithi017 ceph-mon[32549]: pgmap v411: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 267 op/s 2024-02-14T15:16:29.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1347578 10.30 MB/sec execute 571 sec latency 157.960 ms 2024-02-14T15:16:30.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1349537 10.30 MB/sec execute 572 sec latency 86.331 ms 2024-02-14T15:16:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:30 smithi050 ceph-mon[36512]: pgmap v412: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:30 smithi076 ceph-mon[37057]: pgmap v412: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:30 smithi017 ceph-mon[32549]: pgmap v412: 129 pgs: 129 active+clean; 585 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:31.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1350878 10.29 MB/sec execute 573 sec latency 19.788 ms 2024-02-14T15:16:32.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1353115 10.30 MB/sec execute 574 sec latency 15.098 ms 2024-02-14T15:16:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:32 smithi050 ceph-mon[36512]: pgmap v413: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:32 smithi076 ceph-mon[37057]: pgmap v413: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:32 smithi017 ceph-mon[32549]: pgmap v413: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:33.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1354531 10.29 MB/sec execute 575 sec latency 17.118 ms 2024-02-14T15:16:34.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1356664 10.30 MB/sec execute 576 sec latency 63.735 ms 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.076+0000 7fccf2965640 1 -- 172.21.15.17:0/2542141910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec101e00 msgr2=0x7fccec105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.076+0000 7fccf2965640 1 --2- 172.21.15.17:0/2542141910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec101e00 0x7fccec105e40 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009930 tx=0x7fcce002bc30 comp rx=0 tx=0).stop 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.076+0000 7fccf2965640 1 -- 172.21.15.17:0/2542141910 shutdown_connections 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.076+0000 7fccf2965640 1 --2- 172.21.15.17:0/2542141910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec101e00 0x7fccec105e40 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.076+0000 7fccf2965640 1 --2- 172.21.15.17:0/2542141910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccec106520 0x7fccec10e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.076+0000 7fccf2965640 1 --2- 172.21.15.17:0/2542141910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 0x7fccec101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.077+0000 7fccf2965640 1 -- 172.21.15.17:0/2542141910 >> 172.21.15.17:0/2542141910 conn(0x7fccec0fccf0 msgr2=0x7fccec0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.077+0000 7fccf2965640 1 -- 172.21.15.17:0/2542141910 shutdown_connections 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.077+0000 7fccf2965640 1 -- 172.21.15.17:0/2542141910 wait complete. 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.077+0000 7fccf2965640 1 Processor -- start 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.077+0000 7fccf2965640 1 -- start start 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf2965640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 0x7fccec19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf2965640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccec106520 0x7fccec19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf2965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec197760 0x7fccec197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf2965640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec198230 con 0x7fccec197760 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf2965640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1983a0 con 0x7fccec106520 2024-02-14T15:16:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf2965640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec198560 con 0x7fccec101450 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 0x7fccec19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf0f02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec197760 0x7fccec197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf0f02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec197760 0x7fccec197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41562/0 (socket says 172.21.15.17:41562) 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf0f02640 1 -- 172.21.15.17:0/1937231281 learned_addr learned my addr 172.21.15.17:0/1937231281 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 0x7fccec19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:51360/0 (socket says 172.21.15.17:51360) 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccec106520 0x7fccec19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 -- 172.21.15.17:0/1937231281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccec106520 msgr2=0x7fccec19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccec106520 0x7fccec19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 -- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec197760 msgr2=0x7fccec197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec197760 0x7fccec197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 -- 172.21.15.17:0/1937231281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0009560 con 0x7fccec101450 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccec106520 0x7fccec19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccf0f02640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec197760 0x7fccec197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fccebfff640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 0x7fccec19d650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.078+0000 7fcce97fa640 1 -- 172.21.15.17:0/1937231281 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00ccc0 con 0x7fccec101450 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.079+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec1a44b0 con 0x7fccec101450 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.079+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1a4990 con 0x7fccec101450 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.079+0000 7fcce97fa640 1 -- 172.21.15.17:0/1937231281 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccdc00ce20 con 0x7fccec101450 2024-02-14T15:16:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.079+0000 7fcce97fa640 1 -- 172.21.15.17:0/1937231281 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0049a0 con 0x7fccec101450 2024-02-14T15:16:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fcce97fa640 1 -- 172.21.15.17:0/1937231281 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fccdc004b00 con 0x7fccec101450 2024-02-14T15:16:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 Processor -- start 2024-02-14T15:16:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 -- start start 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf052b9e0 con 0x556cf05bdee0 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf0528f10 con 0x556cf05cb030 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf5615740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf05246f0 con 0x556cf05bd530 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf0f02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41572/0 (socket says 172.21.15.17:41572) 2024-02-14T15:16:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccf0f02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05bd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:51362/0 (socket says 172.21.15.17:51362) 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fccebfff640 1 -- 172.21.15.17:0/300547918 learned_addr learned my addr 172.21.15.17:0/300547918 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.081+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fcce97fa640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccd40a22a0 0x7fccd40a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccf0f02640 1 -- 172.21.15.17:0/300547918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 msgr2=0x556cf05cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccf0f02640 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccf0f02640 1 -- 172.21.15.17:0/300547918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 msgr2=0x556cf05caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccf0f02640 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccf0f02640 1 -- 172.21.15.17:0/300547918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf0235920 con 0x556cf05bd530 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccebfff640 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccf0f02640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccd40a22a0 0x7fccd40a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fcce97fa640 1 -- 172.21.15.17:0/1937231281 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fccdc014070 con 0x7fccec101450 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccf0f02640 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05bd910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcce40118d0 tx=0x7fcce4011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d9dc72aecb153e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccd27fc640 1 -- 172.21.15.17:0/300547918 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce401bad0 con 0x556cf05bd530 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccd27fc640 1 -- 172.21.15.17:0/300547918 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcce401bc30 con 0x556cf05bd530 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.082+0000 7fccd27fc640 1 -- 172.21.15.17:0/300547918 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400e5b0 con 0x556cf05bd530 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 -- 172.21.15.17:0/300547918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 msgr2=0x556cf05bd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf0f02640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccd40a22a0 0x7fccd40a4760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcce4023000 tx=0x7fcce4024000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05bd910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcce40118d0 tx=0x7fcce4011d90 comp rx=0 tx=0).stop 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 -- 172.21.15.17:0/300547918 shutdown_connections 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 --2- 172.21.15.17:0/300547918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05bd910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 -- 172.21.15.17:0/300547918 >> 172.21.15.17:0/300547918 conn(0x556cf05b7e50 msgr2=0x556cf05b8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 -- 172.21.15.17:0/300547918 shutdown_connections 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 -- 172.21.15.17:0/300547918 wait complete. 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 -- start start 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.083+0000 7fccf5615740 1 Processor -- start 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf5615740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bd530 0x556cf05ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf5615740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf5615740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05cb030 0x556cf05c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf5615740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf04c0fa0 con 0x556cf05bdee0 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf5615740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf01f8250 con 0x556cf05bd530 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf5615740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf04aa200 con 0x556cf05cb030 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf5615740 10 client.0 _ll_register_callbacks cb 0x556cf0519b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bd530 0x556cf05ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41580/0 (socket says 172.21.15.17:41580) 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf0f02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05cb030 0x556cf05c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fcceb7fe640 1 -- 172.21.15.17:0/4089293742 learned_addr learned my addr 172.21.15.17:0/4089293742 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bd530 0x556cf05ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:51300/0 (socket says 172.21.15.17:51300) 2024-02-14T15:16:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 -- 172.21.15.17:0/4089293742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05cb030 msgr2=0x556cf05c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05cb030 0x556cf05c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 -- 172.21.15.17:0/4089293742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 msgr2=0x556cf05c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 -- 172.21.15.17:0/4089293742 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce4011490 con 0x556cf05bd530 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccf0f02640 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05cb030 0x556cf05c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fccebfff640 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bd530 0x556cf05ca870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0cbbd0 tx=0x7fccdc009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b4a0b408745310 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.084+0000 7fcccb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fcccb7fe640 1 -- 172.21.15.17:0/4089293742 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0ce130 con 0x556cf05bd530 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cf0224600 con 0x556cf05bd530 2024-02-14T15:16:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fcccb7fe640 1 -- 172.21.15.17:0/4089293742 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fccdc0d1e20 con 0x556cf05bd530 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fcccb7fe640 1 -- 172.21.15.17:0/4089293742 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccdc09a640 con 0x556cf05bd530 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 10 client.26708 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556cf0522d70 con 0x556cf05bd530 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fcccb7fe640 1 -- 172.21.15.17:0/4089293742 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556cf0224600 con 0x556cf05bd530 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fcccb7fe640 1 -- 172.21.15.17:0/4089293742 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccdc0cbd20 con 0x556cf05bd530 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 10 client.26708 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 10 client.26708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 20 client.26708 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf05dad20 0x556cf05dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 4 client.26708 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.085+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cf02751b0 con 0x556cf05dad20 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.086+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf05dad20 0x556cf05dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.086+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf05dad20 0x556cf05dd160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7157b3ee246e894d server_cookie=8a5875d141de5add in_seq=0 out_seq=0 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.086+0000 7fcccb7fe640 10 client.26708 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.086+0000 7fcccb7fe640 1 -- 172.21.15.17:0/4089293742 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556cf02751b0 con 0x556cf05dad20 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.086+0000 7fcccb7fe640 10 client.26708 handle_command_reply: tid=0 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 client.26708 shutdown 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bd530 msgr2=0x556cf05ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bd530 0x556cf05ca870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0cbbd0 tx=0x7fccdc009f60 comp rx=0 tx=0).stop 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 shutdown_connections 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf05dad20 0x556cf05dd160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05cb030 0x556cf05c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fcccb7fe640 0 client.26708 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 --2- 172.21.15.17:0/4089293742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bd530 0x556cf05ca870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 >> 172.21.15.17:0/4089293742 conn(0x556cf05b7e50 msgr2=0x556cf05b9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 shutdown_connections 2024-02-14T15:16:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 1 -- 172.21.15.17:0/4089293742 wait complete. 2024-02-14T15:16:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.087+0000 7fccf5615740 20 client.26708 trim_cache size 0 max 16384 2024-02-14T15:16:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.100+0000 7fccf5615740 1 Processor -- start 2024-02-14T15:16:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.100+0000 7fccf5615740 1 -- start start 2024-02-14T15:16:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf5615740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf5615740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf5615740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf5615740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf05b30e0 con 0x556cf05bdee0 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf5615740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf05e2350 con 0x556cf05cb030 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf5615740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf05c61a0 con 0x556cf05bd530 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41588/0 (socket says 172.21.15.17:41588) 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 -- 172.21.15.17:0/2448052444 learned_addr learned my addr 172.21.15.17:0/2448052444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccebfff640 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf0f02640 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 -- 172.21.15.17:0/2448052444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 msgr2=0x556cf05ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 -- 172.21.15.17:0/2448052444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 msgr2=0x556cf05c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 -- 172.21.15.17:0/2448052444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf04bf820 con 0x556cf05bdee0 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccebfff640 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccf0f02640 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05ca870 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fcce0036040 tx=0x7fcce00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a33a214fedefc31 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.101+0000 7fccd1ffb640 1 -- 172.21.15.17:0/2448052444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0048de0 con 0x556cf05bdee0 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccd1ffb640 1 -- 172.21.15.17:0/2448052444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcce002c980 con 0x556cf05bdee0 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccd1ffb640 1 -- 172.21.15.17:0/2448052444 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004df0 con 0x556cf05bdee0 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 -- 172.21.15.17:0/2448052444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 msgr2=0x556cf05ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05ca870 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fcce0036040 tx=0x7fcce00058b0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 -- 172.21.15.17:0/2448052444 shutdown_connections 2024-02-14T15:16:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05bdee0 0x556cf05ca870 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05cb030 0x556cf05c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 --2- 172.21.15.17:0/2448052444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 -- 172.21.15.17:0/2448052444 >> 172.21.15.17:0/2448052444 conn(0x556cf05b7e50 msgr2=0x556cf05c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 -- 172.21.15.17:0/2448052444 shutdown_connections 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.102+0000 7fccf5615740 1 -- 172.21.15.17:0/2448052444 wait complete. 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 -- start start 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 Processor -- start 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bdee0 0x556cf05d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05cb030 0x556cf05d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf05d0bc0 con 0x556cf05cb030 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf05d0d30 con 0x556cf05bdee0 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf05d0ef0 con 0x556cf05bd530 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf5615740 10 client.0 _ll_register_callbacks cb 0x7fccdc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf0f02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05cb030 0x556cf05d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf0f02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05cb030 0x556cf05d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41600/0 (socket says 172.21.15.17:41600) 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf0f02640 1 -- 172.21.15.17:0/258737481 learned_addr learned my addr 172.21.15.17:0/258737481 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bdee0 0x556cf05d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccebfff640 1 -- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bdee0 msgr2=0x556cf05d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccebfff640 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bdee0 0x556cf05d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccebfff640 1 -- 172.21.15.17:0/258737481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05cb030 msgr2=0x556cf05d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccebfff640 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05cb030 0x556cf05d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccebfff640 1 -- 172.21.15.17:0/258737481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cf0522d70 con 0x556cf05bd530 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.103+0000 7fccf0f02640 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05cb030 0x556cf05d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccebfff640 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05d1ec0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccdc098b20 tx=0x7fccdc0cbf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b53da596ef99a1d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fcccb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fcccb7fe640 1 -- 172.21.15.17:0/258737481 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0ce130 con 0x556cf05bd530 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fcccb7fe640 1 -- 172.21.15.17:0/258737481 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fccdc09af70 con 0x556cf05bd530 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcce401bc30 con 0x556cf05bd530 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fcccb7fe640 1 -- 172.21.15.17:0/258737481 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccdc09a640 con 0x556cf05bd530 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 10 client.26890 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fcce402d040 con 0x556cf05bd530 2024-02-14T15:16:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fcccb7fe640 1 -- 172.21.15.17:0/258737481 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcce401bc30 con 0x556cf05bd530 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fcccb7fe640 1 -- 172.21.15.17:0/258737481 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccdc099cc0 con 0x556cf05bd530 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 10 client.26890 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 10 client.26890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 20 client.26890 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf0632120 0x556cf0634510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.104+0000 7fccf5615740 4 client.26890 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.105+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556cf0261550 con 0x556cf0632120 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.105+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf0632120 0x556cf0634510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.105+0000 7fcccb7fe640 10 client.26890 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.105+0000 7fcceb7fe640 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf0632120 0x556cf0634510 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694502adbe5342e0 server_cookie=879704c5f91b09dc in_seq=0 out_seq=0 2024-02-14T15:16:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.117+0000 7fcccb7fe640 1 -- 172.21.15.17:0/258737481 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556cf0261550 con 0x556cf0632120 2024-02-14T15:16:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.117+0000 7fcccb7fe640 10 client.26890 handle_command_reply: tid=0 2024-02-14T15:16:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.117+0000 7fccf5615740 1 client.26890 shutdown 2024-02-14T15:16:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 msgr2=0x556cf05d1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05d1ec0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccdc098b20 tx=0x7fccdc0cbf50 comp rx=0 tx=0).stop 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 shutdown_connections 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x556cf0632120 0x556cf0634510 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556cf05cb030 0x556cf05d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556cf05bdee0 0x556cf05d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 --2- 172.21.15.17:0/258737481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556cf05bd530 0x556cf05d1ec0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fcccb7fe640 0 client.26890 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 >> 172.21.15.17:0/258737481 conn(0x556cf05b7e50 msgr2=0x556cf05c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 shutdown_connections 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 1 -- 172.21.15.17:0/258737481 wait complete. 2024-02-14T15:16:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf5615740 20 client.26890 trim_cache size 0 max 16384 2024-02-14T15:16:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:16:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:16:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:16:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:16:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccd40a22a0 msgr2=0x7fccd40a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccd40a22a0 0x7fccd40a4760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcce4023000 tx=0x7fcce4024000 comp rx=0 tx=0).stop 2024-02-14T15:16:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 msgr2=0x7fccec19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 0x7fccec19d650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00eeb0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 shutdown_connections 2024-02-14T15:16:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fccd40a22a0 0x7fccd40a4760 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccec197760 0x7fccec197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccec106520 0x7fccec19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 --2- 172.21.15.17:0/1937231281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccec101450 0x7fccec19d650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.118+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 >> 172.21.15.17:0/1937231281 conn(0x7fccec0fccf0 msgr2=0x7fccec0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.119+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 shutdown_connections 2024-02-14T15:16:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:35.119+0000 7fccf2965640 1 -- 172.21.15.17:0/1937231281 wait complete. 2024-02-14T15:16:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:34 smithi050 ceph-mon[36512]: pgmap v414: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 172 op/s 2024-02-14T15:16:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:34 smithi076 ceph-mon[37057]: pgmap v414: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 172 op/s 2024-02-14T15:16:35.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:34 smithi017 ceph-mon[32549]: pgmap v414: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 172 op/s 2024-02-14T15:16:35.896 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1357723 10.28 MB/sec execute 577 sec latency 56.044 ms 2024-02-14T15:16:36.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1360161 10.29 MB/sec execute 578 sec latency 21.673 ms 2024-02-14T15:16:37.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:36 smithi050 ceph-mon[36512]: pgmap v415: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 172 op/s 2024-02-14T15:16:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:36 smithi076 ceph-mon[37057]: pgmap v415: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 172 op/s 2024-02-14T15:16:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:36 smithi017 ceph-mon[32549]: pgmap v415: 129 pgs: 129 active+clean; 599 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 172 op/s 2024-02-14T15:16:37.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1361355 10.28 MB/sec execute 579 sec latency 29.200 ms 2024-02-14T15:16:38.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1363708 10.29 MB/sec execute 580 sec latency 23.608 ms 2024-02-14T15:16:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:38 smithi050 ceph-mon[36512]: pgmap v416: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 255 op/s 2024-02-14T15:16:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:38 smithi076 ceph-mon[37057]: pgmap v416: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 255 op/s 2024-02-14T15:16:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:38 smithi017 ceph-mon[32549]: pgmap v416: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.4 MiB/s wr, 255 op/s 2024-02-14T15:16:39.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1364614 10.28 MB/sec execute 581 sec latency 69.304 ms 2024-02-14T15:16:40.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1366632 10.28 MB/sec execute 582 sec latency 74.306 ms 2024-02-14T15:16:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:40 smithi050 ceph-mon[36512]: pgmap v417: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 167 op/s 2024-02-14T15:16:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:40 smithi076 ceph-mon[37057]: pgmap v417: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 167 op/s 2024-02-14T15:16:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:40 smithi017 ceph-mon[32549]: pgmap v417: 129 pgs: 129 active+clean; 613 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 167 op/s 2024-02-14T15:16:41.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1367920 10.27 MB/sec execute 583 sec latency 37.006 ms 2024-02-14T15:16:42.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1370369 10.28 MB/sec execute 584 sec latency 15.890 ms 2024-02-14T15:16:43.067 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:42 smithi017 ceph-mon[32549]: pgmap v418: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:42 smithi050 ceph-mon[36512]: pgmap v418: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:43.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:42 smithi076 ceph-mon[37057]: pgmap v418: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:43.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1371448 10.27 MB/sec execute 585 sec latency 18.857 ms 2024-02-14T15:16:44.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1373748 10.27 MB/sec execute 586 sec latency 13.772 ms 2024-02-14T15:16:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:44 smithi050 ceph-mon[36512]: pgmap v419: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:44 smithi076 ceph-mon[37057]: pgmap v419: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:44 smithi017 ceph-mon[32549]: pgmap v419: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:45.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1374713 10.27 MB/sec execute 587 sec latency 75.118 ms 2024-02-14T15:16:46.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1375708 10.29 MB/sec execute 588 sec latency 65.707 ms 2024-02-14T15:16:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:46 smithi050 ceph-mon[36512]: pgmap v420: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:46 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:16:47.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:46 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:16:47.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:46 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:16:47.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:46 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:16:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:46 smithi076 ceph-mon[37057]: pgmap v420: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:46 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:16:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:46 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:16:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:46 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:16:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:46 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:16:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:46 smithi017 ceph-mon[32549]: pgmap v420: 129 pgs: 129 active+clean; 625 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 165 op/s 2024-02-14T15:16:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:46 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:16:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:46 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:16:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:46 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:16:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:46 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:16:47.893 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1378373 10.29 MB/sec execute 589 sec latency 12.847 ms 2024-02-14T15:16:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 -- 172.21.15.17:0/4227752941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 msgr2=0x7fe92010ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 --2- 172.21.15.17:0/4227752941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe92010ffc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe918009ca0 tx=0x7fe91802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 -- 172.21.15.17:0/4227752941 shutdown_connections 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 --2- 172.21.15.17:0/4227752941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe92010ffc0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 --2- 172.21.15.17:0/4227752941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9200693f0 0x7fe92010d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 --2- 172.21.15.17:0/4227752941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920068ad0 0x7fe920068eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 -- 172.21.15.17:0/4227752941 >> 172.21.15.17:0/4227752941 conn(0x7fe920077220 msgr2=0x7fe920077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 -- 172.21.15.17:0/4227752941 shutdown_connections 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.209+0000 7fe927dc9640 1 -- 172.21.15.17:0/4227752941 wait complete. 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 Processor -- start 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 -- start start 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920068ad0 0x7fe92019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9200693f0 0x7fe92019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe9201997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920199e70 con 0x7fe92010dbd0 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920199fe0 con 0x7fe9200693f0 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe927dc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92019a1a0 con 0x7fe920068ad0 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe9201997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe925b65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920068ad0 0x7fe92019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe9201997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60640/0 (socket says 172.21.15.17:60640) 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 -- 172.21.15.17:0/3889574935 learned_addr learned my addr 172.21.15.17:0/3889574935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe925364640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9200693f0 0x7fe92019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 -- 172.21.15.17:0/3889574935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920068ad0 msgr2=0x7fe92019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920068ad0 0x7fe92019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 -- 172.21.15.17:0/3889574935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9200693f0 msgr2=0x7fe92019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9200693f0 0x7fe92019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe926366640 1 -- 172.21.15.17:0/3889574935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918009900 con 0x7fe92010dbd0 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.210+0000 7fe925364640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9200693f0 0x7fe92019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.211+0000 7fe926366640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe9201997d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe91802c700 tx=0x7fe918034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.211+0000 7fe916ffd640 1 -- 172.21.15.17:0/3889574935 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180328b0 con 0x7fe92010dbd0 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.211+0000 7fe916ffd640 1 -- 172.21.15.17:0/3889574935 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe918003e50 con 0x7fe92010dbd0 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.211+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92019a420 con 0x7fe92010dbd0 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.211+0000 7fe916ffd640 1 -- 172.21.15.17:0/3889574935 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180047f0 con 0x7fe92010dbd0 2024-02-14T15:16:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.211+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9201a6690 con 0x7fe92010dbd0 2024-02-14T15:16:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.213+0000 7fe916ffd640 1 -- 172.21.15.17:0/3889574935 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe91800a040 con 0x7fe92010dbd0 2024-02-14T15:16:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.213+0000 7fe916ffd640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe90c0a22a0 0x7fe90c0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.213+0000 7fe925b65640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe90c0a22a0 0x7fe90c0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe916ffd640 1 -- 172.21.15.17:0/3889574935 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe9180f1070 con 0x7fe92010dbd0 2024-02-14T15:16:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 Processor -- start 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 -- start start 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe925b65640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe90c0a22a0 0x7fe90c0a4760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009bd0 tx=0x7fe91c005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1edb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a15b9e0 con 0x55f23a1f6c20 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a158f10 con 0x55f23a1ed720 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe92a98a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a1546f0 con 0x55f23a1ee040 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe925364640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60652/0 (socket says 172.21.15.17:60652) 2024-02-14T15:16:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe926366640 1 -- 172.21.15.17:0/3211227836 learned_addr learned my addr 172.21.15.17:0/3211227836 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.214+0000 7fe925b65640 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe925364640 1 -- 172.21.15.17:0/3211227836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 msgr2=0x55f23a1edb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe925364640 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1edb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe925364640 1 -- 172.21.15.17:0/3211227836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 msgr2=0x55f23a1f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe925364640 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe925364640 1 -- 172.21.15.17:0/3211227836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f239e65920 con 0x55f23a1ee040 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe925364640 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f66e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe92019ab20 tx=0x7fe910010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5d8a563ad65b68 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe90bfff640 1 -- 172.21.15.17:0/3211227836 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910009e50 con 0x55f23a1ee040 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe90bfff640 1 -- 172.21.15.17:0/3211227836 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe910004510 con 0x55f23a1ee040 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.215+0000 7fe90bfff640 1 -- 172.21.15.17:0/3211227836 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100126b0 con 0x55f23a1ee040 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 -- 172.21.15.17:0/3211227836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 msgr2=0x55f23a1f66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f66e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe92019ab20 tx=0x7fe910010740 comp rx=0 tx=0).stop 2024-02-14T15:16:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 -- 172.21.15.17:0/3211227836 shutdown_connections 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f66e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 --2- 172.21.15.17:0/3211227836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1edb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 -- 172.21.15.17:0/3211227836 >> 172.21.15.17:0/3211227836 conn(0x55f23a1e7e50 msgr2=0x55f23a1e8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 -- 172.21.15.17:0/3211227836 shutdown_connections 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 -- 172.21.15.17:0/3211227836 wait complete. 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.216+0000 7fe92a98a740 1 -- start start 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 1 Processor -- start 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a0f0fa0 con 0x55f23a1f6c20 2024-02-14T15:16:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f239e28250 con 0x55f23a1ed720 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a0da200 con 0x55f23a1ee040 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe92a98a740 10 client.0 _ll_register_callbacks cb 0x55f23a149b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60666/0 (socket says 172.21.15.17:60666) 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe925364640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 -- 172.21.15.17:0/2327796716 learned_addr learned my addr 172.21.15.17:0/2327796716 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe925b65640 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 -- 172.21.15.17:0/2327796716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 msgr2=0x55f23a1f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 -- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 msgr2=0x55f23a1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe926366640 1 -- 172.21.15.17:0/2327796716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9100097f0 con 0x55f23a1f6c20 2024-02-14T15:16:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.217+0000 7fe925364640 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe926366640 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f3f80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe918009430 tx=0x7fe9180ee4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39d017bc328adde server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe9097fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:16:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe9097fa640 1 -- 172.21.15.17:0/2327796716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180bd750 con 0x55f23a1f6c20 2024-02-14T15:16:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe9097fa640 1 -- 172.21.15.17:0/2327796716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe9180c18a0 con 0x55f23a1f6c20 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f239e54600 con 0x55f23a1f6c20 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe9097fa640 1 -- 172.21.15.17:0/2327796716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9180c5ef0 con 0x55f23a1f6c20 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 10 client.17391 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f23a152d70 con 0x55f23a1f6c20 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe9097fa640 1 -- 172.21.15.17:0/2327796716 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f239e54600 con 0x55f23a1f6c20 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe9097fa640 1 -- 172.21.15.17:0/2327796716 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9180c5a60 con 0x55f23a1f6c20 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 10 client.17391 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 20 client.17391 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a206660 0x55f23a208a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 4 client.17391 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.218+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f239ea51b0 con 0x55f23a206660 2024-02-14T15:16:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.219+0000 7fe925b65640 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a206660 0x55f23a208a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.219+0000 7fe9097fa640 10 client.17391 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.219+0000 7fe925b65640 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a206660 0x55f23a208a50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe2445d18666971 server_cookie=8b8c566926ccdba in_seq=0 out_seq=0 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.219+0000 7fe9097fa640 1 -- 172.21.15.17:0/2327796716 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f239ea51b0 con 0x55f23a206660 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.219+0000 7fe9097fa640 10 client.17391 handle_command_reply: tid=0 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 client.17391 shutdown 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 msgr2=0x55f23a1f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f3f80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe918009430 tx=0x7fe9180ee4d0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 shutdown_connections 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a206660 0x55f23a208a50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a1f3f80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ee040 0x55f23a1f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe9097fa640 0 client.17391 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2327796716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ed720 0x55f23a1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 >> 172.21.15.17:0/2327796716 conn(0x55f23a1e7e50 msgr2=0x55f23a1e9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 shutdown_connections 2024-02-14T15:16:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 1 -- 172.21.15.17:0/2327796716 wait complete. 2024-02-14T15:16:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.220+0000 7fe92a98a740 20 client.17391 trim_cache size 0 max 16384 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 Processor -- start 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 -- start start 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a20d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1ee040 0x55f23a1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1f6c20 0x55f23a1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a1e8e50 con 0x55f23a1ee040 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a202e10 con 0x55f23a1f6c20 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe92a98a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a1e87d0 con 0x55f23a1ed720 2024-02-14T15:16:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925364640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1ee040 0x55f23a1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925b65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a20d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925364640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1ee040 0x55f23a1f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60670/0 (socket says 172.21.15.17:60670) 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925364640 1 -- 172.21.15.17:0/2537678269 learned_addr learned my addr 172.21.15.17:0/2537678269 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe926366640 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1f6c20 0x55f23a1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925b65640 1 -- 172.21.15.17:0/2537678269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1f6c20 msgr2=0x55f23a1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925b65640 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1f6c20 0x55f23a1f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925b65640 1 -- 172.21.15.17:0/2537678269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1ee040 msgr2=0x55f23a1f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925b65640 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1ee040 0x55f23a1f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925b65640 1 -- 172.21.15.17:0/2537678269 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f23a0ef820 con 0x55f23a1ed720 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.235+0000 7fe925364640 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1ee040 0x55f23a1f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.236+0000 7fe925b65640 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a20d350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55f23a1f31d0 tx=0x7fe91c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb3331a7ef5c170 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.236+0000 7fe90b7fe640 1 -- 172.21.15.17:0/2537678269 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c022030 con 0x55f23a1ed720 2024-02-14T15:16:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.236+0000 7fe90b7fe640 1 -- 172.21.15.17:0/2537678269 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91c023040 con 0x55f23a1ed720 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.236+0000 7fe90b7fe640 1 -- 172.21.15.17:0/2537678269 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c01d6b0 con 0x55f23a1ed720 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 -- 172.21.15.17:0/2537678269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 msgr2=0x55f23a20d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a20d350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55f23a1f31d0 tx=0x7fe91c014000 comp rx=0 tx=0).stop 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 -- 172.21.15.17:0/2537678269 shutdown_connections 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1ee040 0x55f23a1f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1f6c20 0x55f23a1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 --2- 172.21.15.17:0/2537678269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a20d350 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 -- 172.21.15.17:0/2537678269 >> 172.21.15.17:0/2537678269 conn(0x55f23a1e7e50 msgr2=0x55f23a1f2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 -- 172.21.15.17:0/2537678269 shutdown_connections 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 -- 172.21.15.17:0/2537678269 wait complete. 2024-02-14T15:16:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 -- start start 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.237+0000 7fe92a98a740 1 Processor -- start 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a1fc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ee040 0x55f23a1fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a25cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a1fd7e0 con 0x55f23a1f6c20 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a25d020 con 0x55f23a1ee040 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a25d160 con 0x55f23a1ed720 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 10 client.0 _ll_register_callbacks cb 0x7fe910009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a25cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe925b65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a1fc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a25cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60678/0 (socket says 172.21.15.17:60678) 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 -- 172.21.15.17:0/64932655 learned_addr learned my addr 172.21.15.17:0/64932655 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:16:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe925364640 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ee040 0x55f23a1fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 -- 172.21.15.17:0/64932655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 msgr2=0x55f23a1fc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a1fc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 -- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ee040 msgr2=0x55f23a1fcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ee040 0x55f23a1fcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 -- 172.21.15.17:0/64932655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f239e65920 con 0x55f23a1f6c20 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe926366640 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a25cae0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe9180c5dc0 tx=0x7fe918032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae51d83bc551b3b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe9097fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe9097fa640 1 -- 172.21.15.17:0/64932655 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180b99f0 con 0x55f23a1f6c20 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe9097fa640 1 -- 172.21.15.17:0/64932655 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe9180c18a0 con 0x55f23a1f6c20 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.238+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe910004510 con 0x55f23a1f6c20 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe9097fa640 1 -- 172.21.15.17:0/64932655 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9180dd0b0 con 0x55f23a1f6c20 2024-02-14T15:16:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 10 client.17397 fetch_fsmap learned FSMap version 35 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fe91c017070 con 0x55f23a1f6c20 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe9097fa640 1 -- 172.21.15.17:0/64932655 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe910004510 con 0x55f23a1f6c20 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe9097fa640 1 -- 172.21.15.17:0/64932655 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9180b9dc0 con 0x55f23a1f6c20 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 10 client.17397 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 10 client.17397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 20 client.17397 populate_metadata read hostname 'smithi017' 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a25e390 0x55f23a260780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 4 client.17397 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f239e91550 con 0x55f23a25e390 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.239+0000 7fe925b65640 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a25e390 0x55f23a260780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.240+0000 7fe925b65640 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a25e390 0x55f23a260780 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228b12da26d9a7bf server_cookie=c78a04cd4e60bb4 in_seq=0 out_seq=0 2024-02-14T15:16:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.240+0000 7fe9097fa640 10 client.17397 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe9097fa640 1 -- 172.21.15.17:0/64932655 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f239e91550 con 0x55f23a25e390 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe9097fa640 10 client.17397 handle_command_reply: tid=0 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 client.17397 shutdown 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 msgr2=0x55f23a25cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a25cae0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe9180c5dc0 tx=0x7fe918032be0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 shutdown_connections 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55f23a25e390 0x55f23a260780 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f23a1f6c20 0x55f23a25cae0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f23a1ee040 0x55f23a1fcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe9097fa640 0 client.17397 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:16:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 --2- 172.21.15.17:0/64932655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f23a1ed720 0x55f23a1fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 >> 172.21.15.17:0/64932655 conn(0x55f23a1e7e50 msgr2=0x55f23a1e6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 shutdown_connections 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 1 -- 172.21.15.17:0/64932655 wait complete. 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.252+0000 7fe92a98a740 20 client.17397 trim_cache size 0 max 16384 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:16:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:16:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe90c0a22a0 msgr2=0x7fe90c0a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe90c0a22a0 0x7fe90c0a4760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009bd0 tx=0x7fe91c005f90 comp rx=0 tx=0).stop 2024-02-14T15:16:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 msgr2=0x7fe9201997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:16:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe9201997d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe91802c700 tx=0x7fe918034a90 comp rx=0 tx=0).stop 2024-02-14T15:16:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 shutdown_connections 2024-02-14T15:16:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe90c0a22a0 0x7fe90c0a4760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe92010dbd0 0x7fe9201997d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9200693f0 0x7fe92019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 --2- 172.21.15.17:0/3889574935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920068ad0 0x7fe92019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:16:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 >> 172.21.15.17:0/3889574935 conn(0x7fe920077220 msgr2=0x7fe92010efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:16:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 shutdown_connections 2024-02-14T15:16:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:16:48.253+0000 7fe927dc9640 1 -- 172.21.15.17:0/3889574935 wait complete. 2024-02-14T15:16:48.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1379976 10.29 MB/sec execute 590 sec latency 16.137 ms 2024-02-14T15:16:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:48 smithi050 ceph-mon[36512]: pgmap v421: 129 pgs: 129 active+clean; 637 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.2 MiB/s wr, 250 op/s 2024-02-14T15:16:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:48 smithi076 ceph-mon[37057]: pgmap v421: 129 pgs: 129 active+clean; 637 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.2 MiB/s wr, 250 op/s 2024-02-14T15:16:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:48 smithi017 ceph-mon[32549]: pgmap v421: 129 pgs: 129 active+clean; 637 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.2 MiB/s wr, 250 op/s 2024-02-14T15:16:49.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1382286 10.30 MB/sec execute 591 sec latency 15.332 ms 2024-02-14T15:16:50.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1383569 10.29 MB/sec execute 592 sec latency 94.101 ms 2024-02-14T15:16:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:50 smithi050 ceph-mon[36512]: pgmap v422: 129 pgs: 129 active+clean; 637 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-02-14T15:16:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:50 smithi076 ceph-mon[37057]: pgmap v422: 129 pgs: 129 active+clean; 637 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-02-14T15:16:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:50 smithi017 ceph-mon[32549]: pgmap v422: 129 pgs: 129 active+clean; 637 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-02-14T15:16:51.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1385146 10.29 MB/sec execute 593 sec latency 154.627 ms 2024-02-14T15:16:52.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1386899 10.28 MB/sec execute 594 sec latency 18.212 ms 2024-02-14T15:16:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:52 smithi050 ceph-mon[36512]: pgmap v423: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:53.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:52 smithi076 ceph-mon[37057]: pgmap v423: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:52 smithi017 ceph-mon[32549]: pgmap v423: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 249 op/s 2024-02-14T15:16:53.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1389336 10.29 MB/sec execute 595 sec latency 18.796 ms 2024-02-14T15:16:54.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1390886 10.29 MB/sec execute 596 sec latency 19.250 ms 2024-02-14T15:16:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:54 smithi050 ceph-mon[36512]: pgmap v424: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-02-14T15:16:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:54 smithi076 ceph-mon[37057]: pgmap v424: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-02-14T15:16:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:54 smithi017 ceph-mon[32549]: pgmap v424: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-02-14T15:16:55.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1392903 10.29 MB/sec execute 597 sec latency 97.181 ms 2024-02-14T15:16:56.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1394177 10.28 MB/sec execute 598 sec latency 71.021 ms 2024-02-14T15:16:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:56 smithi050 ceph-mon[36512]: pgmap v425: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-02-14T15:16:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:56 smithi076 ceph-mon[37057]: pgmap v425: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-02-14T15:16:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:56 smithi017 ceph-mon[32549]: pgmap v425: 129 pgs: 129 active+clean; 574 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 167 op/s 2024-02-14T15:16:57.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 1396504 10.29 MB/sec execute 599 sec latency 27.167 ms 2024-02-14T15:16:58.894 INFO:tasks.workunit.client.0.smithi017.stdout: 1 cleanup 600 sec 2024-02-14T15:16:59.038 INFO:tasks.workunit.client.0.smithi017.stdout: 0 cleanup 600 sec 2024-02-14T15:16:59.038 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-02-14T15:16:59.038 INFO:tasks.workunit.client.0.smithi017.stdout: Operation Count AvgLat MaxLat 2024-02-14T15:16:59.038 INFO:tasks.workunit.client.0.smithi017.stdout: ---------------------------------------- 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: NTCreateX 196202 1.107 314.676 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Close 144154 0.003 0.219 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Rename 8308 5.010 179.522 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Unlink 39594 1.608 95.442 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Deltree 6 74.476 112.480 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Mkdir 3 0.009 0.015 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Qpathinfo 177779 1.019 5331.394 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Qfileinfo 31195 0.002 0.028 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Qfsinfo 32601 0.211 9.750 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Sfileinfo 15996 0.021 3.463 2024-02-14T15:16:59.039 INFO:tasks.workunit.client.0.smithi017.stdout: Find 68742 0.190 128.832 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stdout: WriteX 98010 0.084 109.362 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stdout: ReadX 307434 0.025 17.201 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stdout: LockX 638 0.908 16.341 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stdout: UnlockX 638 0.775 12.875 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stdout: Flush 13762 4.047 265.555 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stdout:Throughput 10.2875 MB/sec 1 clients 1 procs max_latency=5331.408 ms 2024-02-14T15:16:59.040 INFO:tasks.workunit.client.0.smithi017.stderr:+ dbench 10 2024-02-14T15:16:59.041 INFO:tasks.workunit.client.0.smithi017.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-14T15:16:59.041 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-02-14T15:16:59.041 INFO:tasks.workunit.client.0.smithi017.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-14T15:16:59.041 INFO:tasks.workunit.client.0.smithi017.stdout:8 of 10 processes prepared for launch 0 sec 2024-02-14T15:16:59.139 INFO:tasks.workunit.client.0.smithi017.stdout:10 of 10 processes prepared for launch 0 sec 2024-02-14T15:16:59.140 INFO:tasks.workunit.client.0.smithi017.stdout:releasing clients 2024-02-14T15:16:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:16:58 smithi050 ceph-mon[36512]: pgmap v426: 129 pgs: 129 active+clean; 586 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 246 op/s 2024-02-14T15:16:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:16:58 smithi076 ceph-mon[37057]: pgmap v426: 129 pgs: 129 active+clean; 586 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 246 op/s 2024-02-14T15:16:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:16:58 smithi017 ceph-mon[32549]: pgmap v426: 129 pgs: 129 active+clean; 586 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 246 op/s 2024-02-14T15:17:00.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 350 126.03 MB/sec warmup 1 sec latency 155.930 ms 2024-02-14T15:17:01.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 591 97.29 MB/sec warmup 2 sec latency 475.100 ms 2024-02-14T15:17:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:00 smithi050 ceph-mon[36512]: pgmap v427: 129 pgs: 129 active+clean; 586 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-02-14T15:17:01.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:00 smithi076 ceph-mon[37057]: pgmap v427: 129 pgs: 129 active+clean; 586 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-02-14T15:17:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:00 smithi017 ceph-mon[32549]: pgmap v427: 129 pgs: 129 active+clean; 586 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-02-14T15:17:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.409+0000 7f03c43a1640 1 -- 172.21.15.17:0/3129981177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 msgr2=0x7f03bc10ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.409+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3129981177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc10ffc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009be0 tx=0x7f03b402bf30 comp rx=0 tx=0).stop 2024-02-14T15:17:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.411+0000 7f03c43a1640 1 -- 172.21.15.17:0/3129981177 shutdown_connections 2024-02-14T15:17:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.411+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3129981177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03bc100940 0x7f03bc10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.411+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3129981177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc10ffc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.411+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3129981177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03bc100020 0x7f03bc100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.411+0000 7f03c43a1640 1 -- 172.21.15.17:0/3129981177 >> 172.21.15.17:0/3129981177 conn(0x7f03bc077250 msgr2=0x7f03bc077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.412+0000 7f03c43a1640 1 -- 172.21.15.17:0/3129981177 shutdown_connections 2024-02-14T15:17:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.413+0000 7f03c43a1640 1 -- 172.21.15.17:0/3129981177 wait complete. 2024-02-14T15:17:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.414+0000 7f03c43a1640 1 Processor -- start 2024-02-14T15:17:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.414+0000 7f03c43a1640 1 -- start start 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.414+0000 7f03c43a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03bc100020 0x7f03bc10d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c43a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03bc100940 0x7f03bc104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c43a1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc104c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c43a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc1051d0 con 0x7f03bc100020 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c43a1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc105340 con 0x7f03bc10dbd0 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c43a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc105500 con 0x7f03bc100940 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc104c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc104c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:51034/0 (socket says 172.21.15.17:51034) 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 -- 172.21.15.17:0/3102380209 learned_addr learned my addr 172.21.15.17:0/3102380209 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c193c640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03bc100940 0x7f03bc104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c213d640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03bc100020 0x7f03bc10d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 -- 172.21.15.17:0/3102380209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03bc100940 msgr2=0x7f03bc104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03bc100940 0x7f03bc104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 -- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03bc100020 msgr2=0x7f03bc10d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03bc100020 0x7f03bc10d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.415+0000 7f03c293e640 1 -- 172.21.15.17:0/3102380209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4009840 con 0x7f03bc10dbd0 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.416+0000 7f03c293e640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc104c90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03b4004af0 tx=0x7f03b402cae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.416+0000 7f03b37fe640 1 -- 172.21.15.17:0/3102380209 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4003cc0 con 0x7f03bc10dbd0 2024-02-14T15:17:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.416+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc108d00 con 0x7f03bc10dbd0 2024-02-14T15:17:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.416+0000 7f03b37fe640 1 -- 172.21.15.17:0/3102380209 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03b4003e20 con 0x7f03bc10dbd0 2024-02-14T15:17:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.416+0000 7f03b37fe640 1 -- 172.21.15.17:0/3102380209 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b403ea40 con 0x7f03bc10dbd0 2024-02-14T15:17:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.416+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc109260 con 0x7f03bc10dbd0 2024-02-14T15:17:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.421+0000 7f03c6f71740 1 Processor -- start 2024-02-14T15:17:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.422+0000 7f03b37fe640 1 -- 172.21.15.17:0/3102380209 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f03b4004690 con 0x7f03bc10dbd0 2024-02-14T15:17:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.422+0000 7f03c6f71740 1 -- start start 2024-02-14T15:17:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a13b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03b37fe640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f03a80a22f0 0x7f03a80a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03b37fe640 1 -- 172.21.15.17:0/3102380209 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f03b40f0cf0 con 0x7f03bc10dbd0 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c193c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a13b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c193c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a13b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:53632/0 (socket says 172.21.15.17:53632) 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a14040 0x5600a0a1c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a1cc20 0x5600a0a1f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c6f71740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a09819e0 con 0x5600a0a14040 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c6f71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a097ef10 con 0x5600a0a1cc20 2024-02-14T15:17:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c6f71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a097a6f0 con 0x5600a0a13720 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c193c640 1 -- 172.21.15.17:0/2004173206 learned_addr learned my addr 172.21.15.17:0/2004173206 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.423+0000 7f03c193c640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f03a80a22f0 0x7f03a80a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c293e640 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a1cc20 0x5600a0a1f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c213d640 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a14040 0x5600a0a1c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c193c640 1 -- 172.21.15.17:0/2004173206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a1cc20 msgr2=0x5600a0a1f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c193c640 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a1cc20 0x5600a0a1f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c193c640 1 -- 172.21.15.17:0/2004173206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a14040 msgr2=0x5600a0a1c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c193c640 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a14040 0x5600a0a1c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c193c640 1 -- 172.21.15.17:0/2004173206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a068b920 con 0x5600a0a13720 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c213d640 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a14040 0x5600a0a1c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c293e640 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a1cc20 0x5600a0a1f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:17:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.424+0000 7f03c193c640 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a13b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03b8012a40 tx=0x7f03b8012f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e56d04b21aee2eec server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.425+0000 7f03c193c640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f03a80a22f0 0x7f03a80a47b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f03b8010800 tx=0x7f03b8017000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.425+0000 7f03b0bfe640 1 -- 172.21.15.17:0/2004173206 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8012590 con 0x5600a0a13720 2024-02-14T15:17:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.425+0000 7f03b0bfe640 1 -- 172.21.15.17:0/2004173206 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03b800ba20 con 0x5600a0a13720 2024-02-14T15:17:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.425+0000 7f03b0bfe640 1 -- 172.21.15.17:0/2004173206 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b800da70 con 0x5600a0a13720 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.426+0000 7f03c6f71740 1 -- 172.21.15.17:0/2004173206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 msgr2=0x5600a0a13b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.426+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a13b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03b8012a40 tx=0x7f03b8012f00 comp rx=0 tx=0).stop 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.426+0000 7f03c6f71740 1 -- 172.21.15.17:0/2004173206 shutdown_connections 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.426+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a14040 0x5600a0a1c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.426+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a1cc20 0x5600a0a1f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.426+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2004173206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a13b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.426+0000 7f03c6f71740 1 -- 172.21.15.17:0/2004173206 >> 172.21.15.17:0/2004173206 conn(0x5600a0a0de50 msgr2=0x5600a0a0e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.427+0000 7f03c6f71740 1 -- 172.21.15.17:0/2004173206 shutdown_connections 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.427+0000 7f03c6f71740 1 -- 172.21.15.17:0/2004173206 wait complete. 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.427+0000 7f03c6f71740 1 -- start start 2024-02-14T15:17:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.428+0000 7f03c6f71740 1 Processor -- start 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.428+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a1c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.428+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a19a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c213d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a1c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c213d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a1c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38026/0 (socket says 172.21.15.17:38026) 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c213d640 1 -- 172.21.15.17:0/1069939671 learned_addr learned my addr 172.21.15.17:0/1069939671 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c193c640 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a19a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a1a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0916fa0 con 0x5600a0a13720 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a064e250 con 0x5600a0a14040 2024-02-14T15:17:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0900200 con 0x5600a0a1cc20 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c6f71740 10 client.0 _ll_register_callbacks cb 0x5600a096fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.429+0000 7f03c293e640 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a1a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03c293e640 1 -- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 msgr2=0x5600a0a19a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03c293e640 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a19a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03c293e640 1 -- 172.21.15.17:0/1069939671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 msgr2=0x5600a0a1c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03c293e640 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a1c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03c293e640 1 -- 172.21.15.17:0/1069939671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b801a040 con 0x5600a0a1cc20 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03c293e640 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a1a1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03b40ee190 tx=0x7f03b40bce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecaa3fd687e4fc66 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1069939671 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40c1320 con 0x5600a0a1cc20 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1069939671 <== mon.1 v2:172.21.15.76:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f03b40c5bd0 con 0x5600a0a1cc20 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.430+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a067a600 con 0x5600a0a1cc20 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1069939671 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03b40dd070 con 0x5600a0a1cc20 2024-02-14T15:17:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 10 client.26917 fetch_fsmap learned FSMap version 35 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5600a0978d70 con 0x5600a0a1cc20 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1069939671 <== mon.1 v2:172.21.15.76:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5600a067a600 con 0x5600a0a1cc20 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1069939671 <== mon.1 v2:172.21.15.76:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f03b4050060 con 0x5600a0a1cc20 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 10 client.26917 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 10 client.26917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 20 client.26917 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 4 client.26917 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:17:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.431+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600a06cb1b0 con 0x5600a0a2c300 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.432+0000 7f03c213d640 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a2e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.432+0000 7f03c213d640 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a2e6f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d37846412cbafb31 server_cookie=cec6d13182883166 in_seq=0 out_seq=0 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.432+0000 7f03a5ffb640 10 client.26917 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.433+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1069939671 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600a06cb1b0 con 0x5600a0a2c300 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.433+0000 7f03a5ffb640 10 client.26917 handle_command_reply: tid=0 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.433+0000 7f03c6f71740 1 client.26917 shutdown 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 msgr2=0x5600a0a1a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a1a1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03b40ee190 tx=0x7f03b40bce40 comp rx=0 tx=0).stop 2024-02-14T15:17:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 shutdown_connections 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a2e6f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a1a1a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03a5ffb640 0 client.26917 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a19a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1069939671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a1c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 >> 172.21.15.17:0/1069939671 conn(0x5600a0a0de50 msgr2=0x5600a0a0fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 shutdown_connections 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 1 -- 172.21.15.17:0/1069939671 wait complete. 2024-02-14T15:17:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.434+0000 7f03c6f71740 20 client.26917 trim_cache size 0 max 16384 2024-02-14T15:17:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.459+0000 7f03c6f71740 1 Processor -- start 2024-02-14T15:17:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.459+0000 7f03c6f71740 1 -- start start 2024-02-14T15:17:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.459+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a18a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.459+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a1c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.459+0000 7f03c213d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a18a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c213d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a18a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:53660/0 (socket says 172.21.15.17:53660) 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c193c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a1c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c193c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a1c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:51074/0 (socket says 172.21.15.17:51074) 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c213d640 1 -- 172.21.15.17:0/2698803569 learned_addr learned my addr 172.21.15.17:0/2698803569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a1cc20 0x5600a0a1c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0a095a0 con 0x5600a0a1cc20 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0a0f080 con 0x5600a0a14040 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0a33890 con 0x5600a0a13720 2024-02-14T15:17:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.460+0000 7f03c293e640 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a1cc20 0x5600a0a1c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03c293e640 1 -- 172.21.15.17:0/2698803569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 msgr2=0x5600a0a18a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03c293e640 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a18a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03c293e640 1 -- 172.21.15.17:0/2698803569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 msgr2=0x5600a0a1c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03c293e640 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a1c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03c293e640 1 -- 172.21.15.17:0/2698803569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a0915820 con 0x5600a0a1cc20 2024-02-14T15:17:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03c293e640 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a1cc20 0x5600a0a1c6d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f03b40bd470 tx=0x7f03b40bcf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225d2656198bbd68 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03a7fff640 1 -- 172.21.15.17:0/2698803569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40bd560 con 0x5600a0a1cc20 2024-02-14T15:17:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03a7fff640 1 -- 172.21.15.17:0/2698803569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03b40dd070 con 0x5600a0a1cc20 2024-02-14T15:17:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.461+0000 7f03a7fff640 1 -- 172.21.15.17:0/2698803569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40c1120 con 0x5600a0a1cc20 2024-02-14T15:17:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.462+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a1cc20 msgr2=0x5600a0a1c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.462+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a1cc20 0x5600a0a1c6d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f03b40bd470 tx=0x7f03b40bcf70 comp rx=0 tx=0).stop 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.462+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 shutdown_connections 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.462+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a1cc20 0x5600a0a1c6d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.462+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a1c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.462+0000 7f03c6f71740 1 --2- 172.21.15.17:0/2698803569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a13720 0x5600a0a18a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.462+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 >> 172.21.15.17:0/2698803569 conn(0x5600a0a0de50 msgr2=0x5600a0a18370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.463+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 shutdown_connections 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.463+0000 7f03c6f71740 1 -- 172.21.15.17:0/2698803569 wait complete. 2024-02-14T15:17:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.463+0000 7f03c6f71740 1 -- start start 2024-02-14T15:17:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.463+0000 7f03c6f71740 1 Processor -- start 2024-02-14T15:17:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.463+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a21c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c6f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c213d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a21c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c213d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a21c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38046/0 (socket says 172.21.15.17:38046) 2024-02-14T15:17:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c213d640 1 -- 172.21.15.17:0/1539479659 learned_addr learned my addr 172.21.15.17:0/1539479659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a81760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c193c640 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0a22850 con 0x5600a0a13720 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0a81ca0 con 0x5600a0a14040 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c293e640 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a81760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.464+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a0a81de0 con 0x5600a0a1cc20 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c213d640 1 -- 172.21.15.17:0/1539479659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 msgr2=0x5600a0a81760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c213d640 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a81760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c213d640 1 -- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 msgr2=0x5600a0a22180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c213d640 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a22180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c213d640 1 -- 172.21.15.17:0/1539479659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b801a040 con 0x5600a0a13720 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c6f71740 10 client.0 _ll_register_callbacks cb 0x7f03b800bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a0a28f20 con 0x5600a0a13720 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03c213d640 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a21c40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009dd0 tx=0x7f03ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a40a939ed4b5e69 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03a5ffb640 10 client.17415 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:17:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1539479659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac01c8f0 con 0x5600a0a13720 2024-02-14T15:17:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.465+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1539479659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f03ac016430 con 0x5600a0a13720 2024-02-14T15:17:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.466+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1539479659 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03ac016650 con 0x5600a0a13720 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.466+0000 7f03c6f71740 10 client.17415 fetch_fsmap learned FSMap version 35 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.466+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x5600a093b720 con 0x5600a0a13720 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1539479659 <== mon.0 v2:172.21.15.17:3300/0 4 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f03ac0167b0 con 0x5600a0a13720 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03c6f71740 10 client.17415 fetch_fsmap finished waiting for FSMap version 35 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03c6f71740 10 client.17415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03c6f71740 20 client.17415 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03c6f71740 4 client.17415 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f03b800ba20 con 0x5600a0a2c300 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.467+0000 7f03c193c640 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.468+0000 7f03c193c640 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a854c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14381f8a905d53ed server_cookie=b54b498fee6abac7 in_seq=0 out_seq=0 2024-02-14T15:17:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:01.468+0000 7f03a5ffb640 10 client.17415 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:02.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 603 65.24 MB/sec warmup 3 sec latency 1475.222 ms 2024-02-14T15:17:03.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 774 51.34 MB/sec warmup 4 sec latency 1698.298 ms 2024-02-14T15:17:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:02 smithi050 ceph-mon[36512]: pgmap v428: 129 pgs: 129 active+clean; 628 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 196 op/s 2024-02-14T15:17:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03a5ffb640 1 -- 172.21.15.17:0/1539479659 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f03b800ba20 con 0x5600a0a2c300 2024-02-14T15:17:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03a5ffb640 10 client.17415 handle_command_reply: tid=0 2024-02-14T15:17:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 client.17415 shutdown 2024-02-14T15:17:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 msgr2=0x5600a0a21c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a21c40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009dd0 tx=0x7f03ac008040 comp rx=0 tx=0).stop 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 shutdown_connections 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5600a0a2c300 0x5600a0a854c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5600a0a1cc20 0x5600a0a81760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03a5ffb640 0 client.17415 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a0a14040 0x5600a0a22180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 --2- 172.21.15.17:0/1539479659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5600a0a13720 0x5600a0a21c40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.204+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 >> 172.21.15.17:0/1539479659 conn(0x5600a0a0de50 msgr2=0x5600a0a0f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.205+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 shutdown_connections 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.205+0000 7f03c6f71740 1 -- 172.21.15.17:0/1539479659 wait complete. 2024-02-14T15:17:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.205+0000 7f03c6f71740 20 client.17415 trim_cache size 0 max 16384 2024-02-14T15:17:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:17:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:17:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:17:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:17:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f03a80a22f0 msgr2=0x7f03a80a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f03a80a22f0 0x7f03a80a47b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f03b8010800 tx=0x7f03b8017000 comp rx=0 tx=0).stop 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 msgr2=0x7f03bc104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc104c90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03b4004af0 tx=0x7f03b402cae0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 shutdown_connections 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f03a80a22f0 0x7f03a80a47b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03bc10dbd0 0x7f03bc104c90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03bc100940 0x7f03bc104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 --2- 172.21.15.17:0/3102380209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03bc100020 0x7f03bc10d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 >> 172.21.15.17:0/3102380209 conn(0x7f03bc077250 msgr2=0x7f03bc10f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 shutdown_connections 2024-02-14T15:17:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:03.206+0000 7f03c43a1640 1 -- 172.21.15.17:0/3102380209 wait complete. 2024-02-14T15:17:03.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:02 smithi076 ceph-mon[37057]: pgmap v428: 129 pgs: 129 active+clean; 628 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 196 op/s 2024-02-14T15:17:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:02 smithi017 ceph-mon[32549]: pgmap v428: 129 pgs: 129 active+clean; 628 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 196 op/s 2024-02-14T15:17:04.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 965 42.80 MB/sec warmup 5 sec latency 256.746 ms 2024-02-14T15:17:04.486 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: *** Caught signal (Segmentation fault) ** 2024-02-14T15:17:04.486 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: in thread 7f8619a17700 thread_name:md_log_replay 2024-02-14T15:17:04.486 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:17:04.486 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f862add8d20] 2024-02-14T15:17:04.486 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55a5c6f08d5f] 2024-02-14T15:17:04.486 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a5c6f0af93] 2024-02-14T15:17:04.486 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 4: (MDLog::_replay_thread()+0x647) [0x55a5c6f677b7] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a5c6f71651] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f862adce1ca] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 7: clone() 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2024-02-14T15:17:04.037+0000 7f8619a17700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: in thread 7f8619a17700 thread_name:md_log_replay 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f862add8d20] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55a5c6f08d5f] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a5c6f0af93] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 4: (MDLog::_replay_thread()+0x647) [0x55a5c6f677b7] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a5c6f71651] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f862adce1ca] 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 7: clone() 2024-02-14T15:17:04.487 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 0> 2024-02-14T15:17:04.037+0000 7f8619a17700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: in thread 7f8619a17700 thread_name:md_log_replay 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f862add8d20] 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55a5c6f08d5f] 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a5c6f0af93] 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 4: (MDLog::_replay_thread()+0x647) [0x55a5c6f677b7] 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a5c6f71651] 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f862adce1ca] 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 7: clone() 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 0> 2024-02-14T15:17:04.037+0000 7f8619a17700 -1 *** Caught signal (Segmentation fault) ** 2024-02-14T15:17:04.488 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: in thread 7f8619a17700 thread_name:md_log_replay 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: ceph version 19.0.0-1280-ga6fe3fa3 (a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0) squid (dev) 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f862add8d20] 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2: /usr/bin/ceph-mds(+0x503d5f) [0x55a5c6f08d5f] 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a5c6f0af93] 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 4: (MDLog::_replay_thread()+0x647) [0x55a5c6f677b7] 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a5c6f71651] 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f862adce1ca] 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 7: clone() 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-14T15:17:04.489 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[66970]: 2024-02-14T15:17:05.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 1234 38.60 MB/sec warmup 6 sec latency 221.723 ms 2024-02-14T15:17:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:04 smithi050 ceph-mon[36512]: pgmap v429: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 231 op/s 2024-02-14T15:17:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:04 smithi050 ceph-mon[36512]: osdmap e112: 12 total, 12 up, 12 in 2024-02-14T15:17:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-mon[37057]: pgmap v429: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 231 op/s 2024-02-14T15:17:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:04 smithi076 ceph-mon[37057]: osdmap e112: 12 total, 12 up, 12 in 2024-02-14T15:17:05.234 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:05 smithi076 podman[73033]: 2024-02-14 15:17:05.015983943 +0000 UTC m=+0.743808510 container died 177ff9ea3110e493cb8d17b0a52be39c19db749a6cc45319dbc33ef42c14779f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-14T15:17:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:04 smithi017 ceph-mon[32549]: pgmap v429: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 231 op/s 2024-02-14T15:17:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:04 smithi017 ceph-mon[32549]: osdmap e112: 12 total, 12 up, 12 in 2024-02-14T15:17:06.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 1535 35.31 MB/sec warmup 7 sec latency 195.503 ms 2024-02-14T15:17:06.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:05 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:17:06.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:05 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:17:06.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:05 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:17:07.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 1965 34.21 MB/sec warmup 8 sec latency 1080.741 ms 2024-02-14T15:17:07.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:06 smithi050 ceph-mon[36512]: pgmap v431: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 277 op/s 2024-02-14T15:17:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:06 smithi076 ceph-mon[37057]: pgmap v431: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 277 op/s 2024-02-14T15:17:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:06 smithi017 ceph-mon[32549]: pgmap v431: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 277 op/s 2024-02-14T15:17:08.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 2497 33.55 MB/sec warmup 9 sec latency 2080.770 ms 2024-02-14T15:17:08.184 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:07 smithi076 podman[73033]: 2024-02-14 15:17:07.834557174 +0000 UTC m=+3.562381712 container remove 177ff9ea3110e493cb8d17b0a52be39c19db749a6cc45319dbc33ef42c14779f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-14T15:17:08.184 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:07 smithi076 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-02-14T15:17:08.484 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:08 smithi076 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-02-14T15:17:08.484 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:08 smithi076 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l.service: Consumed 1.032s CPU time. 2024-02-14T15:17:09.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 2893 33.37 MB/sec warmup 10 sec latency 3080.813 ms 2024-02-14T15:17:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:08 smithi050 ceph-mon[36512]: pgmap v432: 129 pgs: 129 active+clean; 714 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 533 op/s 2024-02-14T15:17:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:08 smithi076 ceph-mon[37057]: pgmap v432: 129 pgs: 129 active+clean; 714 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 533 op/s 2024-02-14T15:17:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:08 smithi017 ceph-mon[32549]: pgmap v432: 129 pgs: 129 active+clean; 714 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 533 op/s 2024-02-14T15:17:10.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 3242 32.79 MB/sec warmup 11 sec latency 4080.841 ms 2024-02-14T15:17:11.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 3438 30.63 MB/sec warmup 12 sec latency 5080.888 ms 2024-02-14T15:17:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:10 smithi050 ceph-mon[36512]: pgmap v433: 129 pgs: 129 active+clean; 714 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 533 op/s 2024-02-14T15:17:11.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:10 smithi050 ceph-mon[36512]: osdmap e113: 12 total, 12 up, 12 in 2024-02-14T15:17:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:10 smithi076 ceph-mon[37057]: pgmap v433: 129 pgs: 129 active+clean; 714 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 533 op/s 2024-02-14T15:17:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:10 smithi076 ceph-mon[37057]: osdmap e113: 12 total, 12 up, 12 in 2024-02-14T15:17:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:10 smithi017 ceph-mon[32549]: pgmap v433: 129 pgs: 129 active+clean; 714 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 533 op/s 2024-02-14T15:17:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:10 smithi017 ceph-mon[32549]: osdmap e113: 12 total, 12 up, 12 in 2024-02-14T15:17:12.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 3647 28.61 MB/sec warmup 13 sec latency 6080.928 ms 2024-02-14T15:17:13.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 3988 27.79 MB/sec warmup 14 sec latency 7080.965 ms 2024-02-14T15:17:13.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:12 smithi050 ceph-mon[36512]: pgmap v435: 129 pgs: 129 active+clean; 721 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 495 op/s 2024-02-14T15:17:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:12 smithi076 ceph-mon[37057]: pgmap v435: 129 pgs: 129 active+clean; 721 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 495 op/s 2024-02-14T15:17:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:12 smithi017 ceph-mon[32549]: pgmap v435: 129 pgs: 129 active+clean; 721 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 495 op/s 2024-02-14T15:17:14.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 4256 26.29 MB/sec warmup 15 sec latency 8081.010 ms 2024-02-14T15:17:15.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 4620 26.30 MB/sec warmup 16 sec latency 9081.038 ms 2024-02-14T15:17:15.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:14 smithi050 ceph-mon[36512]: pgmap v436: 129 pgs: 129 active+clean; 728 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 13 MiB/s wr, 474 op/s 2024-02-14T15:17:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:14 smithi076 ceph-mon[37057]: pgmap v436: 129 pgs: 129 active+clean; 728 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 13 MiB/s wr, 474 op/s 2024-02-14T15:17:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:14 smithi017 ceph-mon[32549]: pgmap v436: 129 pgs: 129 active+clean; 728 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 13 MiB/s wr, 474 op/s 2024-02-14T15:17:16.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 5196 26.19 MB/sec warmup 17 sec latency 10081.068 ms 2024-02-14T15:17:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 -- 172.21.15.17:0/744996098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8100cf0 msgr2=0x7f60b8104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/744996098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8100cf0 0x7f60b8104d30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009930 tx=0x7f60a002bc30 comp rx=0 tx=0).stop 2024-02-14T15:17:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 -- 172.21.15.17:0/744996098 shutdown_connections 2024-02-14T15:17:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/744996098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8100cf0 0x7f60b8104d30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/744996098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60b8105410 0x7f60b810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/744996098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60b8100340 0x7f60b8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 -- 172.21.15.17:0/744996098 >> 172.21.15.17:0/744996098 conn(0x7f60b8077220 msgr2=0x7f60b8077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 -- 172.21.15.17:0/744996098 shutdown_connections 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 -- 172.21.15.17:0/744996098 wait complete. 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 Processor -- start 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.303+0000 7f60bd4d6640 1 -- start start 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60b8100340 0x7f60b819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8105410 0x7f60b819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60b8197760 0x7f60b8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b8198230 con 0x7f60b8105410 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b81983a0 con 0x7f60b8197760 2024-02-14T15:17:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b8198560 con 0x7f60b8100340 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8105410 0x7f60b819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8105410 0x7f60b819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51808/0 (socket says 172.21.15.17:51808) 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 -- 172.21.15.17:0/1193065394 learned_addr learned my addr 172.21.15.17:0/1193065394 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60b8100340 0x7f60b819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60b77fe640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60b8197760 0x7f60b8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 -- 172.21.15.17:0/1193065394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60b8100340 msgr2=0x7f60b819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60b8100340 0x7f60b819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 -- 172.21.15.17:0/1193065394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60b8197760 msgr2=0x7f60b8197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60b8197760 0x7f60b8197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 -- 172.21.15.17:0/1193065394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a0009560 con 0x7f60b8105410 2024-02-14T15:17:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60affff640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8105410 0x7f60b819dbe0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009230 tx=0x7f60a002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60b4ff9640 1 -- 172.21.15.17:0/1193065394 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0004350 con 0x7f60b8105410 2024-02-14T15:17:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60b4ff9640 1 -- 172.21.15.17:0/1193065394 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60a00044b0 con 0x7f60b8105410 2024-02-14T15:17:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b81a44a0 con 0x7f60b8105410 2024-02-14T15:17:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60b4ff9640 1 -- 172.21.15.17:0/1193065394 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0003da0 con 0x7f60b8105410 2024-02-14T15:17:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.304+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b81a4980 con 0x7f60b8105410 2024-02-14T15:17:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.307+0000 7f60b4ff9640 1 -- 172.21.15.17:0/1193065394 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f60a0004df0 con 0x7f60b8105410 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.307+0000 7f60c0177740 1 Processor -- start 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.307+0000 7f60c0177740 1 -- start start 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.307+0000 7f60b4ff9640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f60a40a2220 0x7f60a40a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60c0177740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60c0177740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697bd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b77fe640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f60a40a2220 0x7f60a40a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60c0177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60c0177740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581697229e0 con 0x5581697b5040 2024-02-14T15:17:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60c0177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816971ff10 con 0x5581697bdc20 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60c0177740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816971b6f0 con 0x5581697b4720 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b4ff9640 1 -- 172.21.15.17:0/1193065394 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f60a00f0410 con 0x7f60b8105410 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37246/0 (socket says 172.21.15.17:37246) 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 -- 172.21.15.17:0/3912529703 learned_addr learned my addr 172.21.15.17:0/3912529703 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60affff640 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697bd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b77fe640 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 -- 172.21.15.17:0/3912529703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 msgr2=0x5581697c0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 -- 172.21.15.17:0/3912529703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 msgr2=0x5581697bd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697bd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b6ffd640 1 -- 172.21.15.17:0/3912529703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55816942c920 con 0x5581697b4720 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.308+0000 7f60b77fe640 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.309+0000 7f60b77fe640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f60a40a2220 0x7f60a40a46e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5581697affe0 tx=0x7f60b0009aa0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.309+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b4b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009d20 tx=0x7f60a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8303dd24d970e69 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.309+0000 7f60ad7fa640 1 -- 172.21.15.17:0/3912529703 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8019630 con 0x5581697b4720 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.309+0000 7f60ad7fa640 1 -- 172.21.15.17:0/3912529703 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60a8009e10 con 0x5581697b4720 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.309+0000 7f60ad7fa640 1 -- 172.21.15.17:0/3912529703 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a800ebb0 con 0x5581697b4720 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 -- 172.21.15.17:0/3912529703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 msgr2=0x5581697b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b4b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009d20 tx=0x7f60a8011810 comp rx=0 tx=0).stop 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 -- 172.21.15.17:0/3912529703 shutdown_connections 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697bd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 --2- 172.21.15.17:0/3912529703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b4b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 -- 172.21.15.17:0/3912529703 >> 172.21.15.17:0/3912529703 conn(0x5581697aee50 msgr2=0x5581697af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 -- 172.21.15.17:0/3912529703 shutdown_connections 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 -- 172.21.15.17:0/3912529703 wait complete. 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 -- start start 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.310+0000 7f60c0177740 1 Processor -- start 2024-02-14T15:17:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60c0177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b4720 0x5581697bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60c0177740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b5040 0x5581697ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60c0177740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697baeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60c0177740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581696b7fa0 con 0x5581697bdc20 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b5040 0x5581697ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60c0177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581693ef250 con 0x5581697b4720 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b5040 0x5581697ba930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:37260/0 (socket says 172.21.15.17:37260) 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b4720 0x5581697bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60affff640 1 -- 172.21.15.17:0/2742120677 learned_addr learned my addr 172.21.15.17:0/2742120677 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581696a1200 con 0x5581697b5040 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b77fe640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697baeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60c0177740 10 client.0 _ll_register_callbacks cb 0x558169710b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b77fe640 1 -- 172.21.15.17:0/2742120677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b5040 msgr2=0x5581697ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b77fe640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b5040 0x5581697ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b77fe640 1 -- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b4720 msgr2=0x5581697bd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b77fe640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b4720 0x5581697bd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b77fe640 1 -- 172.21.15.17:0/2742120677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a8009950 con 0x5581697bdc20 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60affff640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b5040 0x5581697ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.311+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b4720 0x5581697bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60b77fe640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697baeb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60b0005fd0 tx=0x7f60b0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e254c362cca06f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f609e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f609e7fc640 1 -- 172.21.15.17:0/2742120677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b00045a0 con 0x5581697bdc20 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f609e7fc640 1 -- 172.21.15.17:0/2742120677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f60b0004700 con 0x5581697bdc20 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55816941b600 con 0x5581697bdc20 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f609e7fc640 1 -- 172.21.15.17:0/2742120677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b0010430 con 0x5581697bdc20 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 10 client.17427 fetch_fsmap learned FSMap version 36 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558169719d70 con 0x5581697bdc20 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f609e7fc640 1 -- 172.21.15.17:0/2742120677 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55816941b600 con 0x5581697bdc20 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f609e7fc640 1 -- 172.21.15.17:0/2742120677 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f60b00245e0 con 0x5581697bdc20 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 10 client.17427 fetch_fsmap finished waiting for FSMap version 36 2024-02-14T15:17:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 10 client.17427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 20 client.17427 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5581697cd960 0x5581697cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 4 client.17427 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.312+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55816946c1b0 con 0x5581697cd960 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.313+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5581697cd960 0x5581697cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.313+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5581697cd960 0x5581697cfd80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa301a6b6f707857 server_cookie=64096c14bd096a48 in_seq=0 out_seq=0 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.313+0000 7f609e7fc640 10 client.17427 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f609e7fc640 1 -- 172.21.15.17:0/2742120677 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55816946c1b0 con 0x5581697cd960 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f609e7fc640 10 client.17427 handle_command_reply: tid=0 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 client.17427 shutdown 2024-02-14T15:17:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 msgr2=0x5581697baeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697baeb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60b0005fd0 tx=0x7f60b0009500 comp rx=0 tx=0).stop 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 shutdown_connections 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5581697cd960 0x5581697cfd80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697baeb0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b5040 0x5581697ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f609e7fc640 0 client.17427 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 --2- 172.21.15.17:0/2742120677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b4720 0x5581697bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 >> 172.21.15.17:0/2742120677 conn(0x5581697aee50 msgr2=0x5581697b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 shutdown_connections 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 1 -- 172.21.15.17:0/2742120677 wait complete. 2024-02-14T15:17:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.314+0000 7f60c0177740 20 client.17427 trim_cache size 0 max 16384 2024-02-14T15:17:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.329+0000 7f60c0177740 1 Processor -- start 2024-02-14T15:17:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.329+0000 7f60c0177740 1 -- start start 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60c0177740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60c0177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b5040 0x5581697bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60c0177740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60c0177740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581697b3ec0 con 0x5581697bdc20 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60c0177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581697c9e30 con 0x5581697b5040 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60c0177740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581697cd760 con 0x5581697b4720 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60affff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b5040 0x5581697bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697bd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51842/0 (socket says 172.21.15.17:51842) 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 -- 172.21.15.17:0/136951763 learned_addr learned my addr 172.21.15.17:0/136951763 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 -- 172.21.15.17:0/136951763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 msgr2=0x5581697b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 -- 172.21.15.17:0/136951763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b5040 msgr2=0x5581697bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b5040 0x5581697bd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 -- 172.21.15.17:0/136951763 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581696b6820 con 0x5581697bdc20 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.330+0000 7f60b77fe640 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697bd5a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f60b00249f0 tx=0x7f60b0009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285494def0a1d23 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.331+0000 7f60acff9640 1 -- 172.21.15.17:0/136951763 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0024b60 con 0x5581697bdc20 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.331+0000 7f60acff9640 1 -- 172.21.15.17:0/136951763 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b0024cc0 con 0x5581697bdc20 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.331+0000 7f60acff9640 1 -- 172.21.15.17:0/136951763 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0004770 con 0x5581697bdc20 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.331+0000 7f60c0177740 1 -- 172.21.15.17:0/136951763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 msgr2=0x5581697bd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.331+0000 7f60c0177740 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697bd5a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f60b00249f0 tx=0x7f60b0009ce0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- 172.21.15.17:0/136951763 shutdown_connections 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697bdc20 0x5581697bd5a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697b5040 0x5581697bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 --2- 172.21.15.17:0/136951763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- 172.21.15.17:0/136951763 >> 172.21.15.17:0/136951763 conn(0x5581697aee50 msgr2=0x5581697b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- 172.21.15.17:0/136951763 shutdown_connections 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- 172.21.15.17:0/136951763 wait complete. 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- start start 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 Processor -- start 2024-02-14T15:17:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581697c40c0 con 0x5581697b5040 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581697c4230 con 0x5581697bdc20 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558169823e90 con 0x5581697b4720 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60c0177740 10 client.0 _ll_register_callbacks cb 0x7f60a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60b6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51856/0 (socket says 172.21.15.17:51856) 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.332+0000 7f60affff640 1 -- 172.21.15.17:0/394973019 learned_addr learned my addr 172.21.15.17:0/394973019 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60b77fe640 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60affff640 1 -- 172.21.15.17:0/394973019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 msgr2=0x5581697c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60affff640 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60affff640 1 -- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 msgr2=0x5581697c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60affff640 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60affff640 1 -- 172.21.15.17:0/394973019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558169719d70 con 0x5581697b5040 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60affff640 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697c3400 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5760f8dde44055 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f609e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:17:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f609e7fc640 1 -- 172.21.15.17:0/394973019 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a00bcb50 con 0x5581697b5040 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f609e7fc640 1 -- 172.21.15.17:0/394973019 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f60a00c0ca0 con 0x5581697b5040 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60a8009e10 con 0x5581697b5040 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f609e7fc640 1 -- 172.21.15.17:0/394973019 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a00db0c0 con 0x5581697b5040 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60c0177740 10 client.17433 fetch_fsmap learned FSMap version 36 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5581696dc720 con 0x5581697b5040 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.333+0000 7f609e7fc640 1 -- 172.21.15.17:0/394973019 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60a8009e10 con 0x5581697b5040 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f609e7fc640 1 -- 172.21.15.17:0/394973019 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f60a003c070 con 0x5581697b5040 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60c0177740 10 client.17433 fetch_fsmap finished waiting for FSMap version 36 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60c0177740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60c0177740 20 client.17433 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60c0177740 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558169825440 0x558169827860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60c0177740 4 client.17433 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:17:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558169458550 con 0x558169825440 2024-02-14T15:17:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558169825440 0x558169827860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f609e7fc640 10 client.17433 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.334+0000 7f60b6ffd640 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558169825440 0x558169827860 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291d28e5ce5bf04f server_cookie=d0a3b11912f462ba in_seq=0 out_seq=0 2024-02-14T15:17:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f609e7fc640 1 -- 172.21.15.17:0/394973019 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558169458550 con 0x558169825440 2024-02-14T15:17:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f609e7fc640 10 client.17433 handle_command_reply: tid=0 2024-02-14T15:17:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 client.17433 shutdown 2024-02-14T15:17:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 msgr2=0x5581697c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697c3400 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a0009260 comp rx=0 tx=0).stop 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 shutdown_connections 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558169825440 0x558169827860 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5581697b5040 0x5581697c3400 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5581697bdc20 0x5581697c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f609e7fc640 0 client.17433 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 --2- 172.21.15.17:0/394973019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581697b4720 0x5581697c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 >> 172.21.15.17:0/394973019 conn(0x5581697aee50 msgr2=0x5581697ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 shutdown_connections 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 1 -- 172.21.15.17:0/394973019 wait complete. 2024-02-14T15:17:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60c0177740 20 client.17433 trim_cache size 0 max 16384 2024-02-14T15:17:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:17:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:17:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:17:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:17:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f60a40a2220 msgr2=0x7f60a40a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f60a40a2220 0x7f60a40a46e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5581697affe0 tx=0x7f60b0009aa0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8105410 msgr2=0x7f60b819dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.358+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8105410 0x7f60b819dbe0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009230 tx=0x7f60a002c680 comp rx=0 tx=0).stop 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 shutdown_connections 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f60a40a2220 0x7f60a40a46e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60b8105410 0x7f60b819dbe0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60b8197760 0x7f60b8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 --2- 172.21.15.17:0/1193065394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60b8100340 0x7f60b819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 >> 172.21.15.17:0/1193065394 conn(0x7f60b8077220 msgr2=0x7f60b8068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 shutdown_connections 2024-02-14T15:17:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:16.359+0000 7f60bd4d6640 1 -- 172.21.15.17:0/1193065394 wait complete. 2024-02-14T15:17:17.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 5580 26.33 MB/sec warmup 18 sec latency 11081.100 ms 2024-02-14T15:17:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:16 smithi050 ceph-mon[36512]: pgmap v437: 129 pgs: 129 active+clean; 728 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 441 op/s 2024-02-14T15:17:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:16 smithi076 ceph-mon[37057]: pgmap v437: 129 pgs: 129 active+clean; 728 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 441 op/s 2024-02-14T15:17:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:16 smithi017 ceph-mon[32549]: pgmap v437: 129 pgs: 129 active+clean; 728 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 441 op/s 2024-02-14T15:17:18.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 5935 26.19 MB/sec warmup 19 sec latency 12081.153 ms 2024-02-14T15:17:18.686 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-02-14T15:17:18.686 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 systemd[1]: Stopped Ceph mds.l for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:17:18.686 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l.service: Consumed 1.032s CPU time. 2024-02-14T15:17:18.686 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 systemd[1]: Starting Ceph mds.l for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:17:18.687 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 podman[73127]: 2024-02-14 15:17:18.389217467 +0000 UTC m=+0.024769870 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:17:18.687 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 podman[73127]: 2024-02-14 15:17:18.685876564 +0000 UTC m=+0.321428968 container create 06398c205308b932ad9d6145c19ed4511f6e31e533eb48e16d1ce2d1b7d180fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-14T15:17:18.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:18 smithi076 ceph-mon[37057]: pgmap v438: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-02-14T15:17:18.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 podman[73127]: 2024-02-14 15:17:18.931000305 +0000 UTC m=+0.566552709 container init 06398c205308b932ad9d6145c19ed4511f6e31e533eb48e16d1ce2d1b7d180fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-14T15:17:18.985 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:18 smithi076 podman[73127]: 2024-02-14 15:17:18.935263557 +0000 UTC m=+0.570815956 container start 06398c205308b932ad9d6145c19ed4511f6e31e533eb48e16d1ce2d1b7d180fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-14T15:17:19.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 6136 25.82 MB/sec warmup 20 sec latency 13081.201 ms 2024-02-14T15:17:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:18 smithi050 ceph-mon[36512]: pgmap v438: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-02-14T15:17:19.243 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:19 smithi076 bash[73127]: 06398c205308b932ad9d6145c19ed4511f6e31e533eb48e16d1ce2d1b7d180fd 2024-02-14T15:17:19.244 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:19 smithi076 systemd[1]: Started Ceph mds.l for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:17:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:18 smithi017 ceph-mon[32549]: pgmap v438: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-02-14T15:17:19.821 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:19 smithi076 ceph-mds[73146]: -- 172.21.15.76:0/1440971167 <== mon.1 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561eadb95520 con 0x561eadb6ac00 2024-02-14T15:17:19.821 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:17:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[73142]: starting mds.l at 2024-02-14T15:17:20.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 6362 25.01 MB/sec warmup 21 sec latency 13354.331 ms 2024-02-14T15:17:20.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:19 smithi050 ceph-mon[36512]: daemon mds.l restarted 2024-02-14T15:17:20.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:19 smithi076 ceph-mon[37057]: daemon mds.l restarted 2024-02-14T15:17:20.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:19 smithi017 ceph-mon[32549]: daemon mds.l restarted 2024-02-14T15:17:21.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 6587 24.13 MB/sec warmup 22 sec latency 157.372 ms 2024-02-14T15:17:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:20 smithi050 ceph-mon[36512]: pgmap v439: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-02-14T15:17:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:20 smithi050 ceph-mon[36512]: osdmap e114: 12 total, 12 up, 12 in 2024-02-14T15:17:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:20 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6832/1295210715,v1:172.21.15.76:6833/1295210715] up:boot 2024-02-14T15:17:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:20 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-14T15:17:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:20 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:17:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:20 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:17:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:20 smithi076 ceph-mon[37057]: pgmap v439: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-02-14T15:17:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:20 smithi076 ceph-mon[37057]: osdmap e114: 12 total, 12 up, 12 in 2024-02-14T15:17:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:20 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6832/1295210715,v1:172.21.15.76:6833/1295210715] up:boot 2024-02-14T15:17:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:20 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-14T15:17:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:20 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:17:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:20 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:17:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:20 smithi017 ceph-mon[32549]: pgmap v439: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-02-14T15:17:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:20 smithi017 ceph-mon[32549]: osdmap e114: 12 total, 12 up, 12 in 2024-02-14T15:17:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:20 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6832/1295210715,v1:172.21.15.76:6833/1295210715] up:boot 2024-02-14T15:17:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:20 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-14T15:17:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:20 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:17:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:20 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:17:22.140 INFO:tasks.workunit.client.0.smithi017.stdout: 10 6839 23.65 MB/sec warmup 23 sec latency 65.049 ms 2024-02-14T15:17:23.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 7172 23.46 MB/sec warmup 24 sec latency 240.118 ms 2024-02-14T15:17:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:22 smithi050 ceph-mon[36512]: pgmap v441: 129 pgs: 129 active+clean; 748 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 238 op/s 2024-02-14T15:17:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:22 smithi076 ceph-mon[37057]: pgmap v441: 129 pgs: 129 active+clean; 748 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 238 op/s 2024-02-14T15:17:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:22 smithi017 ceph-mon[32549]: pgmap v441: 129 pgs: 129 active+clean; 748 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 238 op/s 2024-02-14T15:17:24.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 7496 22.98 MB/sec warmup 25 sec latency 406.894 ms 2024-02-14T15:17:25.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 7896 23.09 MB/sec warmup 26 sec latency 227.496 ms 2024-02-14T15:17:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:24 smithi050 ceph-mon[36512]: pgmap v442: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.0 MiB/s wr, 259 op/s 2024-02-14T15:17:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:24 smithi076 ceph-mon[37057]: pgmap v442: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.0 MiB/s wr, 259 op/s 2024-02-14T15:17:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:24 smithi017 ceph-mon[32549]: pgmap v442: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.0 MiB/s wr, 259 op/s 2024-02-14T15:17:26.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 8285 22.73 MB/sec warmup 27 sec latency 345.281 ms 2024-02-14T15:17:27.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 8713 22.78 MB/sec warmup 28 sec latency 220.341 ms 2024-02-14T15:17:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:26 smithi050 ceph-mon[36512]: pgmap v443: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.0 MiB/s wr, 259 op/s 2024-02-14T15:17:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:26 smithi076 ceph-mon[37057]: pgmap v443: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.0 MiB/s wr, 259 op/s 2024-02-14T15:17:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:26 smithi017 ceph-mon[32549]: pgmap v443: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.0 MiB/s wr, 259 op/s 2024-02-14T15:17:28.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 9152 23.05 MB/sec warmup 29 sec latency 176.804 ms 2024-02-14T15:17:29.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 9525 23.20 MB/sec warmup 30 sec latency 184.145 ms 2024-02-14T15:17:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:28 smithi050 ceph-mon[36512]: pgmap v444: 129 pgs: 129 active+clean; 739 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.9 MiB/s wr, 263 op/s 2024-02-14T15:17:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:28 smithi076 ceph-mon[37057]: pgmap v444: 129 pgs: 129 active+clean; 739 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.9 MiB/s wr, 263 op/s 2024-02-14T15:17:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:28 smithi017 ceph-mon[32549]: pgmap v444: 129 pgs: 129 active+clean; 739 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.9 MiB/s wr, 263 op/s 2024-02-14T15:17:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 -- 172.21.15.17:0/798821132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9010105410 msgr2=0x7f901010cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 --2- 172.21.15.17:0/798821132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9010105410 0x7f901010cfe0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9000009930 tx=0x7f900002bc30 comp rx=0 tx=0).stop 2024-02-14T15:17:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 -- 172.21.15.17:0/798821132 shutdown_connections 2024-02-14T15:17:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 --2- 172.21.15.17:0/798821132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9010105410 0x7f901010cfe0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 --2- 172.21.15.17:0/798821132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9010100cf0 0x7f9010104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 --2- 172.21.15.17:0/798821132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9010100340 0x7f9010100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 -- 172.21.15.17:0/798821132 >> 172.21.15.17:0/798821132 conn(0x7f9010077220 msgr2=0x7f9010077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 -- 172.21.15.17:0/798821132 shutdown_connections 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 -- 172.21.15.17:0/798821132 wait complete. 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 Processor -- start 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.451+0000 7f9018516640 1 -- start start 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9018516640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9010100340 0x7f901019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9018516640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9010105410 0x7f901019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9018516640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90101976f0 0x7f9010197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9018516640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101981c0 con 0x7f90101976f0 2024-02-14T15:17:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9018516640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010198330 con 0x7f9010105410 2024-02-14T15:17:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9018516640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101984f0 con 0x7f9010100340 2024-02-14T15:17:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90101976f0 0x7f9010197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90101976f0 0x7f9010197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46676/0 (socket says 172.21.15.17:46676) 2024-02-14T15:17:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 -- 172.21.15.17:0/2668434532 learned_addr learned my addr 172.21.15.17:0/2668434532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:29.456 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f90162b2640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9010100340 0x7f901019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9010105410 0x7f901019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 -- 172.21.15.17:0/2668434532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9010100340 msgr2=0x7f901019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9010100340 0x7f901019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 -- 172.21.15.17:0/2668434532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9010105410 msgr2=0x7f901019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9010105410 0x7f901019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 -- 172.21.15.17:0/2668434532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000009560 con 0x7f90101976f0 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.452+0000 7f9016ab3640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90101976f0 0x7f9010197b70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9000005940 tx=0x7f900002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.453+0000 7f900f7fe640 1 -- 172.21.15.17:0/2668434532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000004350 con 0x7f90101976f0 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.453+0000 7f900f7fe640 1 -- 172.21.15.17:0/2668434532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90000044b0 con 0x7f90101976f0 2024-02-14T15:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.453+0000 7f900f7fe640 1 -- 172.21.15.17:0/2668434532 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000003cc0 con 0x7f90101976f0 2024-02-14T15:17:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.453+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90101a4430 con 0x7f90101976f0 2024-02-14T15:17:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.453+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90101a4910 con 0x7f90101976f0 2024-02-14T15:17:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.455+0000 7f900f7fe640 1 -- 172.21.15.17:0/2668434532 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9000005580 con 0x7f90101976f0 2024-02-14T15:17:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.455+0000 7f901b1b6740 1 Processor -- start 2024-02-14T15:17:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f900f7fe640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8ff80a2220 0x7f8ff80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f901b1b6740 1 -- start start 2024-02-14T15:17:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff472b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff47b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff47e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f901b1b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff3e09e0 con 0x55c5ff472720 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f901b1b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff3ddf10 con 0x55c5ff473040 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f901b1b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff3d96f0 con 0x55c5ff47bc20 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f900f7fe640 1 -- 172.21.15.17:0/2668434532 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f90000f03e0 con 0x7f90101976f0 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8ff80a2220 0x7f8ff80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f90162b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff472b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9015ab1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff47b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f90162b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff472b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46680/0 (socket says 172.21.15.17:46680) 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9016ab3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff47e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f90162b2640 1 -- 172.21.15.17:0/4191033654 learned_addr learned my addr 172.21.15.17:0/4191033654 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9016ab3640 1 -- 172.21.15.17:0/4191033654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 msgr2=0x55c5ff47b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9016ab3640 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff47b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9016ab3640 1 -- 172.21.15.17:0/4191033654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 msgr2=0x55c5ff472b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9016ab3640 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff472b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9016ab3640 1 -- 172.21.15.17:0/4191033654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ff0ea920 con 0x55c5ff47bc20 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f90162b2640 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff472b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9015ab1640 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff47b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8ff80a2220 0x7f8ff80a46e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9010198f40 tx=0x7f900400bac0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.456+0000 7f9016ab3640 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff47e010 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90000ed8e0 tx=0x7f900002c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74253a421f945c75 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f900cbfe640 1 -- 172.21.15.17:0/4191033654 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000bcb80 con 0x55c5ff47bc20 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f900cbfe640 1 -- 172.21.15.17:0/4191033654 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90000c0cd0 con 0x55c5ff47bc20 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f900cbfe640 1 -- 172.21.15.17:0/4191033654 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000edba0 con 0x55c5ff47bc20 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 -- 172.21.15.17:0/4191033654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 msgr2=0x55c5ff47e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff47e010 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90000ed8e0 tx=0x7f900002c740 comp rx=0 tx=0).stop 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 -- 172.21.15.17:0/4191033654 shutdown_connections 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff47e010 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff47b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 --2- 172.21.15.17:0/4191033654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff472b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 -- 172.21.15.17:0/4191033654 >> 172.21.15.17:0/4191033654 conn(0x55c5ff46ce50 msgr2=0x55c5ff46d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.457+0000 7f901b1b6740 1 -- 172.21.15.17:0/4191033654 shutdown_connections 2024-02-14T15:17:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 -- 172.21.15.17:0/4191033654 wait complete. 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 -- start start 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 Processor -- start 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff47b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff478950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff478ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff375fa0 con 0x55c5ff472720 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff0ad250 con 0x55c5ff473040 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff35f200 con 0x55c5ff47bc20 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.458+0000 7f901b1b6740 10 client.0 _ll_register_callbacks cb 0x55c5ff3ceb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9016ab3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff478ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff478950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9016ab3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff478ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:47472/0 (socket says 172.21.15.17:47472) 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff478950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:57412/0 (socket says 172.21.15.17:57412) 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9016ab3640 1 -- 172.21.15.17:0/2944655894 learned_addr learned my addr 172.21.15.17:0/2944655894 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 -- 172.21.15.17:0/2944655894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 msgr2=0x55c5ff478ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff478ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 -- 172.21.15.17:0/2944655894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 msgr2=0x55c5ff47b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f90162b2640 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff47b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff47b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 -- 172.21.15.17:0/2944655894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90101a4910 con 0x55c5ff473040 2024-02-14T15:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f90162b2640 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff47b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff478950 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f900400bfd0 tx=0x7f9004004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f6a3323ad0c9e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f8ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2944655894 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900401ecc0 con 0x55c5ff473040 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5ff0d9600 con 0x55c5ff473040 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.459+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2944655894 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f900401ee20 con 0x55c5ff473040 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2944655894 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9004020430 con 0x55c5ff473040 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 10 client.26774 fetch_fsmap learned FSMap version 38 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c5ff3d7d70 con 0x55c5ff473040 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2944655894 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c5ff0d9600 con 0x55c5ff473040 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2944655894 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9004010720 con 0x55c5ff473040 2024-02-14T15:17:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 10 client.26774 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 10 client.26774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 20 client.26774 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff48b7a0 0x55c5ff48dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 4 client.26774 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5ff12a1b0 con 0x55c5ff48b7a0 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f90162b2640 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff48b7a0 0x55c5ff48dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f8ff5ffb640 10 client.26774 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.460+0000 7f90162b2640 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff48b7a0 0x55c5ff48dbc0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437054db79e625b9 server_cookie=fd97309dab9b0cd in_seq=0 out_seq=0 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2944655894 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5ff12a1b0 con 0x55c5ff48b7a0 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f8ff5ffb640 10 client.26774 handle_command_reply: tid=0 2024-02-14T15:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 client.26774 shutdown 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 msgr2=0x55c5ff478950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff478950 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f900400bfd0 tx=0x7f9004004730 comp rx=0 tx=0).stop 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 shutdown_connections 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff48b7a0 0x55c5ff48dbc0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff478ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff478950 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f8ff5ffb640 0 client.26774 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2944655894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff47b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 >> 172.21.15.17:0/2944655894 conn(0x55c5ff46ce50 msgr2=0x55c5ff46eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 shutdown_connections 2024-02-14T15:17:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 1 -- 172.21.15.17:0/2944655894 wait complete. 2024-02-14T15:17:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.461+0000 7f901b1b6740 20 client.26774 trim_cache size 0 max 16384 2024-02-14T15:17:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 Processor -- start 2024-02-14T15:17:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 -- start start 2024-02-14T15:17:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff472720 0x55c5ff47b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff473040 0x55c5ff47b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff47bc20 0x55c5ff475bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff0d9600 con 0x55c5ff473040 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff4685a0 con 0x55c5ff47bc20 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f901b1b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff487bf0 con 0x55c5ff472720 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f9015ab1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff473040 0x55c5ff47b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f9015ab1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff473040 0x55c5ff47b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46694/0 (socket says 172.21.15.17:46694) 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f9015ab1640 1 -- 172.21.15.17:0/537163745 learned_addr learned my addr 172.21.15.17:0/537163745 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f90162b2640 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff472720 0x55c5ff47b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.476+0000 7f9016ab3640 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff47bc20 0x55c5ff475bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f9015ab1640 1 -- 172.21.15.17:0/537163745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff472720 msgr2=0x55c5ff47b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f9015ab1640 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff472720 0x55c5ff47b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f9015ab1640 1 -- 172.21.15.17:0/537163745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff47bc20 msgr2=0x55c5ff475bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f9015ab1640 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff47bc20 0x55c5ff475bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f9015ab1640 1 -- 172.21.15.17:0/537163745 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ff374820 con 0x55c5ff473040 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f9016ab3640 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff47bc20 0x55c5ff475bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f9015ab1640 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff473040 0x55c5ff47b5c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9004010b00 tx=0x7f90040046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835f1584a1815819 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f8ff7fff640 1 -- 172.21.15.17:0/537163745 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004010c70 con 0x55c5ff473040 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f8ff7fff640 1 -- 172.21.15.17:0/537163745 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9004010dd0 con 0x55c5ff473040 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.477+0000 7f8ff7fff640 1 -- 172.21.15.17:0/537163745 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900402a770 con 0x55c5ff473040 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- 172.21.15.17:0/537163745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff473040 msgr2=0x55c5ff47b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff473040 0x55c5ff47b5c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9004010b00 tx=0x7f90040046a0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- 172.21.15.17:0/537163745 shutdown_connections 2024-02-14T15:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff473040 0x55c5ff47b5c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff47bc20 0x55c5ff475bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 --2- 172.21.15.17:0/537163745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff472720 0x55c5ff47b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- 172.21.15.17:0/537163745 >> 172.21.15.17:0/537163745 conn(0x55c5ff46ce50 msgr2=0x55c5ff477220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- 172.21.15.17:0/537163745 shutdown_connections 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- 172.21.15.17:0/537163745 wait complete. 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- start start 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 Processor -- start 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff482dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff481160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff481850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff481f50 con 0x55c5ff472720 2024-02-14T15:17:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff4e6100 con 0x55c5ff473040 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.478+0000 7f901b1b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ff4e6240 con 0x55c5ff47bc20 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f901b1b6740 10 client.0 _ll_register_callbacks cb 0x7f90000035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff482dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff482dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46708/0 (socket says 172.21.15.17:46708) 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 -- 172.21.15.17:0/2219942247 learned_addr learned my addr 172.21.15.17:0/2219942247 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff481160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 -- 172.21.15.17:0/2219942247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 msgr2=0x55c5ff481850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f9016ab3640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff481850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff481850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 -- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 msgr2=0x55c5ff481160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff481160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 -- 172.21.15.17:0/2219942247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ff0ea920 con 0x55c5ff472720 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f9016ab3640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff481850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff481160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f90162b2640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff482dc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9008009fd0 tx=0x7f9008003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0af9a00eebc4e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f8ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2219942247 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008017040 con 0x55c5ff472720 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2219942247 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f9008025680 con 0x55c5ff472720 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90000c0cd0 con 0x55c5ff472720 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.479+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2219942247 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9008010e60 con 0x55c5ff472720 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 10 client.17454 fetch_fsmap learned FSMap version 38 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f9004012e70 con 0x55c5ff472720 2024-02-14T15:17:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2219942247 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90000c0cd0 con 0x55c5ff472720 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2219942247 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f900800fa00 con 0x55c5ff472720 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 10 client.17454 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 20 client.17454 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff4e7aa0 0x55c5ff4e9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 4 client.17454 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5ff116550 con 0x55c5ff4e7aa0 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff4e7aa0 0x55c5ff4e9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f9015ab1640 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff4e7aa0 0x55c5ff4e9ec0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dda56271b34e742 server_cookie=118e6778f2207d2 in_seq=0 out_seq=0 2024-02-14T15:17:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.480+0000 7f8ff5ffb640 10 client.17454 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f8ff5ffb640 1 -- 172.21.15.17:0/2219942247 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5ff116550 con 0x55c5ff4e7aa0 2024-02-14T15:17:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f8ff5ffb640 10 client.17454 handle_command_reply: tid=0 2024-02-14T15:17:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 client.17454 shutdown 2024-02-14T15:17:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 msgr2=0x55c5ff482dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff482dc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9008009fd0 tx=0x7f9008003970 comp rx=0 tx=0).stop 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 shutdown_connections 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55c5ff4e7aa0 0x55c5ff4e9ec0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c5ff47bc20 0x55c5ff481850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ff473040 0x55c5ff481160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f8ff5ffb640 0 client.17454 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 --2- 172.21.15.17:0/2219942247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c5ff472720 0x55c5ff482dc0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 >> 172.21.15.17:0/2219942247 conn(0x55c5ff46ce50 msgr2=0x55c5ff46eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 shutdown_connections 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 1 -- 172.21.15.17:0/2219942247 wait complete. 2024-02-14T15:17:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.525+0000 7f901b1b6740 20 client.17454 trim_cache size 0 max 16384 2024-02-14T15:17:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:17:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:17:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:17:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8ff80a2220 msgr2=0x7f8ff80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8ff80a2220 0x7f8ff80a46e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9010198f40 tx=0x7f900400bac0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90101976f0 msgr2=0x7f9010197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90101976f0 0x7f9010197b70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9000005940 tx=0x7f900002c650 comp rx=0 tx=0).stop 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 shutdown_connections 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8ff80a2220 0x7f8ff80a46e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90101976f0 0x7f9010197b70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9010105410 0x7f901019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 --2- 172.21.15.17:0/2668434532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9010100340 0x7f901019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 >> 172.21.15.17:0/2668434532 conn(0x7f9010077220 msgr2=0x7f9010068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 shutdown_connections 2024-02-14T15:17:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:29.526+0000 7f9018516640 1 -- 172.21.15.17:0/2668434532 wait complete. 2024-02-14T15:17:30.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 9837 22.95 MB/sec warmup 31 sec latency 103.082 ms 2024-02-14T15:17:31.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 10152 22.69 MB/sec warmup 32 sec latency 179.128 ms 2024-02-14T15:17:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:30 smithi050 ceph-mon[36512]: pgmap v445: 129 pgs: 129 active+clean; 739 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.9 MiB/s wr, 263 op/s 2024-02-14T15:17:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:30 smithi076 ceph-mon[37057]: pgmap v445: 129 pgs: 129 active+clean; 739 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.9 MiB/s wr, 263 op/s 2024-02-14T15:17:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:30 smithi017 ceph-mon[32549]: pgmap v445: 129 pgs: 129 active+clean; 739 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.9 MiB/s wr, 263 op/s 2024-02-14T15:17:32.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 10362 22.27 MB/sec warmup 33 sec latency 222.602 ms 2024-02-14T15:17:33.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 10707 22.13 MB/sec warmup 34 sec latency 238.704 ms 2024-02-14T15:17:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:32 smithi050 ceph-mon[36512]: pgmap v446: 129 pgs: 129 active+clean; 744 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.5 MiB/s wr, 277 op/s 2024-02-14T15:17:33.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:32 smithi076 ceph-mon[37057]: pgmap v446: 129 pgs: 129 active+clean; 744 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.5 MiB/s wr, 277 op/s 2024-02-14T15:17:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:32 smithi017 ceph-mon[32549]: pgmap v446: 129 pgs: 129 active+clean; 744 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.5 MiB/s wr, 277 op/s 2024-02-14T15:17:34.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 11081 21.98 MB/sec warmup 35 sec latency 224.383 ms 2024-02-14T15:17:35.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 11521 22.14 MB/sec warmup 36 sec latency 269.328 ms 2024-02-14T15:17:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:34 smithi050 ceph-mon[36512]: pgmap v447: 129 pgs: 129 active+clean; 754 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 267 op/s 2024-02-14T15:17:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:34 smithi076 ceph-mon[37057]: pgmap v447: 129 pgs: 129 active+clean; 754 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 267 op/s 2024-02-14T15:17:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:34 smithi017 ceph-mon[32549]: pgmap v447: 129 pgs: 129 active+clean; 754 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 267 op/s 2024-02-14T15:17:36.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 11984 21.98 MB/sec warmup 37 sec latency 84.772 ms 2024-02-14T15:17:37.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 12195 21.78 MB/sec warmup 38 sec latency 205.729 ms 2024-02-14T15:17:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:36 smithi050 ceph-mon[36512]: pgmap v448: 129 pgs: 129 active+clean; 754 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-02-14T15:17:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:36 smithi076 ceph-mon[37057]: pgmap v448: 129 pgs: 129 active+clean; 754 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-02-14T15:17:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:36 smithi017 ceph-mon[32549]: pgmap v448: 129 pgs: 129 active+clean; 754 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-02-14T15:17:38.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 12582 22.01 MB/sec warmup 39 sec latency 158.427 ms 2024-02-14T15:17:39.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 12978 22.02 MB/sec warmup 40 sec latency 85.825 ms 2024-02-14T15:17:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:38 smithi050 ceph-mon[36512]: pgmap v449: 129 pgs: 129 active+clean; 788 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 323 op/s 2024-02-14T15:17:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:38 smithi076 ceph-mon[37057]: pgmap v449: 129 pgs: 129 active+clean; 788 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 323 op/s 2024-02-14T15:17:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:38 smithi017 ceph-mon[32549]: pgmap v449: 129 pgs: 129 active+clean; 788 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 323 op/s 2024-02-14T15:17:40.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 13214 21.82 MB/sec warmup 41 sec latency 248.215 ms 2024-02-14T15:17:41.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 13476 21.53 MB/sec warmup 42 sec latency 233.180 ms 2024-02-14T15:17:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:40 smithi076 ceph-mon[37057]: pgmap v450: 129 pgs: 129 active+clean; 788 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 218 op/s 2024-02-14T15:17:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:40 smithi017 ceph-mon[32549]: pgmap v450: 129 pgs: 129 active+clean; 788 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 218 op/s 2024-02-14T15:17:41.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:40 smithi050 ceph-mon[36512]: pgmap v450: 129 pgs: 129 active+clean; 788 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 218 op/s 2024-02-14T15:17:42.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 13644 21.23 MB/sec warmup 43 sec latency 544.421 ms 2024-02-14T15:17:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/2951073700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d180ffd80 msgr2=0x7f8d18100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/2951073700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d180ffd80 0x7f8d18100160 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009930 tx=0x7f8d0802bc30 comp rx=0 tx=0).stop 2024-02-14T15:17:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/2951073700 shutdown_connections 2024-02-14T15:17:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/2951073700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d18106600 0x7f8d1810e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/2951073700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d18100730 0x7f8d18105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/2951073700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d180ffd80 0x7f8d18100160 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/2951073700 >> 172.21.15.17:0/2951073700 conn(0x7f8d18075410 msgr2=0x7f8d18075820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/2951073700 shutdown_connections 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/2951073700 wait complete. 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 Processor -- start 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.617+0000 7f8d1e4ed640 1 -- start start 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1e4ed640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d180ffd80 0x7f8d1819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1e4ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d18106600 0x7f8d1819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1e4ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d18197770 0x7f8d18197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1e4ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18198240 con 0x7f8d18197770 2024-02-14T15:17:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1e4ed640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d181983b0 con 0x7f8d180ffd80 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1e4ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18198570 con 0x7f8d18106600 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d18197770 0x7f8d18197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d18197770 0x7f8d18197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55424/0 (socket says 172.21.15.17:55424) 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 -- 172.21.15.17:0/1332624360 learned_addr learned my addr 172.21.15.17:0/1332624360 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d177fe640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d18106600 0x7f8d1819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d17fff640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d180ffd80 0x7f8d1819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 -- 172.21.15.17:0/1332624360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d18106600 msgr2=0x7f8d1819dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d18106600 0x7f8d1819dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 -- 172.21.15.17:0/1332624360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d180ffd80 msgr2=0x7f8d1819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d180ffd80 0x7f8d1819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.618+0000 7f8d1ca8a640 1 -- 172.21.15.17:0/1332624360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d08009560 con 0x7f8d18197770 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.619+0000 7f8d1ca8a640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d18197770 0x7f8d18197bf0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000eb10 tx=0x7f8d1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.619+0000 7f8d157fa640 1 -- 172.21.15.17:0/1332624360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10004b20 con 0x7f8d18197770 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.619+0000 7f8d157fa640 1 -- 172.21.15.17:0/1332624360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d10004c80 con 0x7f8d18197770 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.619+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d181a4510 con 0x7f8d18197770 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.619+0000 7f8d157fa640 1 -- 172.21.15.17:0/1332624360 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10010650 con 0x7f8d18197770 2024-02-14T15:17:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.619+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d181a49f0 con 0x7f8d18197770 2024-02-14T15:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d157fa640 1 -- 172.21.15.17:0/1332624360 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8d10005020 con 0x7f8d18197770 2024-02-14T15:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 Processor -- start 2024-02-14T15:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 -- start start 2024-02-14T15:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7ea720 0x55cb9d7eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d7589e0 con 0x55cb9d7ea720 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d755f10 con 0x55cb9d7f3c20 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d21199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d7516f0 con 0x55cb9d7eb040 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.621+0000 7f8d157fa640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d000a22a0 0x7f8d000a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7ea720 0x55cb9d7eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:46782/0 (socket says 172.21.15.17:46782) 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7ea720 0x55cb9d7eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55434/0 (socket says 172.21.15.17:55434) 2024-02-14T15:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 -- 172.21.15.17:0/2417986342 learned_addr learned my addr 172.21.15.17:0/2417986342 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d1ca8a640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d000a22a0 0x7f8d000a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d177fe640 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d157fa640 1 -- 172.21.15.17:0/1332624360 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8d10014070 con 0x7f8d18197770 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 -- 172.21.15.17:0/2417986342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7f3c20 msgr2=0x55cb9d7f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 -- 172.21.15.17:0/2417986342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7ea720 msgr2=0x55cb9d7eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7ea720 0x55cb9d7eab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 -- 172.21.15.17:0/2417986342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb9d462920 con 0x55cb9d7eb040 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d177fe640 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d1ca8a640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d000a22a0 0x7f8d000a4760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8d100cc280 tx=0x7f8d1000cf70 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8d17fff640 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f36e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d08036040 tx=0x7f8d0802c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4062e44537669396 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8cfe7fc640 1 -- 172.21.15.17:0/2417986342 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08004800 con 0x55cb9d7eb040 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8cfe7fc640 1 -- 172.21.15.17:0/2417986342 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d08004960 con 0x55cb9d7eb040 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.622+0000 7f8cfe7fc640 1 -- 172.21.15.17:0/2417986342 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08005120 con 0x55cb9d7eb040 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 -- 172.21.15.17:0/2417986342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 msgr2=0x55cb9d7f36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f36e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d08036040 tx=0x7f8d0802c5d0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 -- 172.21.15.17:0/2417986342 shutdown_connections 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f36e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 --2- 172.21.15.17:0/2417986342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7ea720 0x55cb9d7eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 -- 172.21.15.17:0/2417986342 >> 172.21.15.17:0/2417986342 conn(0x55cb9d7e4e50 msgr2=0x55cb9d7e5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 -- 172.21.15.17:0/2417986342 shutdown_connections 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 -- 172.21.15.17:0/2417986342 wait complete. 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.623+0000 7f8d21199740 1 -- start start 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 1 Processor -- start 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d6edfa0 con 0x55cb9d7f3c20 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d425250 con 0x55cb9d7ea720 2024-02-14T15:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d6d7200 con 0x55cb9d7eb040 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d21199740 10 client.0 _ll_register_callbacks cb 0x55cb9d746b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:55710/0 (socket says 172.21.15.17:55710) 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d177fe640 1 -- 172.21.15.17:0/4220357763 learned_addr learned my addr 172.21.15.17:0/4220357763 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d177fe640 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d177fe640 1 -- 172.21.15.17:0/4220357763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 msgr2=0x55cb9d7f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.624+0000 7f8d177fe640 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d177fe640 1 -- 172.21.15.17:0/4220357763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 msgr2=0x55cb9d7f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d177fe640 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d177fe640 1 -- 172.21.15.17:0/4220357763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1810e950 con 0x55cb9d7ea720 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d17fff640 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d177fe640 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8d18076960 tx=0x7f8d0c015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edce90f271e5c3aa server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8cf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8cf77fe640 1 -- 172.21.15.17:0/4220357763 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c005020 con 0x55cb9d7ea720 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb9d451600 con 0x55cb9d7ea720 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8cf77fe640 1 -- 172.21.15.17:0/4220357763 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8d0c005180 con 0x55cb9d7ea720 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8cf77fe640 1 -- 172.21.15.17:0/4220357763 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d0c005340 con 0x55cb9d7ea720 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d21199740 10 client.26792 fetch_fsmap learned FSMap version 38 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cb9d74fd70 con 0x55cb9d7ea720 2024-02-14T15:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8cf77fe640 1 -- 172.21.15.17:0/4220357763 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cb9d451600 con 0x55cb9d7ea720 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.625+0000 7f8cf77fe640 1 -- 172.21.15.17:0/4220357763 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8d0c023430 con 0x55cb9d7ea720 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d21199740 10 client.26792 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d21199740 10 client.26792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d21199740 20 client.26792 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d21199740 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d803950 0x55cb9d805d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d21199740 4 client.26792 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb9d4a21b0 con 0x55cb9d803950 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d17fff640 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d803950 0x55cb9d805d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8d17fff640 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d803950 0x55cb9d805d40 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f69e213e34d504f server_cookie=69903ba3d2379c49 in_seq=0 out_seq=0 2024-02-14T15:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.626+0000 7f8cf77fe640 10 client.26792 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8cf77fe640 1 -- 172.21.15.17:0/4220357763 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb9d4a21b0 con 0x55cb9d803950 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8cf77fe640 10 client.26792 handle_command_reply: tid=0 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 client.26792 shutdown 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 msgr2=0x55cb9d7f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8d18076960 tx=0x7f8d0c015040 comp rx=0 tx=0).stop 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 shutdown_connections 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d803950 0x55cb9d805d40 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 --2- 172.21.15.17:0/4220357763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8cf77fe640 0 client.26792 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 >> 172.21.15.17:0/4220357763 conn(0x55cb9d7e4e50 msgr2=0x55cb9d7e6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 shutdown_connections 2024-02-14T15:17:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 1 -- 172.21.15.17:0/4220357763 wait complete. 2024-02-14T15:17:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.627+0000 7f8d21199740 20 client.26792 trim_cache size 0 max 16384 2024-02-14T15:17:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 Processor -- start 2024-02-14T15:17:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 -- start start 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d4a21b0 con 0x55cb9d7f3c20 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d451600 con 0x55cb9d7ea720 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.641+0000 7f8d21199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d800410 con 0x55cb9d7eb040 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7ef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55448/0 (socket says 172.21.15.17:55448) 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 -- 172.21.15.17:0/526309822 learned_addr learned my addr 172.21.15.17:0/526309822 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d17fff640 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 -- 172.21.15.17:0/526309822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 msgr2=0x55cb9d7f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 -- 172.21.15.17:0/526309822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 msgr2=0x55cb9d7f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 -- 172.21.15.17:0/526309822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb9d6ec820 con 0x55cb9d7f3c20 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8d177fe640 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7ef1e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c010620 tx=0x7f8d0c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213c61bf0179a39a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8cfdffb640 1 -- 172.21.15.17:0/526309822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c023d70 con 0x55cb9d7f3c20 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8cfdffb640 1 -- 172.21.15.17:0/526309822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c027070 con 0x55cb9d7f3c20 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.642+0000 7f8cfdffb640 1 -- 172.21.15.17:0/526309822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c023430 con 0x55cb9d7f3c20 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 -- 172.21.15.17:0/526309822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 msgr2=0x55cb9d7ef1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7ef1e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c010620 tx=0x7f8d0c005f90 comp rx=0 tx=0).stop 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 -- 172.21.15.17:0/526309822 shutdown_connections 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7ef1e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 --2- 172.21.15.17:0/526309822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7ea720 0x55cb9d7f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 -- 172.21.15.17:0/526309822 >> 172.21.15.17:0/526309822 conn(0x55cb9d7e4e50 msgr2=0x55cb9d7e5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 -- 172.21.15.17:0/526309822 shutdown_connections 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 -- 172.21.15.17:0/526309822 wait complete. 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.643+0000 7f8d21199740 1 -- start start 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 1 Processor -- start 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7ea720 0x55cb9d7facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d7f9b50 con 0x55cb9d7f3c20 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d7f9cc0 con 0x55cb9d7eb040 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9d7f9e80 con 0x55cb9d7ea720 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d21199740 10 client.0 _ll_register_callbacks cb 0x7f8d0c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55450/0 (socket says 172.21.15.17:55450) 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 -- 172.21.15.17:0/103607673 learned_addr learned my addr 172.21.15.17:0/103607673 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:55732/0 (socket says 172.21.15.17:55732) 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7ea720 0x55cb9d7facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 -- 172.21.15.17:0/103607673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7ea720 msgr2=0x55cb9d7facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7ea720 0x55cb9d7facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 -- 172.21.15.17:0/103607673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7eb040 msgr2=0x55cb9d7f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 -- 172.21.15.17:0/103607673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb9d74fd70 con 0x55cb9d7f3c20 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8d177fe640 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f9590 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c023890 tx=0x7f8d0c010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8938bcf3b3410f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.644+0000 7f8cf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8cf77fe640 1 -- 172.21.15.17:0/103607673 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c023d70 con 0x55cb9d7f3c20 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8cf77fe640 1 -- 172.21.15.17:0/103607673 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f8d0c000e40 con 0x55cb9d7f3c20 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d08004960 con 0x55cb9d7f3c20 2024-02-14T15:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8cf77fe640 1 -- 172.21.15.17:0/103607673 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d0c023430 con 0x55cb9d7f3c20 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 10 client.17472 fetch_fsmap learned FSMap version 38 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cb9d712720 con 0x55cb9d7f3c20 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8cf77fe640 1 -- 172.21.15.17:0/103607673 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c047080 con 0x55cb9d7f3c20 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8cf77fe640 1 -- 172.21.15.17:0/103607673 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8d0c02f400 con 0x55cb9d7f3c20 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 10 client.17472 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 20 client.17472 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d85b0c0 0x55cb9d85d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 4 client.17472 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb9d48e550 con 0x55cb9d85b0c0 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.645+0000 7f8d17fff640 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d85b0c0 0x55cb9d85d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.646+0000 7f8cf77fe640 10 client.17472 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.646+0000 7f8d17fff640 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d85b0c0 0x55cb9d85d4b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f429d6cffd2a700 server_cookie=56c3bf323f6d7bb2 in_seq=0 out_seq=0 2024-02-14T15:17:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8cf77fe640 1 -- 172.21.15.17:0/103607673 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb9d48e550 con 0x55cb9d85b0c0 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8cf77fe640 10 client.17472 handle_command_reply: tid=0 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 client.17472 shutdown 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 msgr2=0x55cb9d7f9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f9590 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c023890 tx=0x7f8d0c010bf0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 shutdown_connections 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55cb9d85b0c0 0x55cb9d85d4b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb9d7f3c20 0x55cb9d7f9590 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cb9d7eb040 0x55cb9d7f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8cf77fe640 0 client.17472 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 --2- 172.21.15.17:0/103607673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb9d7ea720 0x55cb9d7facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 >> 172.21.15.17:0/103607673 conn(0x55cb9d7e4e50 msgr2=0x55cb9d7e61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 shutdown_connections 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 1 -- 172.21.15.17:0/103607673 wait complete. 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.690+0000 7f8d21199740 20 client.17472 trim_cache size 0 max 16384 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:17:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:17:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:17:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d000a22a0 msgr2=0x7f8d000a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d000a22a0 0x7f8d000a4760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8d100cc280 tx=0x7f8d1000cf70 comp rx=0 tx=0).stop 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d18197770 msgr2=0x7f8d18197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d18197770 0x7f8d18197bf0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000eb10 tx=0x7f8d1000c2d0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 shutdown_connections 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f8d000a22a0 0x7f8d000a4760 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d18197770 0x7f8d18197bf0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d18106600 0x7f8d1819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 --2- 172.21.15.17:0/1332624360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d180ffd80 0x7f8d1819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 >> 172.21.15.17:0/1332624360 conn(0x7f8d18075410 msgr2=0x7f8d180fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 shutdown_connections 2024-02-14T15:17:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:42.691+0000 7f8d1e4ed640 1 -- 172.21.15.17:0/1332624360 wait complete. 2024-02-14T15:17:43.141 INFO:tasks.workunit.client.0.smithi017.stdout: 10 13924 21.07 MB/sec warmup 44 sec latency 352.200 ms 2024-02-14T15:17:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:42 smithi076 ceph-mon[37057]: pgmap v451: 129 pgs: 129 active+clean; 797 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 253 op/s 2024-02-14T15:17:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:42 smithi017 ceph-mon[32549]: pgmap v451: 129 pgs: 129 active+clean; 797 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 253 op/s 2024-02-14T15:17:43.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:42 smithi050 ceph-mon[36512]: pgmap v451: 129 pgs: 129 active+clean; 797 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 253 op/s 2024-02-14T15:17:44.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 14253 20.78 MB/sec warmup 45 sec latency 243.013 ms 2024-02-14T15:17:45.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 14657 20.83 MB/sec warmup 46 sec latency 212.978 ms 2024-02-14T15:17:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:44 smithi076 ceph-mon[37057]: pgmap v452: 129 pgs: 129 active+clean; 815 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 308 op/s 2024-02-14T15:17:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:44 smithi017 ceph-mon[32549]: pgmap v452: 129 pgs: 129 active+clean; 815 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 308 op/s 2024-02-14T15:17:45.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:44 smithi050 ceph-mon[36512]: pgmap v452: 129 pgs: 129 active+clean; 815 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 308 op/s 2024-02-14T15:17:46.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 15145 20.88 MB/sec warmup 47 sec latency 59.478 ms 2024-02-14T15:17:47.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 15513 21.00 MB/sec warmup 48 sec latency 233.496 ms 2024-02-14T15:17:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:46 smithi076 ceph-mon[37057]: pgmap v453: 129 pgs: 129 active+clean; 815 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 241 op/s 2024-02-14T15:17:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:46 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:17:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:46 smithi017 ceph-mon[32549]: pgmap v453: 129 pgs: 129 active+clean; 815 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 241 op/s 2024-02-14T15:17:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:46 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:17:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:46 smithi050 ceph-mon[36512]: pgmap v453: 129 pgs: 129 active+clean; 815 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 241 op/s 2024-02-14T15:17:47.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:46 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:17:48.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 15895 21.11 MB/sec warmup 49 sec latency 329.023 ms 2024-02-14T15:17:48.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:47 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:17:48.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:47 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:17:48.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:47 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:17:48.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:47 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:17:48.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:47 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:17:48.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:47 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:17:48.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:47 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:17:48.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:47 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:17:48.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:47 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:17:49.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 16213 21.09 MB/sec warmup 50 sec latency 293.535 ms 2024-02-14T15:17:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:48 smithi076 ceph-mon[37057]: pgmap v454: 129 pgs: 129 active+clean; 829 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.1 MiB/s wr, 387 op/s 2024-02-14T15:17:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:48 smithi017 ceph-mon[32549]: pgmap v454: 129 pgs: 129 active+clean; 829 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.1 MiB/s wr, 387 op/s 2024-02-14T15:17:49.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:48 smithi050 ceph-mon[36512]: pgmap v454: 129 pgs: 129 active+clean; 829 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.1 MiB/s wr, 387 op/s 2024-02-14T15:17:50.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 16501 21.04 MB/sec warmup 51 sec latency 239.617 ms 2024-02-14T15:17:51.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 16739 20.87 MB/sec warmup 52 sec latency 220.902 ms 2024-02-14T15:17:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:50 smithi076 ceph-mon[37057]: pgmap v455: 129 pgs: 129 active+clean; 829 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 277 op/s 2024-02-14T15:17:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:50 smithi017 ceph-mon[32549]: pgmap v455: 129 pgs: 129 active+clean; 829 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 277 op/s 2024-02-14T15:17:51.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:50 smithi050 ceph-mon[36512]: pgmap v455: 129 pgs: 129 active+clean; 829 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 277 op/s 2024-02-14T15:17:52.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 17006 20.68 MB/sec warmup 53 sec latency 255.122 ms 2024-02-14T15:17:53.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 17219 20.49 MB/sec warmup 54 sec latency 529.255 ms 2024-02-14T15:17:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:52 smithi076 ceph-mon[37057]: pgmap v456: 129 pgs: 129 active+clean; 836 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 340 op/s 2024-02-14T15:17:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:52 smithi017 ceph-mon[32549]: pgmap v456: 129 pgs: 129 active+clean; 836 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 340 op/s 2024-02-14T15:17:53.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:52 smithi050 ceph-mon[36512]: pgmap v456: 129 pgs: 129 active+clean; 836 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 340 op/s 2024-02-14T15:17:54.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 17535 20.34 MB/sec warmup 55 sec latency 217.240 ms 2024-02-14T15:17:55.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 17910 20.30 MB/sec warmup 56 sec latency 99.075 ms 2024-02-14T15:17:55.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:54 smithi076 ceph-mon[37057]: pgmap v457: 129 pgs: 129 active+clean; 839 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 382 op/s 2024-02-14T15:17:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:54 smithi017 ceph-mon[32549]: pgmap v457: 129 pgs: 129 active+clean; 839 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 382 op/s 2024-02-14T15:17:55.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:54 smithi050 ceph-mon[36512]: pgmap v457: 129 pgs: 129 active+clean; 839 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 382 op/s 2024-02-14T15:17:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.788+0000 7f89d2f9a640 1 -- 172.21.15.17:0/3609167262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc102cd0 msgr2=0x7f89cc10b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.788+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/3609167262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc102cd0 0x7f89cc10b1b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009900 tx=0x7f89bc02bcd0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 -- 172.21.15.17:0/3609167262 shutdown_connections 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/3609167262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc102cd0 0x7f89cc10b1b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/3609167262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc10b7d0 0x7f89cc10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/3609167262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89cc1023b0 0x7f89cc102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 -- 172.21.15.17:0/3609167262 >> 172.21.15.17:0/3609167262 conn(0x7f89cc0fbbd0 msgr2=0x7f89cc0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 -- 172.21.15.17:0/3609167262 shutdown_connections 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 -- 172.21.15.17:0/3609167262 wait complete. 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 Processor -- start 2024-02-14T15:17:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 -- start start 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc1023b0 0x7f89cc100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.789+0000 7f89d2f9a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89cc102cd0 0x7f89cc100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d2f9a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc10b7d0 0x7f89cc100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d2f9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc1a5c40 con 0x7f89cc1023b0 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d2f9a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc1a5db0 con 0x7f89cc102cd0 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d2f9a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc1a5f70 con 0x7f89cc10b7d0 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc1023b0 0x7f89cc100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc1023b0 0x7f89cc100000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41392/0 (socket says 172.21.15.17:41392) 2024-02-14T15:17:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 -- 172.21.15.17:0/651165755 learned_addr learned my addr 172.21.15.17:0/651165755 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89c3fff640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89cc102cd0 0x7f89cc100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d1537640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc10b7d0 0x7f89cc100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 -- 172.21.15.17:0/651165755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc10b7d0 msgr2=0x7f89cc100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc10b7d0 0x7f89cc100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 -- 172.21.15.17:0/651165755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89cc102cd0 msgr2=0x7f89cc100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89cc102cd0 0x7f89cc100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 -- 172.21.15.17:0/651165755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc009560 con 0x7f89cc1023b0 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89d0d36640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc1023b0 0x7f89cc100000 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f89c800e9f0 tx=0x7f89c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.790+0000 7f89c1ffb640 1 -- 172.21.15.17:0/651165755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8009700 con 0x7f89cc1023b0 2024-02-14T15:17:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.791+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc1a6250 con 0x7f89cc1023b0 2024-02-14T15:17:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.791+0000 7f89c1ffb640 1 -- 172.21.15.17:0/651165755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89c800cd80 con 0x7f89cc1023b0 2024-02-14T15:17:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.791+0000 7f89c1ffb640 1 -- 172.21.15.17:0/651165755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8018920 con 0x7f89cc1023b0 2024-02-14T15:17:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.791+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc1a6730 con 0x7f89cc1023b0 2024-02-14T15:17:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.793+0000 7f89c1ffb640 1 -- 172.21.15.17:0/651165755 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f89c8018a80 con 0x7f89cc1023b0 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 Processor -- start 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 -- start start 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd4bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd54c20 0x55708fd57010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fcb99e0 con 0x55708fd4b720 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fcb6f10 con 0x55708fd4c040 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fcb26f0 con 0x55708fd54c20 2024-02-14T15:17:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d1537640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd4bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89c1ffb640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f89b40a2220 0x7f89b40a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d1537640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd4bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41394/0 (socket says 172.21.15.17:41394) 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d1537640 1 -- 172.21.15.17:0/946871002 learned_addr learned my addr 172.21.15.17:0/946871002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89c3fff640 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd54c20 0x55708fd57010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d0d36640 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89c1ffb640 1 -- 172.21.15.17:0/651165755 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f89c8014070 con 0x7f89cc1023b0 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89d1537640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f89b40a2220 0x7f89b40a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89c3fff640 1 -- 172.21.15.17:0/946871002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 msgr2=0x55708fd546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89c3fff640 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89c3fff640 1 -- 172.21.15.17:0/946871002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 msgr2=0x55708fd4bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.794+0000 7f89c3fff640 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd4bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.795+0000 7f89c3fff640 1 -- 172.21.15.17:0/946871002 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55708f9c3920 con 0x55708fd54c20 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.795+0000 7f89d1537640 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd4bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.795+0000 7f89c3fff640 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd54c20 0x55708fd57010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc02c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1593b334d19ef07 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.795+0000 7f89d1537640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f89b40a2220 0x7f89b40a46e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f89c40049d0 tx=0x7f89c4004ab0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.795+0000 7f89b2ffd640 1 -- 172.21.15.17:0/946871002 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0047c0 con 0x55708fd54c20 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.795+0000 7f89b2ffd640 1 -- 172.21.15.17:0/946871002 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89bc004920 con 0x55708fd54c20 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.795+0000 7f89b2ffd640 1 -- 172.21.15.17:0/946871002 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0050e0 con 0x55708fd54c20 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- 172.21.15.17:0/946871002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd54c20 msgr2=0x55708fd57010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd54c20 0x55708fd57010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc02c740 comp rx=0 tx=0).stop 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- 172.21.15.17:0/946871002 shutdown_connections 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd54c20 0x55708fd57010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/946871002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd4bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- 172.21.15.17:0/946871002 >> 172.21.15.17:0/946871002 conn(0x55708fd45e50 msgr2=0x55708fd46230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- 172.21.15.17:0/946871002 shutdown_connections 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- 172.21.15.17:0/946871002 wait complete. 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- start start 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 Processor -- start 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd51990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd51ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fc4efa0 con 0x55708fd54c20 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708f986250 con 0x55708fd4c040 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.796+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fc38200 con 0x55708fd4b720 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d5b6c740 10 client.0 _ll_register_callbacks cb 0x55708fca7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd51ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d0d36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd51ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41408/0 (socket says 172.21.15.17:41408) 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 -- 172.21.15.17:0/1691655839 learned_addr learned my addr 172.21.15.17:0/1691655839 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89c3fff640 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd51990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 -- 172.21.15.17:0/1691655839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 msgr2=0x55708fd54560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd54560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 -- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 msgr2=0x55708fd51990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd51990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 -- 172.21.15.17:0/1691655839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc10e8f0 con 0x55708fd54c20 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d0d36640 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d1537640 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd51ed0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f89c400fc20 tx=0x7f89c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a2973fa867efff server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89abfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89abfff640 1 -- 172.21.15.17:0/1691655839 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4022650 con 0x55708fd54c20 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89abfff640 1 -- 172.21.15.17:0/1691655839 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f89c40227b0 con 0x55708fd54c20 2024-02-14T15:17:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55708f9b2600 con 0x55708fd54c20 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89abfff640 1 -- 172.21.15.17:0/1691655839 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c400fe60 con 0x55708fd54c20 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d5b6c740 10 client.17484 fetch_fsmap learned FSMap version 38 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.797+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55708fcb0d70 con 0x55708fd54c20 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89abfff640 1 -- 172.21.15.17:0/1691655839 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55708f9b2600 con 0x55708fd54c20 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89abfff640 1 -- 172.21.15.17:0/1691655839 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f89c4028020 con 0x55708fd54c20 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d5b6c740 10 client.17484 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d5b6c740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d5b6c740 20 client.17484 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fd66cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d5b6c740 4 client.17484 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:17:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55708fa031b0 con 0x55708fd648b0 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d0d36640 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fd66cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89abfff640 10 client.17484 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.798+0000 7f89d0d36640 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fd66cd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abbb595b02e422a6 server_cookie=f07b29e68a0ed8f0 in_seq=0 out_seq=0 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89abfff640 1 -- 172.21.15.17:0/1691655839 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55708fa031b0 con 0x55708fd648b0 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89abfff640 10 client.17484 handle_command_reply: tid=0 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 client.17484 shutdown 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 msgr2=0x55708fd51ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd51ed0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f89c400fc20 tx=0x7f89c4004a40 comp rx=0 tx=0).stop 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 shutdown_connections 2024-02-14T15:17:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fd66cd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd51ed0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89abfff640 0 client.17484 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd51990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/1691655839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 >> 172.21.15.17:0/1691655839 conn(0x55708fd45e50 msgr2=0x55708fd47ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 shutdown_connections 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.799+0000 7f89d5b6c740 1 -- 172.21.15.17:0/1691655839 wait complete. 2024-02-14T15:17:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.800+0000 7f89d5b6c740 20 client.17484 trim_cache size 0 max 16384 2024-02-14T15:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.813+0000 7f89d5b6c740 1 Processor -- start 2024-02-14T15:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.813+0000 7f89d5b6c740 1 -- start start 2024-02-14T15:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.813+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4c040 0x55708fd54020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd54c20 0x55708fd54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fd46e50 con 0x55708fd4b720 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fd63060 con 0x55708fd54c20 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fd524d0 con 0x55708fd4c040 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41418/0 (socket says 172.21.15.17:41418) 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 -- 172.21.15.17:0/4158869294 learned_addr learned my addr 172.21.15.17:0/4158869294 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89c3fff640 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4c040 0x55708fd54020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d1537640 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd54c20 0x55708fd54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 -- 172.21.15.17:0/4158869294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4c040 msgr2=0x55708fd54020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4c040 0x55708fd54020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 -- 172.21.15.17:0/4158869294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd54c20 msgr2=0x55708fd54560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd54c20 0x55708fd54560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 -- 172.21.15.17:0/4158869294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55708fc4d820 con 0x55708fd4b720 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89d0d36640 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd508f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f89c80cbc30 tx=0x7f89c801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564ce611cbf71653 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89b27fc640 1 -- 172.21.15.17:0/4158869294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c80991d0 con 0x55708fd4b720 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89b27fc640 1 -- 172.21.15.17:0/4158869294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89c8010b80 con 0x55708fd4b720 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.814+0000 7f89b27fc640 1 -- 172.21.15.17:0/4158869294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c80a3130 con 0x55708fd4b720 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 -- 172.21.15.17:0/4158869294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 msgr2=0x55708fd508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd508f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f89c80cbc30 tx=0x7f89c801c840 comp rx=0 tx=0).stop 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 -- 172.21.15.17:0/4158869294 shutdown_connections 2024-02-14T15:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd54c20 0x55708fd54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4c040 0x55708fd54020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/4158869294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd4b720 0x55708fd508f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 -- 172.21.15.17:0/4158869294 >> 172.21.15.17:0/4158869294 conn(0x55708fd45e50 msgr2=0x55708fd47b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 -- 172.21.15.17:0/4158869294 shutdown_connections 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 -- 172.21.15.17:0/4158869294 wait complete. 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.815+0000 7f89d5b6c740 1 -- start start 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 1 Processor -- start 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd5a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd5a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd5aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fdbade0 con 0x55708fd54c20 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fdbaf20 con 0x55708fd4c040 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708fdbb0e0 con 0x55708fd4b720 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d5b6c740 10 client.0 _ll_register_callbacks cb 0x7f89c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd5aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d0d36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd5a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd5aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41432/0 (socket says 172.21.15.17:41432) 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89c3fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd5a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 -- 172.21.15.17:0/3801508232 learned_addr learned my addr 172.21.15.17:0/3801508232 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 -- 172.21.15.17:0/3801508232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 msgr2=0x55708fd5a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd5a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 -- 172.21.15.17:0/3801508232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 msgr2=0x55708fd5a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd5a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.816+0000 7f89d1537640 1 -- 172.21.15.17:0/3801508232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc1a6730 con 0x55708fd54c20 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d1537640 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd5aee0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f89c4020400 tx=0x7f89c402aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1f32f56c1a2514 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89abfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89abfff640 1 -- 172.21.15.17:0/3801508232 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4037650 con 0x55708fd54c20 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89abfff640 1 -- 172.21.15.17:0/3801508232 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9387+0+0 (secure 0 0 0) 0x7f89c40377b0 con 0x55708fd54c20 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89bc004920 con 0x55708fd54c20 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89abfff640 1 -- 172.21.15.17:0/3801508232 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c400fe60 con 0x55708fd54c20 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 10 client.17490 fetch_fsmap learned FSMap version 38 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55708fc73720 con 0x55708fd54c20 2024-02-14T15:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89abfff640 1 -- 172.21.15.17:0/3801508232 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89bc004920 con 0x55708fd54c20 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89abfff640 1 -- 172.21.15.17:0/3801508232 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f89c4028040 con 0x55708fd54c20 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 10 client.17490 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 20 client.17490 populate_metadata read hostname 'smithi017' 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fdbe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 4 client.17490 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55708f9ef550 con 0x55708fd648b0 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.817+0000 7f89d0d36640 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fdbe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.818+0000 7f89d0d36640 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fdbe790 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f5e390ea87041b3 server_cookie=1fcdd40784ffc497 in_seq=0 out_seq=0 2024-02-14T15:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.818+0000 7f89abfff640 10 client.17490 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89abfff640 1 -- 172.21.15.17:0/3801508232 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55708f9ef550 con 0x55708fd648b0 2024-02-14T15:17:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89abfff640 10 client.17490 handle_command_reply: tid=0 2024-02-14T15:17:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 client.17490 shutdown 2024-02-14T15:17:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 msgr2=0x55708fd5aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd5aee0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f89c4020400 tx=0x7f89c402aa60 comp rx=0 tx=0).stop 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 shutdown_connections 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55708fd648b0 0x55708fdbe790 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55708fd54c20 0x55708fd5aee0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89abfff640 0 client.17490 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708fd4c040 0x55708fd5a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 --2- 172.21.15.17:0/3801508232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55708fd4b720 0x55708fd5a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 >> 172.21.15.17:0/3801508232 conn(0x55708fd45e50 msgr2=0x55708fd47280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 shutdown_connections 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 1 -- 172.21.15.17:0/3801508232 wait complete. 2024-02-14T15:17:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.860+0000 7f89d5b6c740 20 client.17490 trim_cache size 0 max 16384 2024-02-14T15:17:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:17:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:17:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:17:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:17:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f89b40a2220 msgr2=0x7f89b40a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f89b40a2220 0x7f89b40a46e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f89c40049d0 tx=0x7f89c4004ab0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc1023b0 msgr2=0x7f89cc100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc1023b0 0x7f89cc100000 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f89c800e9f0 tx=0x7f89c800eeb0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 shutdown_connections 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f89b40a2220 0x7f89b40a46e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc10b7d0 0x7f89cc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89cc102cd0 0x7f89cc100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 --2- 172.21.15.17:0/651165755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89cc1023b0 0x7f89cc100000 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 >> 172.21.15.17:0/651165755 conn(0x7f89cc0fbbd0 msgr2=0x7f89cc0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 shutdown_connections 2024-02-14T15:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:17:55.861+0000 7f89d2f9a640 1 -- 172.21.15.17:0/651165755 wait complete. 2024-02-14T15:17:56.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 18404 20.35 MB/sec warmup 57 sec latency 195.135 ms 2024-02-14T15:17:57.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 18830 20.37 MB/sec warmup 58 sec latency 181.953 ms 2024-02-14T15:17:57.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:56 smithi076 ceph-mon[37057]: pgmap v458: 129 pgs: 129 active+clean; 839 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 285 op/s 2024-02-14T15:17:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:56 smithi017 ceph-mon[32549]: pgmap v458: 129 pgs: 129 active+clean; 839 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 285 op/s 2024-02-14T15:17:57.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:56 smithi050 ceph-mon[36512]: pgmap v458: 129 pgs: 129 active+clean; 839 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 285 op/s 2024-02-14T15:17:58.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 19147 20.40 MB/sec warmup 59 sec latency 296.158 ms 2024-02-14T15:17:59.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 19482 20.39 MB/sec warmup 60 sec latency 174.060 ms 2024-02-14T15:17:59.194 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:17:58 smithi050 ceph-mon[36512]: pgmap v459: 129 pgs: 129 active+clean; 850 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 419 op/s 2024-02-14T15:17:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:17:58 smithi076 ceph-mon[37057]: pgmap v459: 129 pgs: 129 active+clean; 850 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 419 op/s 2024-02-14T15:17:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:17:58 smithi017 ceph-mon[32549]: pgmap v459: 129 pgs: 129 active+clean; 850 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 419 op/s 2024-02-14T15:18:00.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 19818 20.43 MB/sec warmup 61 sec latency 196.731 ms 2024-02-14T15:18:01.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 20093 20.36 MB/sec warmup 62 sec latency 214.449 ms 2024-02-14T15:18:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:00 smithi076 ceph-mon[37057]: pgmap v460: 129 pgs: 129 active+clean; 850 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 273 op/s 2024-02-14T15:18:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:00 smithi017 ceph-mon[32549]: pgmap v460: 129 pgs: 129 active+clean; 850 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 273 op/s 2024-02-14T15:18:01.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:00 smithi050 ceph-mon[36512]: pgmap v460: 129 pgs: 129 active+clean; 850 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 273 op/s 2024-02-14T15:18:02.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 20269 20.11 MB/sec warmup 63 sec latency 214.133 ms 2024-02-14T15:18:03.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 20380 19.92 MB/sec warmup 64 sec latency 438.942 ms 2024-02-14T15:18:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:02 smithi076 ceph-mon[37057]: pgmap v461: 129 pgs: 129 active+clean; 854 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 322 op/s 2024-02-14T15:18:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:02 smithi017 ceph-mon[32549]: pgmap v461: 129 pgs: 129 active+clean; 854 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 322 op/s 2024-02-14T15:18:03.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:02 smithi050 ceph-mon[36512]: pgmap v461: 129 pgs: 129 active+clean; 854 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 322 op/s 2024-02-14T15:18:04.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 20597 19.72 MB/sec warmup 65 sec latency 233.630 ms 2024-02-14T15:18:05.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 20849 19.64 MB/sec warmup 66 sec latency 104.573 ms 2024-02-14T15:18:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:04 smithi076 ceph-mon[37057]: pgmap v462: 129 pgs: 129 active+clean; 862 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.5 MiB/s wr, 376 op/s 2024-02-14T15:18:05.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:04 smithi017 ceph-mon[32549]: pgmap v462: 129 pgs: 129 active+clean; 862 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.5 MiB/s wr, 376 op/s 2024-02-14T15:18:05.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:04 smithi050 ceph-mon[36512]: pgmap v462: 129 pgs: 129 active+clean; 862 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.5 MiB/s wr, 376 op/s 2024-02-14T15:18:06.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 21256 19.60 MB/sec warmup 67 sec latency 67.698 ms 2024-02-14T15:18:07.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 21687 19.68 MB/sec warmup 68 sec latency 90.422 ms 2024-02-14T15:18:07.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:06 smithi076 ceph-mon[37057]: pgmap v463: 129 pgs: 129 active+clean; 862 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 299 op/s 2024-02-14T15:18:07.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:06 smithi017 ceph-mon[32549]: pgmap v463: 129 pgs: 129 active+clean; 862 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 299 op/s 2024-02-14T15:18:07.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:06 smithi050 ceph-mon[36512]: pgmap v463: 129 pgs: 129 active+clean; 862 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 299 op/s 2024-02-14T15:18:08.142 INFO:tasks.workunit.client.0.smithi017.stdout: 10 21899 19.54 MB/sec warmup 69 sec latency 437.960 ms 2024-02-14T15:18:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.974+0000 7f3001ff7640 1 -- 172.21.15.17:0/1797385739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff4098b20 msgr2=0x7f2ff409cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.974+0000 7f3001ff7640 1 --2- 172.21.15.17:0/1797385739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff4098b20 0x7f2ff409cb60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009890 tx=0x7f2fe802bb90 comp rx=0 tx=0).stop 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.974+0000 7f3001ff7640 1 -- 172.21.15.17:0/1797385739 shutdown_connections 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.974+0000 7f3001ff7640 1 --2- 172.21.15.17:0/1797385739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ff409d240 0x7f2ff40a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.974+0000 7f3001ff7640 1 --2- 172.21.15.17:0/1797385739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff4098b20 0x7f2ff409cb60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.974+0000 7f3001ff7640 1 --2- 172.21.15.17:0/1797385739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ff4098170 0x7f2ff4098550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.974+0000 7f3001ff7640 1 -- 172.21.15.17:0/1797385739 >> 172.21.15.17:0/1797385739 conn(0x7f2ff4093a10 msgr2=0x7f2ff4095e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.975+0000 7f3001ff7640 1 -- 172.21.15.17:0/1797385739 shutdown_connections 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.975+0000 7f3001ff7640 1 -- 172.21.15.17:0/1797385739 wait complete. 2024-02-14T15:18:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 Processor -- start 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 -- start start 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ff4098170 0x7f2ff4009930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ff409d240 0x7f2ff4009e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff400a3b0 0x7f2ff414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400dac0 con 0x7f2ff4098170 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41501b0 con 0x7f2ff400a3b0 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f3001ff7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff4150340 con 0x7f2ff409d240 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f30017f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff400a3b0 0x7f2ff414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.976+0000 7f30017f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff400a3b0 0x7f2ff414fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49252/0 (socket says 172.21.15.17:49252) 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.977+0000 7f30017f6640 1 -- 172.21.15.17:0/293379864 learned_addr learned my addr 172.21.15.17:0/293379864 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.977+0000 7f30017f6640 1 -- 172.21.15.17:0/293379864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ff409d240 msgr2=0x7f2ff4009e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.977+0000 7f30017f6640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ff409d240 0x7f2ff4009e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.977+0000 7f30017f6640 1 -- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ff4098170 msgr2=0x7f2ff4009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.977+0000 7f30017f6640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ff4098170 0x7f2ff4009930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.977+0000 7f30017f6640 1 -- 172.21.15.17:0/293379864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8009490 con 0x7f2ff400a3b0 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.977+0000 7f30017f6640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff400a3b0 0x7f2ff414fc70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000b60 tx=0x7f2ffc06c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.978+0000 7f2ff9ffb640 1 -- 172.21.15.17:0/293379864 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc06cb90 con 0x7f2ff400a3b0 2024-02-14T15:18:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.978+0000 7f2ff9ffb640 1 -- 172.21.15.17:0/293379864 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc06ccf0 con 0x7f2ff400a3b0 2024-02-14T15:18:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.980+0000 7f2ff9ffb640 1 -- 172.21.15.17:0/293379864 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc071600 con 0x7f2ff400a3b0 2024-02-14T15:18:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.980+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff41505f0 con 0x7f2ff400a3b0 2024-02-14T15:18:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.980+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff4150ab0 con 0x7f2ff400a3b0 2024-02-14T15:18:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.983+0000 7f3005e21740 1 Processor -- start 2024-02-14T15:18:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.984+0000 7f3005e21740 1 -- start start 2024-02-14T15:18:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.984+0000 7f3005e21740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707447900 0x561707447ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.984+0000 7f3005e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x5617074508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.984+0000 7f3005e21740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x5617074531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.984+0000 7f3005e21740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617073b59e0 con 0x561707447900 2024-02-14T15:18:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.984+0000 7f3005e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617073b2f10 con 0x561707448220 2024-02-14T15:18:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.984+0000 7f3005e21740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617073ae6f0 con 0x561707450e00 2024-02-14T15:18:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.985+0000 7f3000ff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x5617074508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.985+0000 7f3000ff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x5617074508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49260/0 (socket says 172.21.15.17:49260) 2024-02-14T15:18:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.985+0000 7f3000ff5640 1 -- 172.21.15.17:0/1093383160 learned_addr learned my addr 172.21.15.17:0/1093383160 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.985+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x5617074531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.985+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707447900 0x561707447ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.986+0000 7f2ff9ffb640 1 -- 172.21.15.17:0/293379864 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2ffc072400 con 0x7f2ff400a3b0 2024-02-14T15:18:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.986+0000 7f3000ff5640 1 -- 172.21.15.17:0/1093383160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 msgr2=0x5617074531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.986+0000 7f3000ff5640 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x5617074531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.986+0000 7f3000ff5640 1 -- 172.21.15.17:0/1093383160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707447900 msgr2=0x561707447ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.986+0000 7f3000ff5640 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707447900 0x561707447ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.986+0000 7f3000ff5640 1 -- 172.21.15.17:0/1093383160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617070bf920 con 0x561707448220 2024-02-14T15:18:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.987+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707447900 0x561707447ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:18:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.987+0000 7f3000ff5640 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x5617074508c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009c40 tx=0x7f2ff000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28688aed2d501c41 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.987+0000 7f2ff9ffb640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2fec0a2310 0x7f2fec0a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.988+0000 7f2ff9ffb640 1 -- 172.21.15.17:0/293379864 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9563+0+0 (secure 0 0 0) 0x7f2ffc130f20 con 0x7f2ff400a3b0 2024-02-14T15:18:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.988+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2fec0a2310 0x7f2fec0a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.988+0000 7f2fe6ffd640 1 -- 172.21.15.17:0/1093383160 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0009790 con 0x561707448220 2024-02-14T15:18:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.988+0000 7f2fe6ffd640 1 -- 172.21.15.17:0/1093383160 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ff0005500 con 0x561707448220 2024-02-14T15:18:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.988+0000 7f2fe6ffd640 1 -- 172.21.15.17:0/1093383160 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0010690 con 0x561707448220 2024-02-14T15:18:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.988+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2fec0a2310 0x7f2fec0a47d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8037040 tx=0x7f2fe8005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.991+0000 7f3005e21740 1 -- 172.21.15.17:0/1093383160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 msgr2=0x5617074508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.991+0000 7f3005e21740 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x5617074508c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009c40 tx=0x7f2ff000e720 comp rx=0 tx=0).stop 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 -- 172.21.15.17:0/1093383160 shutdown_connections 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x5617074531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x5617074508c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 --2- 172.21.15.17:0/1093383160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707447900 0x561707447ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 -- 172.21.15.17:0/1093383160 >> 172.21.15.17:0/1093383160 conn(0x561707442030 msgr2=0x561707442410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 -- 172.21.15.17:0/1093383160 shutdown_connections 2024-02-14T15:18:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 -- 172.21.15.17:0/1093383160 wait complete. 2024-02-14T15:18:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.992+0000 7f3005e21740 1 -- start start 2024-02-14T15:18:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 1 Processor -- start 2024-02-14T15:18:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707447900 0x5617074507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707448220 0x56170744db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x56170744e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56170734afa0 con 0x561707448220 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561707082250 con 0x561707447900 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561707334200 con 0x561707450e00 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3005e21740 10 client.0 _ll_register_callbacks cb 0x5617073a3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3000ff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707447900 0x5617074507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3000ff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707447900 0x5617074507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49268/0 (socket says 172.21.15.17:49268) 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.993+0000 7f3000ff5640 1 -- 172.21.15.17:0/298109923 learned_addr learned my addr 172.21.15.17:0/298109923 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707448220 0x56170744db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f30017f6640 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x56170744e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f3000ff5640 1 -- 172.21.15.17:0/298109923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 msgr2=0x56170744e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f3000ff5640 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x56170744e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f3000ff5640 1 -- 172.21.15.17:0/298109923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707448220 msgr2=0x56170744db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f3000ff5640 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707448220 0x56170744db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f3000ff5640 1 -- 172.21.15.17:0/298109923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff00098f0 con 0x561707447900 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.994+0000 7f3000ff5640 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707447900 0x5617074507a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000c00 tx=0x7f2ff0005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa683e469a02cfdd server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.995+0000 7f2fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.995+0000 7f2fdbfff640 1 -- 172.21.15.17:0/298109923 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0004b90 con 0x561707447900 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.995+0000 7f2fdbfff640 1 -- 172.21.15.17:0/298109923 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9563+0+0 (secure 0 0 0) 0x7f2ff0014070 con 0x561707447900 2024-02-14T15:18:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.995+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617070ae600 con 0x561707447900 2024-02-14T15:18:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.995+0000 7f2fdbfff640 1 -- 172.21.15.17:0/298109923 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ff00049b0 con 0x561707447900 2024-02-14T15:18:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 10 client.26828 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5617073acd70 con 0x561707447900 2024-02-14T15:18:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f2fdbfff640 1 -- 172.21.15.17:0/298109923 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5617070ae600 con 0x561707447900 2024-02-14T15:18:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f2fdbfff640 1 -- 172.21.15.17:0/298109923 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2ff00038f0 con 0x561707447900 2024-02-14T15:18:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 10 client.26828 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 20 client.26828 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561707460ab0 0x561707462ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 4 client.26828 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.996+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617070ff1b0 con 0x561707460ab0 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.997+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561707460ab0 0x561707462ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.997+0000 7f2fdbfff640 10 client.26828 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.998+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561707460ab0 0x561707462ea0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8ce7701b2c469e server_cookie=9a09f36d9e741159 in_seq=0 out_seq=0 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f2fdbfff640 1 -- 172.21.15.17:0/298109923 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5617070ff1b0 con 0x561707460ab0 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f2fdbfff640 10 client.26828 handle_command_reply: tid=0 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 client.26828 shutdown 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707447900 msgr2=0x5617074507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707447900 0x5617074507a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000c00 tx=0x7f2ff0005670 comp rx=0 tx=0).stop 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 shutdown_connections 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561707460ab0 0x561707462ea0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707448220 0x56170744db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x56170744e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 --2- 172.21.15.17:0/298109923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707447900 0x5617074507a0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000c00 tx=0x7f2ff0005670 comp rx=0 tx=0).stop 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f2fdbfff640 0 client.26828 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:08.999+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 >> 172.21.15.17:0/298109923 conn(0x561707442030 msgr2=0x561707443d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.000+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 shutdown_connections 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.000+0000 7f3005e21740 1 -- 172.21.15.17:0/298109923 wait complete. 2024-02-14T15:18:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.000+0000 7f3005e21740 20 client.26828 trim_cache size 0 max 16384 2024-02-14T15:18:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.018+0000 7f3005e21740 1 Processor -- start 2024-02-14T15:18:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.018+0000 7f3005e21740 1 -- start start 2024-02-14T15:18:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f3005e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x56170743ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f3005e21740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707450e00 0x5617074677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f3005e21740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707467d10 0x56170744d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f3005e21740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561707447560 con 0x561707450e00 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f3005e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561707443260 con 0x561707448220 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707450e00 0x5617074677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707450e00 0x5617074677d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60880/0 (socket says 172.21.15.17:60880) 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 -- 172.21.15.17:0/1424046160 learned_addr learned my addr 172.21.15.17:0/1424046160 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f3005e21740 1 -- 172.21.15.17:0/1424046160 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561707449750 con 0x561707467d10 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f3000ff5640 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x56170743ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f30017f6640 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707467d10 0x56170744d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 -- 172.21.15.17:0/1424046160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707467d10 msgr2=0x56170744d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707467d10 0x56170744d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 -- 172.21.15.17:0/1424046160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 msgr2=0x56170743ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x56170743ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f2ffbfff640 1 -- 172.21.15.17:0/1424046160 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff40a5620 con 0x561707450e00 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.019+0000 7f30017f6640 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707467d10 0x56170744d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.020+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707450e00 0x5617074677d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8032510 tx=0x7f2fe8046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f31f382140e9a10 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.020+0000 7f2fe67fc640 1 -- 172.21.15.17:0/1424046160 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8078300 con 0x561707450e00 2024-02-14T15:18:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.020+0000 7f2fe67fc640 1 -- 172.21.15.17:0/1424046160 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe802c530 con 0x561707450e00 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.020+0000 7f2fe67fc640 1 -- 172.21.15.17:0/1424046160 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8032860 con 0x561707450e00 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 -- 172.21.15.17:0/1424046160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707450e00 msgr2=0x5617074677d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707450e00 0x5617074677d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8032510 tx=0x7f2fe8046000 comp rx=0 tx=0).stop 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 -- 172.21.15.17:0/1424046160 shutdown_connections 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707450e00 0x5617074677d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707467d10 0x56170744d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 --2- 172.21.15.17:0/1424046160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x56170743ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 -- 172.21.15.17:0/1424046160 >> 172.21.15.17:0/1424046160 conn(0x561707442030 msgr2=0x56170745c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 -- 172.21.15.17:0/1424046160 shutdown_connections 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 -- 172.21.15.17:0/1424046160 wait complete. 2024-02-14T15:18:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 -- start start 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.021+0000 7f3005e21740 1 Processor -- start 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3005e21740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x561707456a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3005e21740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x561707456f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3005e21740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707467d10 0x5617074574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3005e21740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617074b7120 con 0x561707467d10 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3005e21740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617074b7290 con 0x561707448220 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3005e21740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617074b7450 con 0x561707450e00 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3005e21740 10 client.0 _ll_register_callbacks cb 0x7f2ff0009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x561707456a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f30017f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707467d10 0x5617074574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x561707456a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:49284/0 (socket says 172.21.15.17:49284) 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 -- 172.21.15.17:0/2080751311 learned_addr learned my addr 172.21.15.17:0/2080751311 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x561707456f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 -- 172.21.15.17:0/2080751311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 msgr2=0x561707456f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x561707456f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 -- 172.21.15.17:0/2080751311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707467d10 msgr2=0x5617074574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707467d10 0x5617074574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 -- 172.21.15.17:0/2080751311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe8030070 con 0x561707448220 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f30017f6640 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707467d10 0x5617074574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f3000ff5640 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x561707456a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000e760 tx=0x7f2ff0009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83145a534e54097 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.022+0000 7f2fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f2fdbfff640 1 -- 172.21.15.17:0/2080751311 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0005a70 con 0x561707448220 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f2fdbfff640 1 -- 172.21.15.17:0/2080751311 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9563+0+0 (secure 0 0 0) 0x7f2ff0014040 con 0x561707448220 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ff0005500 con 0x561707448220 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f2fdbfff640 1 -- 172.21.15.17:0/2080751311 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ff0005d30 con 0x561707448220 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 10 client.26834 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561707349820 con 0x561707448220 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f2fdbfff640 1 -- 172.21.15.17:0/2080751311 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ff0005500 con 0x561707448220 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f2fdbfff640 1 -- 172.21.15.17:0/2080751311 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2ff00039c0 con 0x561707448220 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 10 client.26834 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 20 client.26834 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5617074b88c0 0x5617074bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 4 client.26834 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:18:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.023+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617070eb550 con 0x5617074b88c0 2024-02-14T15:18:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.024+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5617074b88c0 0x5617074bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.024+0000 7f2ffbfff640 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5617074b88c0 0x5617074bacb0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca78c508d3bfae21 server_cookie=cc869e0181c7d337 in_seq=0 out_seq=0 2024-02-14T15:18:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.024+0000 7f2fdbfff640 10 client.26834 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f2fdbfff640 1 -- 172.21.15.17:0/2080751311 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617070eb550 con 0x5617074b88c0 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f2fdbfff640 10 client.26834 handle_command_reply: tid=0 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 client.26834 shutdown 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 msgr2=0x561707456a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x561707456a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000e760 tx=0x7f2ff0009b10 comp rx=0 tx=0).stop 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 shutdown_connections 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x5617074b88c0 0x5617074bacb0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561707467d10 0x5617074574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f2fdbfff640 0 client.26834 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561707450e00 0x561707456f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 --2- 172.21.15.17:0/2080751311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561707448220 0x561707456a10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 >> 172.21.15.17:0/2080751311 conn(0x561707442030 msgr2=0x56170744d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 shutdown_connections 2024-02-14T15:18:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 1 -- 172.21.15.17:0/2080751311 wait complete. 2024-02-14T15:18:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.071+0000 7f3005e21740 20 client.26834 trim_cache size 0 max 16384 2024-02-14T15:18:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:18:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:18:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:18:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:18:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2fec0a2310 msgr2=0x7f2fec0a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2fec0a2310 0x7f2fec0a47d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8037040 tx=0x7f2fe8005f90 comp rx=0 tx=0).stop 2024-02-14T15:18:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff400a3b0 msgr2=0x7f2ff414fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff400a3b0 0x7f2ff414fc70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000b60 tx=0x7f2ffc06c4b0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 shutdown_connections 2024-02-14T15:18:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f2fec0a2310 0x7f2fec0a47d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff400a3b0 0x7f2ff414fc70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ff409d240 0x7f2ff4009e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 --2- 172.21.15.17:0/293379864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ff4098170 0x7f2ff4009930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 >> 172.21.15.17:0/293379864 conn(0x7f2ff4093a10 msgr2=0x7f2ff4094e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 shutdown_connections 2024-02-14T15:18:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:09.072+0000 7f3001ff7640 1 -- 172.21.15.17:0/293379864 wait complete. 2024-02-14T15:18:09.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 22278 19.58 MB/sec warmup 70 sec latency 252.850 ms 2024-02-14T15:18:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:08 smithi076 ceph-mon[37057]: pgmap v464: 129 pgs: 129 active+clean; 781 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.7 MiB/s wr, 547 op/s 2024-02-14T15:18:09.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:08 smithi017 ceph-mon[32549]: pgmap v464: 129 pgs: 129 active+clean; 781 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.7 MiB/s wr, 547 op/s 2024-02-14T15:18:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:08 smithi050 ceph-mon[36512]: pgmap v464: 129 pgs: 129 active+clean; 781 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.7 MiB/s wr, 547 op/s 2024-02-14T15:18:10.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 22721 19.69 MB/sec warmup 71 sec latency 222.816 ms 2024-02-14T15:18:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:09 smithi076 ceph-mon[37057]: osdmap e115: 12 total, 12 up, 12 in 2024-02-14T15:18:10.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:09 smithi017 ceph-mon[32549]: osdmap e115: 12 total, 12 up, 12 in 2024-02-14T15:18:10.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:09 smithi050 ceph-mon[36512]: osdmap e115: 12 total, 12 up, 12 in 2024-02-14T15:18:11.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 23054 19.72 MB/sec warmup 72 sec latency 210.481 ms 2024-02-14T15:18:11.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:10 smithi076 ceph-mon[37057]: pgmap v466: 129 pgs: 129 active+clean; 781 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.2 MiB/s wr, 496 op/s 2024-02-14T15:18:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:10 smithi017 ceph-mon[32549]: pgmap v466: 129 pgs: 129 active+clean; 781 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.2 MiB/s wr, 496 op/s 2024-02-14T15:18:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:10 smithi050 ceph-mon[36512]: pgmap v466: 129 pgs: 129 active+clean; 781 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.2 MiB/s wr, 496 op/s 2024-02-14T15:18:12.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 23323 19.71 MB/sec warmup 73 sec latency 229.511 ms 2024-02-14T15:18:13.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 23499 19.58 MB/sec warmup 74 sec latency 428.055 ms 2024-02-14T15:18:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:12 smithi076 ceph-mon[37057]: pgmap v467: 129 pgs: 129 active+clean; 780 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.9 MiB/s wr, 484 op/s 2024-02-14T15:18:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:12 smithi017 ceph-mon[32549]: pgmap v467: 129 pgs: 129 active+clean; 780 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.9 MiB/s wr, 484 op/s 2024-02-14T15:18:13.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:12 smithi050 ceph-mon[36512]: pgmap v467: 129 pgs: 129 active+clean; 780 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.9 MiB/s wr, 484 op/s 2024-02-14T15:18:14.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 23692 19.43 MB/sec warmup 75 sec latency 241.429 ms 2024-02-14T15:18:15.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 23909 19.27 MB/sec warmup 76 sec latency 244.360 ms 2024-02-14T15:18:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:14 smithi076 ceph-mon[37057]: pgmap v468: 129 pgs: 129 active+clean; 776 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-02-14T15:18:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:14 smithi017 ceph-mon[32549]: pgmap v468: 129 pgs: 129 active+clean; 776 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-02-14T15:18:15.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:14 smithi050 ceph-mon[36512]: pgmap v468: 129 pgs: 129 active+clean; 776 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-02-14T15:18:16.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 24231 19.23 MB/sec warmup 77 sec latency 63.473 ms 2024-02-14T15:18:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:16 smithi050 ceph-mon[36512]: pgmap v469: 129 pgs: 129 active+clean; 776 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-02-14T15:18:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:16 smithi050 ceph-mon[36512]: osdmap e116: 12 total, 12 up, 12 in 2024-02-14T15:18:17.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 24608 19.19 MB/sec warmup 78 sec latency 195.628 ms 2024-02-14T15:18:17.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:16 smithi076 ceph-mon[37057]: pgmap v469: 129 pgs: 129 active+clean; 776 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-02-14T15:18:17.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:16 smithi076 ceph-mon[37057]: osdmap e116: 12 total, 12 up, 12 in 2024-02-14T15:18:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:16 smithi017 ceph-mon[32549]: pgmap v469: 129 pgs: 129 active+clean; 776 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.6 MiB/s wr, 485 op/s 2024-02-14T15:18:17.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:16 smithi017 ceph-mon[32549]: osdmap e116: 12 total, 12 up, 12 in 2024-02-14T15:18:18.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 24809 19.02 MB/sec warmup 79 sec latency 300.413 ms 2024-02-14T15:18:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:18 smithi050 ceph-mon[36512]: pgmap v471: 129 pgs: 129 active+clean; 804 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.4 MiB/s wr, 438 op/s 2024-02-14T15:18:19.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 25122 19.02 MB/sec warmup 80 sec latency 265.450 ms 2024-02-14T15:18:19.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:18 smithi076 ceph-mon[37057]: pgmap v471: 129 pgs: 129 active+clean; 804 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.4 MiB/s wr, 438 op/s 2024-02-14T15:18:19.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:18 smithi017 ceph-mon[32549]: pgmap v471: 129 pgs: 129 active+clean; 804 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.4 MiB/s wr, 438 op/s 2024-02-14T15:18:20.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 25475 19.01 MB/sec warmup 81 sec latency 342.938 ms 2024-02-14T15:18:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:20 smithi050 ceph-mon[36512]: pgmap v472: 129 pgs: 129 active+clean; 804 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.7 MiB/s wr, 361 op/s 2024-02-14T15:18:21.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 25918 19.11 MB/sec warmup 82 sec latency 224.260 ms 2024-02-14T15:18:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:20 smithi076 ceph-mon[37057]: pgmap v472: 129 pgs: 129 active+clean; 804 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.7 MiB/s wr, 361 op/s 2024-02-14T15:18:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:20 smithi017 ceph-mon[32549]: pgmap v472: 129 pgs: 129 active+clean; 804 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.7 MiB/s wr, 361 op/s 2024-02-14T15:18:22.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 26312 19.21 MB/sec warmup 83 sec latency 201.767 ms 2024-02-14T15:18:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 -- 172.21.15.17:0/1982744765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20101450 msgr2=0x7fba20101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 --2- 172.21.15.17:0/1982744765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20101450 0x7fba20101830 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fba10009930 tx=0x7fba1002bc30 comp rx=0 tx=0).stop 2024-02-14T15:18:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 -- 172.21.15.17:0/1982744765 shutdown_connections 2024-02-14T15:18:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 --2- 172.21.15.17:0/1982744765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba20106520 0x7fba2010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 --2- 172.21.15.17:0/1982744765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba20101e00 0x7fba20105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 --2- 172.21.15.17:0/1982744765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20101450 0x7fba20101830 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 -- 172.21.15.17:0/1982744765 >> 172.21.15.17:0/1982744765 conn(0x7fba200fccb0 msgr2=0x7fba200ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 -- 172.21.15.17:0/1982744765 shutdown_connections 2024-02-14T15:18:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.163+0000 7fba263a1640 1 -- 172.21.15.17:0/1982744765 wait complete. 2024-02-14T15:18:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 Processor -- start 2024-02-14T15:18:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 -- start start 2024-02-14T15:18:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba20101450 0x7fba2019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20106520 0x7fba2019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba20197740 0x7fba20197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba20198210 con 0x7fba20106520 2024-02-14T15:18:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba20198380 con 0x7fba20101450 2024-02-14T15:18:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba263a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba20198540 con 0x7fba20197740 2024-02-14T15:18:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba1f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20106520 0x7fba2019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba1f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20106520 0x7fba2019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42440/0 (socket says 172.21.15.17:42440) 2024-02-14T15:18:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba1f7fe640 1 -- 172.21.15.17:0/1750279155 learned_addr learned my addr 172.21.15.17:0/1750279155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba20101450 0x7fba2019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.164+0000 7fba2493e640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba20197740 0x7fba20197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1f7fe640 1 -- 172.21.15.17:0/1750279155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba20197740 msgr2=0x7fba20197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1f7fe640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba20197740 0x7fba20197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1f7fe640 1 -- 172.21.15.17:0/1750279155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba20101450 msgr2=0x7fba2019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1f7fe640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba20101450 0x7fba2019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1f7fe640 1 -- 172.21.15.17:0/1750279155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10009560 con 0x7fba20106520 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1f7fe640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20106520 0x7fba2019db70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9c0 tx=0x7fba1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1d7fa640 1 -- 172.21.15.17:0/1750279155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14009700 con 0x7fba20106520 2024-02-14T15:18:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba201a4490 con 0x7fba20106520 2024-02-14T15:18:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1d7fa640 1 -- 172.21.15.17:0/1750279155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba1400cd80 con 0x7fba20106520 2024-02-14T15:18:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba1d7fa640 1 -- 172.21.15.17:0/1750279155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14010430 con 0x7fba20106520 2024-02-14T15:18:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.165+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba201a4970 con 0x7fba20106520 2024-02-14T15:18:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.167+0000 7fba1d7fa640 1 -- 172.21.15.17:0/1750279155 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fba14005000 con 0x7fba20106520 2024-02-14T15:18:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 Processor -- start 2024-02-14T15:18:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 -- start start 2024-02-14T15:18:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1d7fa640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba080a2220 0x7fba080a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b0cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b18010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618a7a9e0 con 0x558618b0c720 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618a77f10 con 0x558618b0d040 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba28f71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618a736f0 con 0x558618b15c20 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1d7fa640 1 -- 172.21.15.17:0/1750279155 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fba14014070 con 0x7fba20106520 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba2493e640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba080a2220 0x7fba080a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b0cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b0cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42456/0 (socket says 172.21.15.17:42456) 2024-02-14T15:18:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 -- 172.21.15.17:0/1128208882 learned_addr learned my addr 172.21.15.17:0/1128208882 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba2493e640 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b18010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1f7fe640 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 -- 172.21.15.17:0/1128208882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 msgr2=0x558618b18010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b18010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 -- 172.21.15.17:0/1128208882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 msgr2=0x558618b156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b156e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.168+0000 7fba1ffff640 1 -- 172.21.15.17:0/1128208882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558618784920 con 0x558618b0c720 2024-02-14T15:18:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.169+0000 7fba2493e640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba080a2220 0x7fba080a46e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba18009fd0 tx=0x7fba18009d80 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.169+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b0cb00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fba10036040 tx=0x7fba1002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bac6fc31d0b722e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.169+0000 7fba067fc640 1 -- 172.21.15.17:0/1128208882 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10004800 con 0x558618b0c720 2024-02-14T15:18:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.169+0000 7fba067fc640 1 -- 172.21.15.17:0/1128208882 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba10004960 con 0x558618b0c720 2024-02-14T15:18:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.169+0000 7fba067fc640 1 -- 172.21.15.17:0/1128208882 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10005050 con 0x558618b0c720 2024-02-14T15:18:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 -- 172.21.15.17:0/1128208882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 msgr2=0x558618b0cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b0cb00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fba10036040 tx=0x7fba1002c5d0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 -- 172.21.15.17:0/1128208882 shutdown_connections 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b18010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 --2- 172.21.15.17:0/1128208882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b0cb00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 -- 172.21.15.17:0/1128208882 >> 172.21.15.17:0/1128208882 conn(0x558618b06e50 msgr2=0x558618b07230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 -- 172.21.15.17:0/1128208882 shutdown_connections 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 -- 172.21.15.17:0/1128208882 wait complete. 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 -- start start 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.170+0000 7fba28f71740 1 Processor -- start 2024-02-14T15:18:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba28f71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba28f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b12910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba28f71740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b12e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba28f71740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618a0ffa0 con 0x558618b15c20 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba28f71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618747250 con 0x558618b0d040 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba28f71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586189f9200 con 0x558618b0c720 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba28f71740 10 client.0 _ll_register_callbacks cb 0x558618a68b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b12e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba1f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b12910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b12e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42470/0 (socket says 172.21.15.17:42470) 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 -- 172.21.15.17:0/3063540899 learned_addr learned my addr 172.21.15.17:0/3063540899 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b15510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:54172/0 (socket says 172.21.15.17:54172) 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 -- 172.21.15.17:0/3063540899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 msgr2=0x558618b15510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b15510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 -- 172.21.15.17:0/3063540899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 msgr2=0x558618b12910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b12910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 -- 172.21.15.17:0/3063540899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2010e900 con 0x558618b15c20 2024-02-14T15:18:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fba2493e640 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b12e70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fba1800fe10 tx=0x7fba1801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b92c1cc51986b05 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.171+0000 7fb9ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:18:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/3063540899 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18013330 con 0x558618b15c20 2024-02-14T15:18:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/3063540899 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fba18025030 con 0x558618b15c20 2024-02-14T15:18:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558618773600 con 0x558618b15c20 2024-02-14T15:18:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/3063540899 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba18003e00 con 0x558618b15c20 2024-02-14T15:18:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 10 client.17517 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558618a71d70 con 0x558618b15c20 2024-02-14T15:18:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/3063540899 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558618773600 con 0x558618b15c20 2024-02-14T15:18:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/3063540899 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fba18010cf0 con 0x558618b15c20 2024-02-14T15:18:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 10 client.17517 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 20 client.17517 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b29d60 0x558618b2c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 4 client.17517 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:18:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.172+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586187c41b0 con 0x558618b29d60 2024-02-14T15:18:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.173+0000 7fba1ffff640 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b29d60 0x558618b2c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.173+0000 7fb9ff7fe640 10 client.17517 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.173+0000 7fba1ffff640 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b29d60 0x558618b2c180 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d4023be06fc0a0 server_cookie=d17844a797e8f645 in_seq=0 out_seq=0 2024-02-14T15:18:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/3063540899 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586187c41b0 con 0x558618b29d60 2024-02-14T15:18:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fb9ff7fe640 10 client.17517 handle_command_reply: tid=0 2024-02-14T15:18:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 client.17517 shutdown 2024-02-14T15:18:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 msgr2=0x558618b12e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b12e70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fba1800fe10 tx=0x7fba1801d000 comp rx=0 tx=0).stop 2024-02-14T15:18:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 shutdown_connections 2024-02-14T15:18:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b29d60 0x558618b2c180 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b12e70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b12910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 --2- 172.21.15.17:0/3063540899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b15510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fb9ff7fe640 0 client.17517 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 >> 172.21.15.17:0/3063540899 conn(0x558618b06e50 msgr2=0x558618b08ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 shutdown_connections 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 1 -- 172.21.15.17:0/3063540899 wait complete. 2024-02-14T15:18:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.174+0000 7fba28f71740 20 client.17517 trim_cache size 0 max 16384 2024-02-14T15:18:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 Processor -- start 2024-02-14T15:18:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 -- start start 2024-02-14T15:18:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b12e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b0fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618b0b810 con 0x558618b0c720 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618b26820 con 0x558618b0d040 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba28f71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618b13da0 con 0x558618b15c20 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b12e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b12e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42486/0 (socket says 172.21.15.17:42486) 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba1ffff640 1 -- 172.21.15.17:0/1674480583 learned_addr learned my addr 172.21.15.17:0/1674480583 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba2493e640 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.188+0000 7fba1f7fe640 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b0fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba1ffff640 1 -- 172.21.15.17:0/1674480583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 msgr2=0x558618b100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba1ffff640 1 -- 172.21.15.17:0/1674480583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 msgr2=0x558618b0fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b0fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba1ffff640 1 -- 172.21.15.17:0/1674480583 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558618a0e820 con 0x558618b0c720 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba2493e640 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b12e60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fba10036040 tx=0x7fba100043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847568be5e8f29d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba05ffb640 1 -- 172.21.15.17:0/1674480583 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1002c7e0 con 0x558618b0c720 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba05ffb640 1 -- 172.21.15.17:0/1674480583 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba10051820 con 0x558618b0c720 2024-02-14T15:18:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.189+0000 7fba05ffb640 1 -- 172.21.15.17:0/1674480583 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100051e0 con 0x558618b0c720 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- 172.21.15.17:0/1674480583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 msgr2=0x558618b12e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b12e60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fba10036040 tx=0x7fba100043a0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- 172.21.15.17:0/1674480583 shutdown_connections 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b15c20 0x558618b100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b0fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 --2- 172.21.15.17:0/1674480583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b0c720 0x558618b12e60 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- 172.21.15.17:0/1674480583 >> 172.21.15.17:0/1674480583 conn(0x558618b06e50 msgr2=0x558618b148c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- 172.21.15.17:0/1674480583 shutdown_connections 2024-02-14T15:18:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- 172.21.15.17:0/1674480583 wait complete. 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- start start 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 Processor -- start 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b1fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b20330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b80290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618b20c00 con 0x558618b15c20 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618b807d0 con 0x558618b0d040 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.190+0000 7fba28f71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558618b80910 con 0x558618b0c720 2024-02-14T15:18:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba28f71740 10 client.0 _ll_register_callbacks cb 0x7fba14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b80290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b1fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b80290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42498/0 (socket says 172.21.15.17:42498) 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 -- 172.21.15.17:0/1870434331 learned_addr learned my addr 172.21.15.17:0/1870434331 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba1f7fe640 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b20330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 -- 172.21.15.17:0/1870434331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 msgr2=0x558618b1fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b1fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 -- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 msgr2=0x558618b20330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b20330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 -- 172.21.15.17:0/1870434331 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558618a71d70 con 0x558618b15c20 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba2493e640 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b80290 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fba1800fe10 tx=0x7fba180135e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9636c86e7468e111 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fb9ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/1870434331 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18009ad0 con 0x558618b15c20 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/1870434331 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fba18025030 con 0x558618b15c20 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba10004960 con 0x558618b15c20 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/1870434331 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba18003e00 con 0x558618b15c20 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba28f71740 10 client.17523 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.191+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558618784920 con 0x558618b15c20 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/1870434331 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba10004960 con 0x558618b15c20 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/1870434331 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fba180055b0 con 0x558618b15c20 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba28f71740 10 client.17523 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba28f71740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba28f71740 20 client.17523 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba28f71740 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b81fe0 0x558618b84400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba28f71740 4 client.17523 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5586187b0550 con 0x558618b81fe0 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b81fe0 0x558618b84400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fb9ff7fe640 10 client.17523 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.192+0000 7fba1ffff640 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b81fe0 0x558618b84400 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8a4f2c1a811688 server_cookie=3ec7efecf5926534 in_seq=0 out_seq=0 2024-02-14T15:18:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.233+0000 7fb9ff7fe640 1 -- 172.21.15.17:0/1870434331 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5586187b0550 con 0x558618b81fe0 2024-02-14T15:18:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.233+0000 7fb9ff7fe640 10 client.17523 handle_command_reply: tid=0 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.233+0000 7fba28f71740 1 client.17523 shutdown 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.233+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 msgr2=0x558618b80290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.233+0000 7fba28f71740 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b80290 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fba1800fe10 tx=0x7fba180135e0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 shutdown_connections 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x558618b81fe0 0x558618b84400 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558618b15c20 0x558618b80290 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fb9ff7fe640 0 client.17523 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558618b0d040 0x558618b20330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.236 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 --2- 172.21.15.17:0/1870434331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558618b0c720 0x558618b1fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 >> 172.21.15.17:0/1870434331 conn(0x558618b06e50 msgr2=0x558618b13f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 shutdown_connections 2024-02-14T15:18:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 1 -- 172.21.15.17:0/1870434331 wait complete. 2024-02-14T15:18:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba28f71740 20 client.17523 trim_cache size 0 max 16384 2024-02-14T15:18:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:18:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:18:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:18:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba080a2220 msgr2=0x7fba080a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba080a2220 0x7fba080a46e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba18009fd0 tx=0x7fba18009d80 comp rx=0 tx=0).stop 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20106520 msgr2=0x7fba2019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20106520 0x7fba2019db70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9c0 tx=0x7fba1400ee80 comp rx=0 tx=0).stop 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 shutdown_connections 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fba080a2220 0x7fba080a46e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba20106520 0x7fba2019db70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba20197740 0x7fba20197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 --2- 172.21.15.17:0/1750279155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba20101450 0x7fba2019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 >> 172.21.15.17:0/1750279155 conn(0x7fba200fccb0 msgr2=0x7fba200fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.234+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 shutdown_connections 2024-02-14T15:18:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:22.235+0000 7fba263a1640 1 -- 172.21.15.17:0/1750279155 wait complete. 2024-02-14T15:18:23.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:22 smithi050 ceph-mon[36512]: pgmap v473: 129 pgs: 129 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.3 MiB/s wr, 436 op/s 2024-02-14T15:18:23.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 26569 19.18 MB/sec warmup 84 sec latency 418.993 ms 2024-02-14T15:18:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:22 smithi076 ceph-mon[37057]: pgmap v473: 129 pgs: 129 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.3 MiB/s wr, 436 op/s 2024-02-14T15:18:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:22 smithi017 ceph-mon[32549]: pgmap v473: 129 pgs: 129 active+clean; 820 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.3 MiB/s wr, 436 op/s 2024-02-14T15:18:24.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 26808 19.14 MB/sec warmup 85 sec latency 279.570 ms 2024-02-14T15:18:25.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:24 smithi050 ceph-mon[36512]: pgmap v474: 129 pgs: 129 active+clean; 828 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 385 op/s 2024-02-14T15:18:25.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 27104 19.05 MB/sec warmup 86 sec latency 213.958 ms 2024-02-14T15:18:25.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:24 smithi076 ceph-mon[37057]: pgmap v474: 129 pgs: 129 active+clean; 828 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 385 op/s 2024-02-14T15:18:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:24 smithi017 ceph-mon[32549]: pgmap v474: 129 pgs: 129 active+clean; 828 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 385 op/s 2024-02-14T15:18:26.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 27328 18.91 MB/sec warmup 87 sec latency 75.298 ms 2024-02-14T15:18:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:26 smithi050 ceph-mon[36512]: pgmap v475: 129 pgs: 129 active+clean; 828 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 385 op/s 2024-02-14T15:18:27.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 27687 18.92 MB/sec warmup 88 sec latency 62.362 ms 2024-02-14T15:18:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:26 smithi076 ceph-mon[37057]: pgmap v475: 129 pgs: 129 active+clean; 828 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 385 op/s 2024-02-14T15:18:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:26 smithi017 ceph-mon[32549]: pgmap v475: 129 pgs: 129 active+clean; 828 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 9.1 MiB/s wr, 385 op/s 2024-02-14T15:18:28.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 27997 18.88 MB/sec warmup 89 sec latency 274.082 ms 2024-02-14T15:18:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:28 smithi050 ceph-mon[36512]: pgmap v476: 129 pgs: 129 active+clean; 843 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 11 MiB/s wr, 478 op/s 2024-02-14T15:18:29.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 28285 18.83 MB/sec warmup 90 sec latency 398.841 ms 2024-02-14T15:18:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:28 smithi076 ceph-mon[37057]: pgmap v476: 129 pgs: 129 active+clean; 843 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 11 MiB/s wr, 478 op/s 2024-02-14T15:18:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:28 smithi017 ceph-mon[32549]: pgmap v476: 129 pgs: 129 active+clean; 843 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 11 MiB/s wr, 478 op/s 2024-02-14T15:18:30.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 28782 18.90 MB/sec warmup 91 sec latency 197.191 ms 2024-02-14T15:18:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:30 smithi050 ceph-mon[36512]: pgmap v477: 129 pgs: 129 active+clean; 843 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 309 op/s 2024-02-14T15:18:31.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 29251 18.99 MB/sec warmup 92 sec latency 222.918 ms 2024-02-14T15:18:31.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:30 smithi076 ceph-mon[37057]: pgmap v477: 129 pgs: 129 active+clean; 843 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 309 op/s 2024-02-14T15:18:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:30 smithi017 ceph-mon[32549]: pgmap v477: 129 pgs: 129 active+clean; 843 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 309 op/s 2024-02-14T15:18:32.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 29633 19.06 MB/sec warmup 93 sec latency 192.471 ms 2024-02-14T15:18:33.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:32 smithi050 ceph-mon[36512]: pgmap v478: 129 pgs: 129 active+clean; 852 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.6 MiB/s wr, 379 op/s 2024-02-14T15:18:33.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 29927 19.10 MB/sec warmup 94 sec latency 275.438 ms 2024-02-14T15:18:33.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:32 smithi076 ceph-mon[37057]: pgmap v478: 129 pgs: 129 active+clean; 852 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.6 MiB/s wr, 379 op/s 2024-02-14T15:18:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:32 smithi017 ceph-mon[32549]: pgmap v478: 129 pgs: 129 active+clean; 852 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.6 MiB/s wr, 379 op/s 2024-02-14T15:18:34.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 30213 19.11 MB/sec warmup 95 sec latency 150.444 ms 2024-02-14T15:18:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:34 smithi050 ceph-mon[36512]: pgmap v479: 129 pgs: 129 active+clean; 858 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 344 op/s 2024-02-14T15:18:35.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 30447 18.97 MB/sec warmup 96 sec latency 110.295 ms 2024-02-14T15:18:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:34 smithi076 ceph-mon[37057]: pgmap v479: 129 pgs: 129 active+clean; 858 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 344 op/s 2024-02-14T15:18:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:34 smithi017 ceph-mon[32549]: pgmap v479: 129 pgs: 129 active+clean; 858 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 344 op/s 2024-02-14T15:18:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 -- 172.21.15.17:0/727109605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f252406a500 msgr2=0x7f2524105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 --2- 172.21.15.17:0/727109605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f252406a500 0x7f2524105ec0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2518009930 tx=0x7f251802bc30 comp rx=0 tx=0).stop 2024-02-14T15:18:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 -- 172.21.15.17:0/727109605 shutdown_connections 2024-02-14T15:18:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 --2- 172.21.15.17:0/727109605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f252406a500 0x7f2524105ec0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 --2- 172.21.15.17:0/727109605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25241065a0 0x7f252410e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 --2- 172.21.15.17:0/727109605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2524069b50 0x7f2524069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 -- 172.21.15.17:0/727109605 >> 172.21.15.17:0/727109605 conn(0x7f2524077220 msgr2=0x7f2524077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 -- 172.21.15.17:0/727109605 shutdown_connections 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.333+0000 7f2528b16640 1 -- 172.21.15.17:0/727109605 wait complete. 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 Processor -- start 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 -- start start 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2524069b50 0x7f252419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25241065a0 0x7f252419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2524197760 0x7f2524197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524198230 con 0x7f25241065a0 2024-02-14T15:18:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25241983a0 con 0x7f2524069b50 2024-02-14T15:18:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2528b16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524198560 con 0x7f2524197760 2024-02-14T15:18:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2521d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25241065a0 0x7f252419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2521d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25241065a0 0x7f252419db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34962/0 (socket says 172.21.15.17:34962) 2024-02-14T15:18:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2521d9b640 1 -- 172.21.15.17:0/1816594426 learned_addr learned my addr 172.21.15.17:0/1816594426 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.334+0000 7f2522d9d640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2524197760 0x7f2524197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f252259c640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2524069b50 0x7f252419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2521d9b640 1 -- 172.21.15.17:0/1816594426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2524197760 msgr2=0x7f2524197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2521d9b640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2524197760 0x7f2524197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2521d9b640 1 -- 172.21.15.17:0/1816594426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2524069b50 msgr2=0x7f252419d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2521d9b640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2524069b50 0x7f252419d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2521d9b640 1 -- 172.21.15.17:0/1816594426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518009560 con 0x7f25241065a0 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2521d9b640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25241065a0 0x7f252419db40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f2518005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f25137fe640 1 -- 172.21.15.17:0/1816594426 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518004350 con 0x7f25241065a0 2024-02-14T15:18:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25241a4400 con 0x7f25241065a0 2024-02-14T15:18:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f25137fe640 1 -- 172.21.15.17:0/1816594426 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25180044b0 con 0x7f25241065a0 2024-02-14T15:18:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f25137fe640 1 -- 172.21.15.17:0/1816594426 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518003cd0 con 0x7f25241065a0 2024-02-14T15:18:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.335+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25241a4930 con 0x7f25241065a0 2024-02-14T15:18:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.338+0000 7f25137fe640 1 -- 172.21.15.17:0/1816594426 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2518004df0 con 0x7f25241065a0 2024-02-14T15:18:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.338+0000 7f252b7b6740 1 Processor -- start 2024-02-14T15:18:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.338+0000 7f252b7b6740 1 -- start start 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8d190 0x561107b8d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8db40 0x561107b8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b9f350 0x561107ba1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f252b7b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107afb9e0 con 0x561107b8db40 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f252b7b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107af8f10 con 0x561107b8d190 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f252b7b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107af46f0 con 0x561107b9f350 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8d190 0x561107b8d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f252259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8db40 0x561107b8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8d190 0x561107b8d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:52452/0 (socket says 172.21.15.17:52452) 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 -- 172.21.15.17:0/2359512130 learned_addr learned my addr 172.21.15.17:0/2359512130 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f25137fe640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f250c0a2220 0x7f250c0a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2521d9b640 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b9f350 0x561107ba1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f250c0a2220 0x7f250c0a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 -- 172.21.15.17:0/2359512130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b9f350 msgr2=0x561107ba1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b9f350 0x561107ba1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 -- 172.21.15.17:0/2359512130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8db40 msgr2=0x561107b8dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8db40 0x561107b8dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.339+0000 7f2522d9d640 1 -- 172.21.15.17:0/2359512130 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561107805920 con 0x561107b8d190 2024-02-14T15:18:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.340+0000 7f252259c640 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8db40 0x561107b8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.340+0000 7f25137fe640 1 -- 172.21.15.17:0/1816594426 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f25180f0390 con 0x7f25241065a0 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.340+0000 7f2522d9d640 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8d190 0x561107b8d570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b920 tx=0x7f251c00bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b809bb83adc78a20 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.340+0000 7f2510ff9640 1 -- 172.21.15.17:0/2359512130 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c021030 con 0x561107b8d190 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.340+0000 7f2510ff9640 1 -- 172.21.15.17:0/2359512130 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f251c0042e0 con 0x561107b8d190 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.340+0000 7f2510ff9640 1 -- 172.21.15.17:0/2359512130 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c017070 con 0x561107b8d190 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.340+0000 7f2522d9d640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f250c0a2220 0x7f250c0a46e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f251c021000 tx=0x7f251c01e000 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 -- 172.21.15.17:0/2359512130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8d190 msgr2=0x561107b8d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8d190 0x561107b8d570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b920 tx=0x7f251c00bde0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 -- 172.21.15.17:0/2359512130 shutdown_connections 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8db40 0x561107b8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b9f350 0x561107ba1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 --2- 172.21.15.17:0/2359512130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8d190 0x561107b8d570 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 -- 172.21.15.17:0/2359512130 >> 172.21.15.17:0/2359512130 conn(0x561107b87e50 msgr2=0x561107b88230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 -- 172.21.15.17:0/2359512130 shutdown_connections 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 -- 172.21.15.17:0/2359512130 wait complete. 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 -- start start 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 Processor -- start 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107b92070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.341+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107b8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107b8f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107a90fa0 con 0x561107b8d190 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611077c8250 con 0x561107b9f350 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107a7a200 con 0x561107b8db40 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 10 client.0 _ll_register_callbacks cb 0x561107ae9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107b92070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2521d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107b8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2521d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107b8f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:49416/0 (socket says 172.21.15.17:49416) 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107b92070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34980/0 (socket says 172.21.15.17:34980) 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2521d9b640 1 -- 172.21.15.17:0/3893239059 learned_addr learned my addr 172.21.15.17:0/3893239059 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2522d9d640 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107b8f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 -- 172.21.15.17:0/3893239059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 msgr2=0x561107b8f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107b8f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 -- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 msgr2=0x561107b8f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107b8f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 -- 172.21.15.17:0/3893239059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c004460 con 0x561107b8d190 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2522d9d640 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107b8f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252259c640 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107b92070 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2514005fd0 tx=0x7f251400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d270da651464388b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2509ffb640 1 -- 172.21.15.17:0/3893239059 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514016070 con 0x561107b8d190 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611078451b0 con 0x561107b8d190 2024-02-14T15:18:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2509ffb640 1 -- 172.21.15.17:0/3893239059 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f2514003970 con 0x561107b8d190 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f2509ffb640 1 -- 172.21.15.17:0/3893239059 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2514003b30 con 0x561107b8d190 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 10 client.17535 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.342+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561107af2d70 con 0x561107b8d190 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f2509ffb640 1 -- 172.21.15.17:0/3893239059 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611078451b0 con 0x561107b8d190 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f2509ffb640 1 -- 172.21.15.17:0/3893239059 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2514019470 con 0x561107b8d190 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f252b7b6740 10 client.17535 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f252b7b6740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f252b7b6740 20 client.17535 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107baf170 0x561107bb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f252b7b6740 4 client.17535 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:18:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561107831550 con 0x561107baf170 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f2521d9b640 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107baf170 0x561107bb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f2521d9b640 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107baf170 0x561107bb1590 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c85c583b2fcd6f server_cookie=bf40534b333a87b1 in_seq=0 out_seq=0 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.343+0000 7f2509ffb640 10 client.17535 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f2509ffb640 1 -- 172.21.15.17:0/3893239059 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561107831550 con 0x561107baf170 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f2509ffb640 10 client.17535 handle_command_reply: tid=0 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 client.17535 shutdown 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 msgr2=0x561107b92070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107b92070 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2514005fd0 tx=0x7f251400f790 comp rx=0 tx=0).stop 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 shutdown_connections 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107baf170 0x561107bb1590 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107b8f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107b8f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3893239059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107b92070 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f2509ffb640 0 client.17535 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 >> 172.21.15.17:0/3893239059 conn(0x561107b87e50 msgr2=0x561107b89ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 shutdown_connections 2024-02-14T15:18:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 1 -- 172.21.15.17:0/3893239059 wait complete. 2024-02-14T15:18:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.344+0000 7f252b7b6740 20 client.17535 trim_cache size 0 max 16384 2024-02-14T15:18:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.359+0000 7f252b7b6740 1 Processor -- start 2024-02-14T15:18:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.359+0000 7f252b7b6740 1 -- start start 2024-02-14T15:18:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8d190 0x561107bb5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8db40 0x561107ba62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b9f350 0x561107ba6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f252b7b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107baef70 con 0x561107b9f350 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f252b7b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107b92750 con 0x561107b8db40 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f252b7b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107b88790 con 0x561107b8d190 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b9f350 0x561107ba6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b9f350 0x561107ba6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34982/0 (socket says 172.21.15.17:34982) 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 -- 172.21.15.17:0/625276397 learned_addr learned my addr 172.21.15.17:0/625276397 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2521d9b640 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8db40 0x561107ba62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f252259c640 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8d190 0x561107bb5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 -- 172.21.15.17:0/625276397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8d190 msgr2=0x561107bb5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8d190 0x561107bb5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 -- 172.21.15.17:0/625276397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8db40 msgr2=0x561107ba62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8db40 0x561107ba62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 -- 172.21.15.17:0/625276397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561107a8f820 con 0x561107b9f350 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f2522d9d640 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b9f350 0x561107ba6830 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f251c00bfd0 tx=0x7f251c01ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3163d0105a7d9c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f250bfff640 1 -- 172.21.15.17:0/625276397 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c017070 con 0x561107b9f350 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f250bfff640 1 -- 172.21.15.17:0/625276397 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f251c02c040 con 0x561107b9f350 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.360+0000 7f250bfff640 1 -- 172.21.15.17:0/625276397 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c01bb00 con 0x561107b9f350 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 -- 172.21.15.17:0/625276397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b9f350 msgr2=0x561107ba6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b9f350 0x561107ba6830 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f251c00bfd0 tx=0x7f251c01ccd0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 -- 172.21.15.17:0/625276397 shutdown_connections 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b9f350 0x561107ba6830 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b8db40 0x561107ba62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 --2- 172.21.15.17:0/625276397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8d190 0x561107bb5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 -- 172.21.15.17:0/625276397 >> 172.21.15.17:0/625276397 conn(0x561107b87e50 msgr2=0x561107b8df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 -- 172.21.15.17:0/625276397 shutdown_connections 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.361+0000 7f252b7b6740 1 -- 172.21.15.17:0/625276397 wait complete. 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 -- start start 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 Processor -- start 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107ba2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107ba31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107ba36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107ba6b90 con 0x561107b8d190 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107c05670 con 0x561107b9f350 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561107c057b0 con 0x561107b8db40 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252b7b6740 10 client.0 _ll_register_callbacks cb 0x7f251c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107ba2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107ba2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34984/0 (socket says 172.21.15.17:34984) 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 -- 172.21.15.17:0/3118406855 learned_addr learned my addr 172.21.15.17:0/3118406855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f2521d9b640 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107ba31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f2522d9d640 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107ba36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 -- 172.21.15.17:0/3118406855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 msgr2=0x561107ba31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107ba31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 -- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 msgr2=0x561107ba36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107ba36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 -- 172.21.15.17:0/3118406855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561107805920 con 0x561107b8d190 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f252259c640 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107ba2c70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2514005fd0 tx=0x7f2514009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1480f8d1ec9bb963 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.362+0000 7f2509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f2509ffb640 1 -- 172.21.15.17:0/3118406855 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514016070 con 0x561107b8d190 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f2509ffb640 1 -- 172.21.15.17:0/3118406855 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f2514019bd0 con 0x561107b8d190 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611077f4600 con 0x561107b8d190 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f2509ffb640 1 -- 172.21.15.17:0/3118406855 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2514019370 con 0x561107b8d190 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 10 client.17541 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561107ab5720 con 0x561107b8d190 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f2509ffb640 1 -- 172.21.15.17:0/3118406855 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5611077f4600 con 0x561107b8d190 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f2509ffb640 1 -- 172.21.15.17:0/3118406855 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2514022040 con 0x561107b8d190 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 10 client.17541 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 20 client.17541 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107c06fe0 0x561107c09400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 4 client.17541 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611077a0520 con 0x561107c06fe0 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.363+0000 7f2521d9b640 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107c06fe0 0x561107c09400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.364+0000 7f2521d9b640 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107c06fe0 0x561107c09400 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ad7362722f17cf server_cookie=43d73717bd683f7a in_seq=0 out_seq=0 2024-02-14T15:18:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.364+0000 7f2509ffb640 10 client.17541 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.448+0000 7f2509ffb640 1 -- 172.21.15.17:0/3118406855 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611077a0520 con 0x561107c06fe0 2024-02-14T15:18:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.448+0000 7f2509ffb640 10 client.17541 handle_command_reply: tid=0 2024-02-14T15:18:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 client.17541 shutdown 2024-02-14T15:18:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:18:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:18:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:18:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:18:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 msgr2=0x561107ba2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107ba2c70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2514005fd0 tx=0x7f2514009be0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 shutdown_connections 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x561107c06fe0 0x561107c09400 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561107b9f350 0x561107ba36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561107b8db40 0x561107ba31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 --2- 172.21.15.17:0/3118406855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561107b8d190 0x561107ba2c70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 >> 172.21.15.17:0/3118406855 conn(0x561107b87e50 msgr2=0x561107bb5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f2509ffb640 0 client.17541 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 shutdown_connections 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.449+0000 7f252b7b6740 1 -- 172.21.15.17:0/3118406855 wait complete. 2024-02-14T15:18:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.450+0000 7f252b7b6740 20 client.17541 trim_cache size 0 max 16384 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f250c0a2220 msgr2=0x7f250c0a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f250c0a2220 0x7f250c0a46e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f251c021000 tx=0x7f251c01e000 comp rx=0 tx=0).stop 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25241065a0 msgr2=0x7f252419db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25241065a0 0x7f252419db40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f2518005e70 comp rx=0 tx=0).stop 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 shutdown_connections 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f250c0a2220 0x7f250c0a46e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25241065a0 0x7f252419db40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2524197760 0x7f2524197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 --2- 172.21.15.17:0/1816594426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2524069b50 0x7f252419d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 >> 172.21.15.17:0/1816594426 conn(0x7f2524077220 msgr2=0x7f25240fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 shutdown_connections 2024-02-14T15:18:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:35.451+0000 7f2528b16640 1 -- 172.21.15.17:0/1816594426 wait complete. 2024-02-14T15:18:36.143 INFO:tasks.workunit.client.0.smithi017.stdout: 10 30651 18.83 MB/sec warmup 97 sec latency 162.501 ms 2024-02-14T15:18:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:36 smithi050 ceph-mon[36512]: pgmap v480: 129 pgs: 129 active+clean; 858 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 269 op/s 2024-02-14T15:18:37.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 30906 18.76 MB/sec warmup 98 sec latency 266.401 ms 2024-02-14T15:18:37.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:36 smithi076 ceph-mon[37057]: pgmap v480: 129 pgs: 129 active+clean; 858 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 269 op/s 2024-02-14T15:18:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:36 smithi017 ceph-mon[32549]: pgmap v480: 129 pgs: 129 active+clean; 858 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 269 op/s 2024-02-14T15:18:38.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 31206 18.71 MB/sec warmup 99 sec latency 227.065 ms 2024-02-14T15:18:39.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:38 smithi050 ceph-mon[36512]: pgmap v481: 129 pgs: 129 active+clean; 873 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.6 MiB/s wr, 380 op/s 2024-02-14T15:18:39.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 31515 18.67 MB/sec warmup 100 sec latency 311.356 ms 2024-02-14T15:18:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:38 smithi076 ceph-mon[37057]: pgmap v481: 129 pgs: 129 active+clean; 873 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.6 MiB/s wr, 380 op/s 2024-02-14T15:18:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:38 smithi017 ceph-mon[32549]: pgmap v481: 129 pgs: 129 active+clean; 873 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.6 MiB/s wr, 380 op/s 2024-02-14T15:18:40.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 31924 18.71 MB/sec warmup 101 sec latency 261.410 ms 2024-02-14T15:18:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:40 smithi050 ceph-mon[36512]: pgmap v482: 129 pgs: 129 active+clean; 873 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-02-14T15:18:41.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 32453 18.78 MB/sec warmup 102 sec latency 165.436 ms 2024-02-14T15:18:41.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:40 smithi076 ceph-mon[37057]: pgmap v482: 129 pgs: 129 active+clean; 873 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-02-14T15:18:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:40 smithi017 ceph-mon[32549]: pgmap v482: 129 pgs: 129 active+clean; 873 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-02-14T15:18:42.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 32877 18.87 MB/sec warmup 103 sec latency 213.790 ms 2024-02-14T15:18:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:42 smithi050 ceph-mon[36512]: pgmap v483: 129 pgs: 129 active+clean; 805 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-02-14T15:18:43.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 33209 18.94 MB/sec warmup 104 sec latency 230.824 ms 2024-02-14T15:18:43.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:42 smithi076 ceph-mon[37057]: pgmap v483: 129 pgs: 129 active+clean; 805 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-02-14T15:18:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:42 smithi017 ceph-mon[32549]: pgmap v483: 129 pgs: 129 active+clean; 805 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-02-14T15:18:44.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 33418 18.90 MB/sec warmup 105 sec latency 448.845 ms 2024-02-14T15:18:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:44 smithi050 ceph-mon[36512]: pgmap v484: 129 pgs: 129 active+clean; 789 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-02-14T15:18:45.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 33647 18.83 MB/sec warmup 106 sec latency 310.835 ms 2024-02-14T15:18:45.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:44 smithi076 ceph-mon[37057]: pgmap v484: 129 pgs: 129 active+clean; 789 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-02-14T15:18:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:44 smithi017 ceph-mon[32549]: pgmap v484: 129 pgs: 129 active+clean; 789 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-02-14T15:18:46.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 33900 18.70 MB/sec warmup 107 sec latency 230.452 ms 2024-02-14T15:18:47.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:46 smithi050 ceph-mon[36512]: pgmap v485: 129 pgs: 129 active+clean; 789 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 251 op/s 2024-02-14T15:18:47.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 34121 18.59 MB/sec warmup 108 sec latency 266.006 ms 2024-02-14T15:18:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:46 smithi076 ceph-mon[37057]: pgmap v485: 129 pgs: 129 active+clean; 789 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 251 op/s 2024-02-14T15:18:47.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:46 smithi017 ceph-mon[32549]: pgmap v485: 129 pgs: 129 active+clean; 789 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 251 op/s 2024-02-14T15:18:48.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 34490 18.61 MB/sec warmup 109 sec latency 81.462 ms 2024-02-14T15:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 -- 172.21.15.17:0/932723075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084103c20 msgr2=0x7f30841040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 --2- 172.21.15.17:0/932723075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084103c20 0x7f30841040d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f307c009ca0 tx=0x7f307c02c100 comp rx=0 tx=0).stop 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 -- 172.21.15.17:0/932723075 shutdown_connections 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 --2- 172.21.15.17:0/932723075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084103c20 0x7f30841040d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 --2- 172.21.15.17:0/932723075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3084069010 0x7f3084069470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 --2- 172.21.15.17:0/932723075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3084106120 0x7f3084068ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 -- 172.21.15.17:0/932723075 >> 172.21.15.17:0/932723075 conn(0x7f3084077220 msgr2=0x7f3084077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 -- 172.21.15.17:0/932723075 shutdown_connections 2024-02-14T15:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.587+0000 7f3089823640 1 -- 172.21.15.17:0/932723075 wait complete. 2024-02-14T15:18:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.588+0000 7f3089823640 1 Processor -- start 2024-02-14T15:18:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.588+0000 7f3089823640 1 -- start start 2024-02-14T15:18:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.588+0000 7f3089823640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3084069010 0x7f308419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.588+0000 7f3089823640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3084103c20 0x7f308419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.588+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3084069010 0x7f308419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3084069010 0x7f308419dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:55282/0 (socket says 172.21.15.17:55282) 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f3082ffd640 1 -- 172.21.15.17:0/3898762288 learned_addr learned my addr 172.21.15.17:0/3898762288 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f30827fc640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3084103c20 0x7f308419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f3089823640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084106120 0x7f30841a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a2d80 con 0x7f3084106120 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a2ef0 con 0x7f3084069010 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a30b0 con 0x7f3084103c20 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f30837fe640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084106120 0x7f30841a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f30827fc640 1 -- 172.21.15.17:0/3898762288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3084069010 msgr2=0x7f308419dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f30827fc640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3084069010 0x7f308419dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f30827fc640 1 -- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084106120 msgr2=0x7f30841a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.589+0000 7f30827fc640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084106120 0x7f30841a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.590+0000 7f30827fc640 1 -- 172.21.15.17:0/3898762288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c009900 con 0x7f3084103c20 2024-02-14T15:18:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.590+0000 7f30827fc640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3084103c20 0x7f308419e400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f307400d680 tx=0x7f307400db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.590+0000 7f3088821640 1 -- 172.21.15.17:0/3898762288 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074005220 con 0x7f3084103c20 2024-02-14T15:18:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.590+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841a3390 con 0x7f3084103c20 2024-02-14T15:18:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.590+0000 7f3088821640 1 -- 172.21.15.17:0/3898762288 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3074009ca0 con 0x7f3084103c20 2024-02-14T15:18:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.590+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841aac10 con 0x7f3084103c20 2024-02-14T15:18:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.591+0000 7f3088821640 1 -- 172.21.15.17:0/3898762288 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074004d90 con 0x7f3084103c20 2024-02-14T15:18:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f3088821640 1 -- 172.21.15.17:0/3898762288 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f30740097c0 con 0x7f3084103c20 2024-02-14T15:18:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 Processor -- start 2024-02-14T15:18:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 -- start start 2024-02-14T15:18:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d13fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d140040 0x55773d1486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d0ad9e0 con 0x55773d13f720 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d0aaf10 con 0x55773d148c20 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.594+0000 7f308c3f8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d0a66f0 con 0x55773d140040 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d13fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d13fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33696/0 (socket says 172.21.15.17:33696) 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 -- 172.21.15.17:0/233304327 learned_addr learned my addr 172.21.15.17:0/233304327 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30827fc640 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f3082ffd640 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d140040 0x55773d1486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 -- 172.21.15.17:0/233304327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d140040 msgr2=0x55773d1486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d140040 0x55773d1486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 -- 172.21.15.17:0/233304327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 msgr2=0x55773d14b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 -- 172.21.15.17:0/233304327 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55773cdb7920 con 0x55773d13f720 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f3088821640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f306c0a2320 0x7f306c0a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f30837fe640 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d13fb00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f307c009430 tx=0x7f307c032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a81027a0d97e82 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f3069ffb640 1 -- 172.21.15.17:0/233304327 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c032540 con 0x55773d13f720 2024-02-14T15:18:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.595+0000 7f3088821640 1 -- 172.21.15.17:0/3898762288 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f30740ce110 con 0x7f3084103c20 2024-02-14T15:18:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f3082ffd640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f306c0a2320 0x7f306c0a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f3069ffb640 1 -- 172.21.15.17:0/233304327 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f307c034440 con 0x55773d13f720 2024-02-14T15:18:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f3069ffb640 1 -- 172.21.15.17:0/233304327 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c004040 con 0x55773d13f720 2024-02-14T15:18:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 -- 172.21.15.17:0/233304327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 msgr2=0x55773d13fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f3082ffd640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f306c0a2320 0x7f306c0a47e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55773d13f1f0 tx=0x7f30780091c0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d13fb00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f307c009430 tx=0x7f307c032e40 comp rx=0 tx=0).stop 2024-02-14T15:18:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 -- 172.21.15.17:0/233304327 shutdown_connections 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d140040 0x55773d1486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 --2- 172.21.15.17:0/233304327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d13fb00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 -- 172.21.15.17:0/233304327 >> 172.21.15.17:0/233304327 conn(0x55773d139e50 msgr2=0x55773d13a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 -- 172.21.15.17:0/233304327 shutdown_connections 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.596+0000 7f308c3f8740 1 -- 172.21.15.17:0/233304327 wait complete. 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 -- start start 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 Processor -- start 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d148560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d140040 0x55773d145940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d145e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d042fa0 con 0x55773d13f720 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773cd7a250 con 0x55773d140040 2024-02-14T15:18:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.597+0000 7f308c3f8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d02c200 con 0x55773d148c20 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f308c3f8740 10 client.0 _ll_register_callbacks cb 0x55773d09bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33704/0 (socket says 172.21.15.17:33704) 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 -- 172.21.15.17:0/1323445151 learned_addr learned my addr 172.21.15.17:0/1323445151 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f30827fc640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d140040 0x55773d145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f30837fe640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 -- 172.21.15.17:0/1323445151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 msgr2=0x55773d145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 -- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d140040 msgr2=0x55773d145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d140040 0x55773d145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 -- 172.21.15.17:0/1323445151 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3084114fc0 con 0x55773d13f720 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f30827fc640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d140040 0x55773d145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3082ffd640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d148560 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f30780149f0 tx=0x7f30780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647799963e2bd6a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:18:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3062ffd640 1 -- 172.21.15.17:0/1323445151 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078021600 con 0x55773d13f720 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f3062ffd640 1 -- 172.21.15.17:0/1323445151 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f307801c040 con 0x55773d13f720 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.598+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55773cda6600 con 0x55773d13f720 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f3062ffd640 1 -- 172.21.15.17:0/1323445151 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3078021760 con 0x55773d13f720 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 10 client.17553 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55773d0a4d70 con 0x55773d13f720 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f3062ffd640 1 -- 172.21.15.17:0/1323445151 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55773cda6600 con 0x55773d13f720 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f3062ffd640 1 -- 172.21.15.17:0/1323445151 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3078029ad0 con 0x55773d13f720 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 10 client.17553 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 10 client.17553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 20 client.17553 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1589e0 0x55773d15add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 4 client.17553 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55773cdf71b0 con 0x55773d1589e0 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.599+0000 7f30827fc640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1589e0 0x55773d15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.600+0000 7f3062ffd640 10 client.17553 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.600+0000 7f30827fc640 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1589e0 0x55773d15add0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75835ecd317e50b0 server_cookie=40f0193ba25206e8 in_seq=0 out_seq=0 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.600+0000 7f3062ffd640 1 -- 172.21.15.17:0/1323445151 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55773cdf71b0 con 0x55773d1589e0 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.600+0000 7f3062ffd640 10 client.17553 handle_command_reply: tid=0 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.600+0000 7f308c3f8740 1 client.17553 shutdown 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 msgr2=0x55773d148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d148560 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f30780149f0 tx=0x7f30780046c0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 shutdown_connections 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1589e0 0x55773d15add0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d140040 0x55773d145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f3062ffd640 0 client.17553 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 --2- 172.21.15.17:0/1323445151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d13f720 0x55773d148560 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 >> 172.21.15.17:0/1323445151 conn(0x55773d139e50 msgr2=0x55773d13bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 shutdown_connections 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 1 -- 172.21.15.17:0/1323445151 wait complete. 2024-02-14T15:18:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.601+0000 7f308c3f8740 20 client.17553 trim_cache size 0 max 16384 2024-02-14T15:18:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.614+0000 7f308c3f8740 1 Processor -- start 2024-02-14T15:18:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.614+0000 7f308c3f8740 1 -- start start 2024-02-14T15:18:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d13f720 0x55773d148020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d148560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d144210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f30827fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d13f720 0x55773d148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d13f720 0x55773d148020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:55312/0 (socket says 172.21.15.17:55312) 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f30827fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33708/0 (socket says 172.21.15.17:33708) 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 -- 172.21.15.17:0/895824658 learned_addr learned my addr 172.21.15.17:0/895824658 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d1355a0 con 0x55773d140040 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d155450 con 0x55773d13f720 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d13f380 con 0x55773d148c20 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f30837fe640 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 -- 172.21.15.17:0/895824658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 msgr2=0x55773d144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 -- 172.21.15.17:0/895824658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 msgr2=0x55773d148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 -- 172.21.15.17:0/895824658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55773d041820 con 0x55773d13f720 2024-02-14T15:18:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f30827fc640 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:18:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.615+0000 7f3082ffd640 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d13f720 0x55773d148020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3078006df0 tx=0x7f3078014eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ce670c1867b318 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f30697fa640 1 -- 172.21.15.17:0/895824658 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078013040 con 0x55773d13f720 2024-02-14T15:18:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f30697fa640 1 -- 172.21.15.17:0/895824658 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30780131a0 con 0x55773d13f720 2024-02-14T15:18:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f30697fa640 1 -- 172.21.15.17:0/895824658 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078021600 con 0x55773d13f720 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d13f720 msgr2=0x55773d148020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d13f720 0x55773d148020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3078006df0 tx=0x7f3078014eb0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 shutdown_connections 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d148c20 0x55773d144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 --2- 172.21.15.17:0/895824658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d13f720 0x55773d148020 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 >> 172.21.15.17:0/895824658 conn(0x55773d139e50 msgr2=0x55773d15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 shutdown_connections 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.616+0000 7f308c3f8740 1 -- 172.21.15.17:0/895824658 wait complete. 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.617+0000 7f308c3f8740 1 -- start start 2024-02-14T15:18:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.617+0000 7f308c3f8740 1 Processor -- start 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.617+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d13f720 0x55773d14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.617+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f308c3f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d13f720 0x55773d14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d14ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33722/0 (socket says 172.21.15.17:33722) 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f3082ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d13f720 0x55773d14fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.17:59752/0 (socket says 172.21.15.17:59752) 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 -- 172.21.15.17:0/9160357 learned_addr learned my addr 172.21.15.17:0/9160357 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d14e700 con 0x55773d140040 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d14e870 con 0x55773d148c20 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d14ea30 con 0x55773d13f720 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30837fe640 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f308c3f8740 10 client.0 _ll_register_callbacks cb 0x7f307c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:18:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 -- 172.21.15.17:0/9160357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d13f720 msgr2=0x55773d14fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d13f720 0x55773d14fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 -- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 msgr2=0x55773d14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 -- 172.21.15.17:0/9160357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841aac10 con 0x55773d140040 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f3082ffd640 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d13f720 0x55773d14fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f30827fc640 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d14ffd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3074004f80 tx=0x7f30740ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293cd4768c4ebb5a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f3062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f3062ffd640 1 -- 172.21.15.17:0/9160357 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740850b0 con 0x55773d140040 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f3062ffd640 1 -- 172.21.15.17:0/9160357 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f307409ade0 con 0x55773d140040 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.618+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f307c034440 con 0x55773d140040 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f3062ffd640 1 -- 172.21.15.17:0/9160357 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f307409a6e0 con 0x55773d140040 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 10 client.17559 fetch_fsmap learned FSMap version 38 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55773d0a4d70 con 0x55773d140040 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f3062ffd640 1 -- 172.21.15.17:0/9160357 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f307c034440 con 0x55773d140040 2024-02-14T15:18:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f3062ffd640 1 -- 172.21.15.17:0/9160357 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3074023070 con 0x55773d140040 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 10 client.17559 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 10 client.17559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 20 client.17559 populate_metadata read hostname 'smithi017' 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1b0140 0x55773d1b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 4 client.17559 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55773cde3550 con 0x55773d1b0140 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.619+0000 7f3082ffd640 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1b0140 0x55773d1b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.620+0000 7f3082ffd640 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1b0140 0x55773d1b2560 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ca3c67f56173ac server_cookie=2882790379a4725d in_seq=0 out_seq=0 2024-02-14T15:18:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.620+0000 7f3062ffd640 10 client.17559 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.717+0000 7f3062ffd640 1 -- 172.21.15.17:0/9160357 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55773cde3550 con 0x55773d1b0140 2024-02-14T15:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.717+0000 7f3062ffd640 10 client.17559 handle_command_reply: tid=0 2024-02-14T15:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.717+0000 7f308c3f8740 1 client.17559 shutdown 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.717+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 msgr2=0x55773d14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.717+0000 7f308c3f8740 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d14ffd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3074004f80 tx=0x7f30740ba040 comp rx=0 tx=0).stop 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.717+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 shutdown_connections 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.717+0000 7f308c3f8740 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x55773d1b0140 0x55773d1b2560 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f308c3f8740 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55773d140040 0x55773d14ffd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f308c3f8740 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55773d148c20 0x55773d14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f308c3f8740 1 --2- 172.21.15.17:0/9160357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55773d13f720 0x55773d14fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 >> 172.21.15.17:0/9160357 conn(0x55773d139e50 msgr2=0x55773d144170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3062ffd640 0 client.17559 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 shutdown_connections 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f308c3f8740 1 -- 172.21.15.17:0/9160357 wait complete. 2024-02-14T15:18:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f308c3f8740 20 client.17559 trim_cache size 0 max 16384 2024-02-14T15:18:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:18:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:18:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:18:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f306c0a2320 msgr2=0x7f306c0a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f306c0a2320 0x7f306c0a47e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55773d13f1f0 tx=0x7f30780091c0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3084103c20 msgr2=0x7f308419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3084103c20 0x7f308419e400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f307400d680 tx=0x7f307400db40 comp rx=0 tx=0).stop 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 shutdown_connections 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f306c0a2320 0x7f306c0a47e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3084106120 0x7f30841a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3084103c20 0x7f308419e400 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 --2- 172.21.15.17:0/3898762288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3084069010 0x7f308419dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.718+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 >> 172.21.15.17:0/3898762288 conn(0x7f3084077220 msgr2=0x7f3084102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.719+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 shutdown_connections 2024-02-14T15:18:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:18:48.719+0000 7f3089823640 1 -- 172.21.15.17:0/3898762288 wait complete. 2024-02-14T15:18:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:48 smithi050 ceph-mon[36512]: pgmap v486: 129 pgs: 129 active+clean; 802 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 372 op/s 2024-02-14T15:18:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:48 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:18:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:48 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:18:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:48 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:18:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:48 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:18:49.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 34810 18.59 MB/sec warmup 110 sec latency 272.400 ms 2024-02-14T15:18:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:48 smithi076 ceph-mon[37057]: pgmap v486: 129 pgs: 129 active+clean; 802 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 372 op/s 2024-02-14T15:18:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:48 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:18:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:48 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:18:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:48 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:18:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:48 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:18:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:48 smithi017 ceph-mon[32549]: pgmap v486: 129 pgs: 129 active+clean; 802 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 372 op/s 2024-02-14T15:18:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:48 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:18:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:48 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:18:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:48 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:18:49.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:48 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:18:50.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 35221 18.65 MB/sec warmup 111 sec latency 181.045 ms 2024-02-14T15:18:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:50 smithi050 ceph-mon[36512]: pgmap v487: 129 pgs: 129 active+clean; 802 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 261 op/s 2024-02-14T15:18:51.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 35731 18.66 MB/sec warmup 112 sec latency 225.147 ms 2024-02-14T15:18:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:50 smithi076 ceph-mon[37057]: pgmap v487: 129 pgs: 129 active+clean; 802 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 261 op/s 2024-02-14T15:18:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:50 smithi017 ceph-mon[32549]: pgmap v487: 129 pgs: 129 active+clean; 802 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 261 op/s 2024-02-14T15:18:52.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 36163 18.84 MB/sec warmup 113 sec latency 202.843 ms 2024-02-14T15:18:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:52 smithi050 ceph-mon[36512]: pgmap v488: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.7 MiB/s wr, 367 op/s 2024-02-14T15:18:53.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 36569 18.90 MB/sec warmup 114 sec latency 235.723 ms 2024-02-14T15:18:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:52 smithi076 ceph-mon[37057]: pgmap v488: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.7 MiB/s wr, 367 op/s 2024-02-14T15:18:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:52 smithi017 ceph-mon[32549]: pgmap v488: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.7 MiB/s wr, 367 op/s 2024-02-14T15:18:54.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 36725 18.80 MB/sec warmup 115 sec latency 471.054 ms 2024-02-14T15:18:55.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:54 smithi050 ceph-mon[36512]: pgmap v489: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 324 op/s 2024-02-14T15:18:55.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 36909 18.74 MB/sec warmup 116 sec latency 176.396 ms 2024-02-14T15:18:55.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:54 smithi076 ceph-mon[37057]: pgmap v489: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 324 op/s 2024-02-14T15:18:55.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:54 smithi017 ceph-mon[32549]: pgmap v489: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 324 op/s 2024-02-14T15:18:56.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 37146 18.66 MB/sec warmup 117 sec latency 85.776 ms 2024-02-14T15:18:57.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:56 smithi050 ceph-mon[36512]: pgmap v490: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 262 op/s 2024-02-14T15:18:57.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 37391 18.55 MB/sec warmup 118 sec latency 97.134 ms 2024-02-14T15:18:57.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:56 smithi076 ceph-mon[37057]: pgmap v490: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 262 op/s 2024-02-14T15:18:57.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:56 smithi017 ceph-mon[32549]: pgmap v490: 129 pgs: 129 active+clean; 817 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 262 op/s 2024-02-14T15:18:58.145 INFO:tasks.workunit.client.0.smithi017.stdout: 10 37645 18.49 MB/sec warmup 119 sec latency 282.862 ms 2024-02-14T15:18:59.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:18:58 smithi050 ceph-mon[36512]: pgmap v491: 129 pgs: 129 active+clean; 831 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.2 MiB/s wr, 388 op/s 2024-02-14T15:18:59.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:18:58 smithi076 ceph-mon[37057]: pgmap v491: 129 pgs: 129 active+clean; 831 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.2 MiB/s wr, 388 op/s 2024-02-14T15:18:59.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:18:58 smithi017 ceph-mon[32549]: pgmap v491: 129 pgs: 129 active+clean; 831 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.2 MiB/s wr, 388 op/s 2024-02-14T15:19:00.144 INFO:tasks.workunit.client.0.smithi017.stdout: 10 38263 25.46 MB/sec execute 1 sec latency 574.051 ms 2024-02-14T15:19:01.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:00 smithi050 ceph-mon[36512]: pgmap v492: 129 pgs: 129 active+clean; 831 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 267 op/s 2024-02-14T15:19:01.145 INFO:tasks.workunit.client.0.smithi017.stdout: 10 38664 19.23 MB/sec execute 2 sec latency 311.054 ms 2024-02-14T15:19:01.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:00 smithi076 ceph-mon[37057]: pgmap v492: 129 pgs: 129 active+clean; 831 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 267 op/s 2024-02-14T15:19:01.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:00 smithi017 ceph-mon[32549]: pgmap v492: 129 pgs: 129 active+clean; 831 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 267 op/s 2024-02-14T15:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.809+0000 7f9a40732640 1 -- 172.21.15.17:0/236407103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 msgr2=0x7f9a3810e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.809+0000 7f9a40732640 1 --2- 172.21.15.17:0/236407103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a3810e0c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009ca0 tx=0x7f9a3002c180 comp rx=0 tx=0).stop 2024-02-14T15:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- 172.21.15.17:0/236407103 shutdown_connections 2024-02-14T15:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 --2- 172.21.15.17:0/236407103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a3810e0c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 --2- 172.21.15.17:0/236407103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a38069860 0x7f9a38069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 --2- 172.21.15.17:0/236407103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a38107320 0x7f9a38107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- 172.21.15.17:0/236407103 >> 172.21.15.17:0/236407103 conn(0x7f9a38076f80 msgr2=0x7f9a38077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- 172.21.15.17:0/236407103 shutdown_connections 2024-02-14T15:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- 172.21.15.17:0/236407103 wait complete. 2024-02-14T15:19:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 Processor -- start 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- start start 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a38069860 0x7f9a38100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a38100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a38107320 0x7f9a38100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a5ba0 con 0x7f9a3806a200 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a5d10 con 0x7f9a38069860 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a40732640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a5ed0 con 0x7f9a38107320 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a3dccd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a38100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a3dccd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a38100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42760/0 (socket says 172.21.15.17:42760) 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a3dccd640 1 -- 172.21.15.17:0/1498680467 learned_addr learned my addr 172.21.15.17:0/1498680467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.810+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a38107320 0x7f9a38100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a38069860 0x7f9a38100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a3dccd640 1 -- 172.21.15.17:0/1498680467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a38107320 msgr2=0x7f9a38100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a38107320 0x7f9a38100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a3dccd640 1 -- 172.21.15.17:0/1498680467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a38069860 msgr2=0x7f9a38100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a38069860 0x7f9a38100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a3dccd640 1 -- 172.21.15.17:0/1498680467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30009900 con 0x7f9a3806a200 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a38100990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d950 tx=0x7f9a2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a2f7fe640 1 -- 172.21.15.17:0/1498680467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28009c30 con 0x7f9a3806a200 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a381a61b0 con 0x7f9a3806a200 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a2f7fe640 1 -- 172.21.15.17:0/1498680467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a28009d90 con 0x7f9a3806a200 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a2f7fe640 1 -- 172.21.15.17:0/1498680467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2800f7e0 con 0x7f9a3806a200 2024-02-14T15:19:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.811+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a381a66c0 con 0x7f9a3806a200 2024-02-14T15:19:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.813+0000 7f9a2f7fe640 1 -- 172.21.15.17:0/1498680467 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9a28010ad0 con 0x7f9a3806a200 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 Processor -- start 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 -- start start 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a2f7fe640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9a240a2220 0x7f9a240a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc15b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc21010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bb839e0 con 0x56480bc15720 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bb80f10 con 0x56480bc16040 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a433d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bb7c6f0 con 0x56480bc1ec20 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a2f7fe640 1 -- 172.21.15.17:0/1498680467 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9a280ce110 con 0x7f9a3806a200 2024-02-14T15:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9a240a2220 0x7f9a240a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a3e4ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc15b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a3e4ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc15b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42766/0 (socket says 172.21.15.17:42766) 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/348629824 learned_addr learned my addr 172.21.15.17:0/348629824 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc21010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.814+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/348629824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 msgr2=0x56480bc21010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc21010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/348629824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 msgr2=0x56480bc1e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/348629824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56480b88d920 con 0x56480bc15720 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9a240a2220 0x7f9a240a46e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a30004af0 tx=0x7f9a300324f0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc15b00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400dc30 tx=0x7f9a3400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed103257d0ae071 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a2cff9640 1 -- 172.21.15.17:0/348629824 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34011440 con 0x56480bc15720 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a2cff9640 1 -- 172.21.15.17:0/348629824 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a34011a80 con 0x56480bc15720 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.815+0000 7f9a2cff9640 1 -- 172.21.15.17:0/348629824 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a340109c0 con 0x56480bc15720 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- 172.21.15.17:0/348629824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 msgr2=0x56480bc15b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc15b00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400dc30 tx=0x7f9a3400f730 comp rx=0 tx=0).stop 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- 172.21.15.17:0/348629824 shutdown_connections 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc21010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 --2- 172.21.15.17:0/348629824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc15b00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- 172.21.15.17:0/348629824 >> 172.21.15.17:0/348629824 conn(0x56480bc0fe50 msgr2=0x56480bc10230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- 172.21.15.17:0/348629824 shutdown_connections 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- 172.21.15.17:0/348629824 wait complete. 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- start start 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 Processor -- start 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc1beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bb18fa0 con 0x56480bc15720 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480b850250 con 0x56480bc16040 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bb02200 con 0x56480bc1ec20 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a433d6740 10 client.0 _ll_register_callbacks cb 0x56480bb71b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:19:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a3e4ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.816+0000 7f9a3e4ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42768/0 (socket says 172.21.15.17:42768) 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/4201950749 learned_addr learned my addr 172.21.15.17:0/4201950749 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc1beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/4201950749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 msgr2=0x56480bc1beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc1beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 msgr2=0x56480bc1b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/4201950749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3400d860 con 0x56480bc15720 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc1beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:19:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1e530 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400df50 tx=0x7f9a3400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76146a5d1456173f server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a21ffb640 1 -- 172.21.15.17:0/4201950749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34022070 con 0x56480bc15720 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a21ffb640 1 -- 172.21.15.17:0/4201950749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9a34004240 con 0x56480bc15720 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56480b87c600 con 0x56480bc15720 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a21ffb640 1 -- 172.21.15.17:0/4201950749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a34004400 con 0x56480bc15720 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a433d6740 10 client.17571 fetch_fsmap learned FSMap version 38 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56480bb7ad70 con 0x56480bc15720 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a21ffb640 1 -- 172.21.15.17:0/4201950749 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56480b87c600 con 0x56480bc15720 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.817+0000 7f9a21ffb640 1 -- 172.21.15.17:0/4201950749 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9a34010740 con 0x56480bc15720 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a433d6740 10 client.17571 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a433d6740 10 client.17571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:19:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a433d6740 20 client.17571 populate_metadata read hostname 'smithi017' 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a433d6740 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc2e990 0x56480bc30db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a433d6740 4 client.17571 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56480b8cd1b0 con 0x56480bc2e990 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc2e990 0x56480bc30db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc2e990 0x56480bc30db0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1080a3a698edd435 server_cookie=d0bb16ca92c19ca1 in_seq=0 out_seq=0 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.818+0000 7f9a21ffb640 10 client.17571 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a21ffb640 1 -- 172.21.15.17:0/4201950749 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56480b8cd1b0 con 0x56480bc2e990 2024-02-14T15:19:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a21ffb640 10 client.17571 handle_command_reply: tid=0 2024-02-14T15:19:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 client.17571 shutdown 2024-02-14T15:19:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 msgr2=0x56480bc1e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1e530 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400df50 tx=0x7f9a3400ff60 comp rx=0 tx=0).stop 2024-02-14T15:19:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 shutdown_connections 2024-02-14T15:19:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc2e990 0x56480bc30db0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc1ec20 0x56480bc1beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a21ffb640 0 client.17571 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc16040 0x56480bc1b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 --2- 172.21.15.17:0/4201950749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1e530 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 >> 172.21.15.17:0/4201950749 conn(0x56480bc0fe50 msgr2=0x56480bc11ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 shutdown_connections 2024-02-14T15:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 1 -- 172.21.15.17:0/4201950749 wait complete. 2024-02-14T15:19:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.819+0000 7f9a433d6740 20 client.17571 trim_cache size 0 max 16384 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 Processor -- start 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 -- start start 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc16040 0x56480bc1e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc18bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bc2ae30 con 0x56480bc15720 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bc1cc80 con 0x56480bc1ec20 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a433d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bc10e50 con 0x56480bc16040 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a3e4ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.832+0000 7f9a3e4ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42782/0 (socket says 172.21.15.17:42782) 2024-02-14T15:19:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/599126950 learned_addr learned my addr 172.21.15.17:0/599126950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc18bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/599126950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc16040 msgr2=0x56480bc1e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc16040 0x56480bc1e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/599126950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 msgr2=0x56480bc18bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc18bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3e4ce640 1 -- 172.21.15.17:0/599126950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56480bb17820 con 0x56480bc15720 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1b580 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9a34010aa0 tx=0x7f9a34005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c95677657223cc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a23fff640 1 -- 172.21.15.17:0/599126950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34022040 con 0x56480bc15720 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a23fff640 1 -- 172.21.15.17:0/599126950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a3400d4a0 con 0x56480bc15720 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a23fff640 1 -- 172.21.15.17:0/599126950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a340035f0 con 0x56480bc15720 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a433d6740 1 -- 172.21.15.17:0/599126950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 msgr2=0x56480bc1b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.833+0000 7f9a433d6740 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1b580 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9a34010aa0 tx=0x7f9a34005f50 comp rx=0 tx=0).stop 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- 172.21.15.17:0/599126950 shutdown_connections 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc18bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc16040 0x56480bc1e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 --2- 172.21.15.17:0/599126950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc15720 0x56480bc1b580 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- 172.21.15.17:0/599126950 >> 172.21.15.17:0/599126950 conn(0x56480bc0fe50 msgr2=0x56480bc1ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- 172.21.15.17:0/599126950 shutdown_connections 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- 172.21.15.17:0/599126950 wait complete. 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- start start 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 Processor -- start 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc15720 0x56480bc25b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc16040 0x56480bc240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc24610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bc263f0 con 0x56480bc16040 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bc24b50 con 0x56480bc1ec20 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56480bc24d10 con 0x56480bc15720 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a433d6740 10 client.0 _ll_register_callbacks cb 0x7f9a3400d3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a3dccd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc16040 0x56480bc240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a3dccd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc16040 0x56480bc240d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42794/0 (socket says 172.21.15.17:42794) 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a3dccd640 1 -- 172.21.15.17:0/3173791005 learned_addr learned my addr 172.21.15.17:0/3173791005 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a3eccf640 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc24610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc15720 0x56480bc25b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.834+0000 7f9a3dccd640 1 -- 172.21.15.17:0/3173791005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc15720 msgr2=0x56480bc25b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc15720 0x56480bc25b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a3dccd640 1 -- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 msgr2=0x56480bc24610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc24610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a3dccd640 1 -- 172.21.15.17:0/3173791005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56480bb7ad70 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a3dccd640 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc16040 0x56480bc240d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009fd0 tx=0x7f9a28010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d63167e9328920b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a21ffb640 1 -- 172.21.15.17:0/3173791005 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a280035f0 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a21ffb640 1 -- 172.21.15.17:0/3173791005 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9a2809ae40 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a34011a80 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a21ffb640 1 -- 172.21.15.17:0/3173791005 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a2809a740 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 10 client.17577 fetch_fsmap learned FSMap version 38 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56480bb3d720 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a21ffb640 1 -- 172.21.15.17:0/3173791005 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a34011a80 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a21ffb640 1 -- 172.21.15.17:0/3173791005 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9a28096040 con 0x56480bc16040 2024-02-14T15:19:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 10 client.17577 fetch_fsmap finished waiting for FSMap version 38 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 10 client.17577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 20 client.17577 populate_metadata read hostname 'smithi017' 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc86270 0x56480bc88690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 4 client.17577 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.835+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 --> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56480b8b9550 con 0x56480bc86270 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.836+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc86270 0x56480bc88690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.836+0000 7f9a21ffb640 10 client.17577 ms_handle_connect on v2:172.21.15.50:6838/916171404 2024-02-14T15:19:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.836+0000 7f9a3e4ce640 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc86270 0x56480bc88690 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544585b0ef839534 server_cookie=1df6165716919c2e in_seq=0 out_seq=0 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.907+0000 7f9a21ffb640 1 -- 172.21.15.17:0/3173791005 <== mds.0 v2:172.21.15.50:6838/916171404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56480b8b9550 con 0x56480bc86270 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.907+0000 7f9a21ffb640 10 client.17577 handle_command_reply: tid=0 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 client.17577 shutdown 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc16040 msgr2=0x56480bc240d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc16040 0x56480bc240d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009fd0 tx=0x7f9a28010650 comp rx=0 tx=0).stop 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 shutdown_connections 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:6838/916171404,v1:172.21.15.50:6839/916171404] conn(0x56480bc86270 0x56480bc88690 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56480bc16040 0x56480bc240d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56480bc1ec20 0x56480bc24610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 --2- 172.21.15.17:0/3173791005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56480bc15720 0x56480bc25b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 >> 172.21.15.17:0/3173791005 conn(0x56480bc0fe50 msgr2=0x56480bc1b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a21ffb640 0 client.17577 ms_handle_reset on v2:172.21.15.50:6838/916171404 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 shutdown_connections 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 1 -- 172.21.15.17:0/3173791005 wait complete. 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.908+0000 7f9a433d6740 20 client.17577 trim_cache size 0 max 16384 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:{ 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "message": "", 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout: "return_code": 0 2024-02-14T15:19:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stdout:} 2024-02-14T15:19:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9a240a2220 msgr2=0x7f9a240a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9a240a2220 0x7f9a240a46e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a30004af0 tx=0x7f9a300324f0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 msgr2=0x7f9a38100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a38100990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d950 tx=0x7f9a2800de10 comp rx=0 tx=0).stop 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 shutdown_connections 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9a240a2220 0x7f9a240a46e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a3806a200 0x7f9a38100990 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a38107320 0x7f9a38100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 --2- 172.21.15.17:0/1498680467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a38069860 0x7f9a38100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 >> 172.21.15.17:0/1498680467 conn(0x7f9a38076f80 msgr2=0x7f9a380fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 shutdown_connections 2024-02-14T15:19:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi017.front.sepia.ceph.com.smithi017.stderr:2024-02-14T15:19:01.909+0000 7f9a40732640 1 -- 172.21.15.17:0/1498680467 wait complete. 2024-02-14T15:19:01.995 INFO:tasks.workunit.client.0.smithi017.stdout:[41162] write failed on handle 10247 (Resource temporarily unavailable) 2024-02-14T15:19:01.995 INFO:tasks.workunit.client.0.smithi017.stdout:Child failed with status 1 2024-02-14T15:19:01.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:19:01.999 INFO:tasks.workunit.client.0.smithi017.stdout:[41162] write failed on handle 10247 (Resource temporarily unavailable) 2024-02-14T15:19:02.009 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-02-14T15:19:02.010 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-14T15:19:02.319 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi017 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-02-14T15:19:02.512 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e555d8617d2541938f36130109b2d731 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi017 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-02-14T15:19:02.514 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-02-14T15:19:02.524 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:19:02.539 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-14T15:19:02.550 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-02-14T15:19:02.550 INFO:teuthology.task.exec:Executing custom commands... 2024-02-14T15:19:02.551 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:19:02.551 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-02-14T15:19:02.608 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-02-14T15:19:02.678 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-14T15:19:02.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/2646860050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100d40 msgr2=0x7fc3d010cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:02.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/2646860050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100d40 0x7fc3d010cfc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009ca0 tx=0x7fc3c802c140 comp rx=0 tx=0).stop 2024-02-14T15:19:02.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/2646860050 shutdown_connections 2024-02-14T15:19:02.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/2646860050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100d40 0x7fc3d010cfc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:02.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/2646860050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3d0100380 0x7fc3d0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:02.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/2646860050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3d0106240 0x7fc3d0106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:02.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/2646860050 >> 172.21.15.17:0/2646860050 conn(0x7fc3d0076f80 msgr2=0x7fc3d0077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/2646860050 shutdown_connections 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/2646860050 wait complete. 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.820+0000 7fc3d6a9a640 1 Processor -- start 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d6a9a640 1 -- start start 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d6a9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100380 0x7fc3d019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d6a9a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3d0100d40 0x7fc3d019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d6a9a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3d0106240 0x7fc3d0197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:02.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d6a9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0197e10 con 0x7fc3d0100380 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d6a9a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0197f80 con 0x7fc3d0106240 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d6a9a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0198140 con 0x7fc3d0100d40 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100380 0x7fc3d019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100380 0x7fc3d019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42814/0 (socket says 172.21.15.17:42814) 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 -- 172.21.15.17:0/1425129276 learned_addr learned my addr 172.21.15.17:0/1425129276 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d5037640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3d0106240 0x7fc3d0197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3cffff640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3d0100d40 0x7fc3d019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 -- 172.21.15.17:0/1425129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3d0100d40 msgr2=0x7fc3d019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:02.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3d0100d40 0x7fc3d019db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 -- 172.21.15.17:0/1425129276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3d0106240 msgr2=0x7fc3d0197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3d0106240 0x7fc3d0197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 -- 172.21.15.17:0/1425129276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8009900 con 0x7fc3d0100380 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d5037640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3d0106240 0x7fc3d0197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3d4836640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100380 0x7fc3d019d5c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d950 tx=0x7fc3c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.821+0000 7fc3cdffb640 1 -- 172.21.15.17:0/1425129276 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4009c30 con 0x7fc3d0100380 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.822+0000 7fc3cdffb640 1 -- 172.21.15.17:0/1425129276 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3c4009d90 con 0x7fc3d0100380 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.822+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d0198420 con 0x7fc3d0100380 2024-02-14T15:19:02.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.822+0000 7fc3cdffb640 1 -- 172.21.15.17:0/1425129276 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400f7e0 con 0x7fc3d0100380 2024-02-14T15:19:02.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.822+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d01a4540 con 0x7fc3d0100380 2024-02-14T15:19:02.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.823+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d0069050 con 0x7fc3d0100380 2024-02-14T15:19:02.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.824+0000 7fc3cdffb640 1 -- 172.21.15.17:0/1425129276 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc3c4010ad0 con 0x7fc3d0100380 2024-02-14T15:19:02.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.824+0000 7fc3cdffb640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc3c00a2220 0x7fc3c00a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:02.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.824+0000 7fc3cdffb640 1 -- 172.21.15.17:0/1425129276 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fc3c40ce0f0 con 0x7fc3d0100380 2024-02-14T15:19:02.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.824+0000 7fc3cffff640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc3c00a2220 0x7fc3c00a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:02.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.825+0000 7fc3cffff640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc3c00a2220 0x7fc3c00a46e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0198ed0 tx=0x7fc3bc005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:02.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:02.829+0000 7fc3cdffb640 1 -- 172.21.15.17:0/1425129276 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3c40969c0 con 0x7fc3d0100380 2024-02-14T15:19:03.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.040+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 --> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d01102f0 con 0x7fc3c00a2220 2024-02-14T15:19:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.050+0000 7fc3cdffb640 1 -- 172.21.15.17:0/1425129276 <== mgr.15681 v2:172.21.15.17:6800/877978339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fc3d01102f0 con 0x7fc3c00a2220 2024-02-14T15:19:03.052 INFO:teuthology.orchestra.run.smithi017.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-14T00:00:00", "created": "2024-02-14T15:03:25", "first": "2024-02-14T15:04:00", "last": "2024-02-14T15:19:00", "last_pruned": "2024-02-14T15:19:01", "created_count": 16, "pruned_count": 10, "active": true} 2024-02-14T15:19:03.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.052+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc3c00a2220 msgr2=0x7fc3c00a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.052+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc3c00a2220 0x7fc3c00a46e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0198ed0 tx=0x7fc3bc005f90 comp rx=0 tx=0).stop 2024-02-14T15:19:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.052+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100380 msgr2=0x7fc3d019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.052+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100380 0x7fc3d019d5c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d950 tx=0x7fc3c400de10 comp rx=0 tx=0).stop 2024-02-14T15:19:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 shutdown_connections 2024-02-14T15:19:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc3c00a2220 0x7fc3c00a46e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3d0106240 0x7fc3d0197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3d0100d40 0x7fc3d019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 --2- 172.21.15.17:0/1425129276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3d0100380 0x7fc3d019d5c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 >> 172.21.15.17:0/1425129276 conn(0x7fc3d0076f80 msgr2=0x7fc3d00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:03.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 shutdown_connections 2024-02-14T15:19:03.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.053+0000 7fc3d6a9a640 1 -- 172.21.15.17:0/1425129276 wait complete. 2024-02-14T15:19:03.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-14T15:19:03.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:02 smithi050 ceph-mon[36512]: pgmap v493: 129 pgs: 129 active+clean; 843 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 348 op/s 2024-02-14T15:19:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.168+0000 7f880301c640 1 -- 172.21.15.17:0/714845900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc100340 msgr2=0x7f87fc100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.168+0000 7f880301c640 1 --2- 172.21.15.17:0/714845900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc100340 0x7f87fc100720 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009cd0 tx=0x7f87f402c0f0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.168+0000 7f880301c640 1 -- 172.21.15.17:0/714845900 shutdown_connections 2024-02-14T15:19:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 --2- 172.21.15.17:0/714845900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87fc105410 0x7f87fc10cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 --2- 172.21.15.17:0/714845900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87fc100cf0 0x7f87fc104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 --2- 172.21.15.17:0/714845900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc100340 0x7f87fc100720 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 -- 172.21.15.17:0/714845900 >> 172.21.15.17:0/714845900 conn(0x7f87fc077220 msgr2=0x7f87fc077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:03.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 -- 172.21.15.17:0/714845900 shutdown_connections 2024-02-14T15:19:03.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 -- 172.21.15.17:0/714845900 wait complete. 2024-02-14T15:19:03.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 Processor -- start 2024-02-14T15:19:03.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 -- start start 2024-02-14T15:19:03.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87fc100340 0x7f87fc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87fc105410 0x7f87fc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc1990a0 0x7f87fc1a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1a3d20 con 0x7f87fc1990a0 2024-02-14T15:19:03.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1a3e90 con 0x7f87fc100340 2024-02-14T15:19:03.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f880301c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1a4050 con 0x7f87fc105410 2024-02-14T15:19:03.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f88015b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc1990a0 0x7f87fc1a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f88015b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc1990a0 0x7f87fc1a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42846/0 (socket says 172.21.15.17:42846) 2024-02-14T15:19:03.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.169+0000 7f88015b9640 1 -- 172.21.15.17:0/23244351 learned_addr learned my addr 172.21.15.17:0/23244351 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:03.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f8800db8640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87fc100340 0x7f87fc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f87fbfff640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87fc105410 0x7f87fc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f88015b9640 1 -- 172.21.15.17:0/23244351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87fc105410 msgr2=0x7f87fc198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f88015b9640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87fc105410 0x7f87fc198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f88015b9640 1 -- 172.21.15.17:0/23244351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87fc100340 msgr2=0x7f87fc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f88015b9640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87fc100340 0x7f87fc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f88015b9640 1 -- 172.21.15.17:0/23244351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009900 con 0x7f87fc1990a0 2024-02-14T15:19:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f8800db8640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87fc100340 0x7f87fc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:19:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f88015b9640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc1990a0 0x7f87fc1a3650 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c970 tx=0x7f87e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f87f9ffb640 1 -- 172.21.15.17:0/23244351 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4007bb0 con 0x7f87fc1990a0 2024-02-14T15:19:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f87f9ffb640 1 -- 172.21.15.17:0/23244351 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87e4007d10 con 0x7f87fc1990a0 2024-02-14T15:19:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc1a4330 con 0x7f87fc1990a0 2024-02-14T15:19:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f87f9ffb640 1 -- 172.21.15.17:0/23244351 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4018770 con 0x7f87fc1990a0 2024-02-14T15:19:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.170+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc1a47f0 con 0x7f87fc1990a0 2024-02-14T15:19:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.171+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc04f7e0 con 0x7f87fc1990a0 2024-02-14T15:19:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.172+0000 7f87f9ffb640 1 -- 172.21.15.17:0/23244351 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f87e400f450 con 0x7f87fc1990a0 2024-02-14T15:19:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.173+0000 7f87f9ffb640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87e80a2220 0x7f87e80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.173+0000 7f8800db8640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87e80a2220 0x7f87e80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.173+0000 7f87f9ffb640 1 -- 172.21.15.17:0/23244351 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f87e40ce5d0 con 0x7f87fc1990a0 2024-02-14T15:19:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.173+0000 7f8800db8640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87e80a2220 0x7f87e80a46e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006010 tx=0x7f87f4005c50 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.177+0000 7f87f9ffb640 1 -- 172.21.15.17:0/23244351 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f87e4096e40 con 0x7f87fc1990a0 2024-02-14T15:19:03.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:02 smithi076 ceph-mon[37057]: pgmap v493: 129 pgs: 129 active+clean; 843 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 348 op/s 2024-02-14T15:19:03.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:02 smithi017 ceph-mon[32549]: pgmap v493: 129 pgs: 129 active+clean; 843 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 348 op/s 2024-02-14T15:19:03.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.382+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 --> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f87fc1086b0 con 0x7f87e80a2220 2024-02-14T15:19:03.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.392+0000 7f87f9ffb640 1 -- 172.21.15.17:0/23244351 <== mgr.15681 v2:172.21.15.17:6800/877978339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f87fc1086b0 con 0x7f87e80a2220 2024-02-14T15:19:03.394 INFO:teuthology.orchestra.run.smithi017.stdout:/ 1m 6m3h 2024-02-14T15:19:03.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87e80a2220 msgr2=0x7f87e80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87e80a2220 0x7f87e80a46e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006010 tx=0x7f87f4005c50 comp rx=0 tx=0).stop 2024-02-14T15:19:03.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc1990a0 msgr2=0x7f87fc1a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc1990a0 0x7f87fc1a3650 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c970 tx=0x7f87e400ce30 comp rx=0 tx=0).stop 2024-02-14T15:19:03.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 shutdown_connections 2024-02-14T15:19:03.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f87e80a2220 0x7f87e80a46e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87fc1990a0 0x7f87fc1a3650 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87fc105410 0x7f87fc198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 --2- 172.21.15.17:0/23244351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87fc100340 0x7f87fc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 >> 172.21.15.17:0/23244351 conn(0x7f87fc077220 msgr2=0x7f87fc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:03.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 shutdown_connections 2024-02-14T15:19:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.394+0000 7f880301c640 1 -- 172.21.15.17:0/23244351 wait complete. 2024-02-14T15:19:03.407 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-14T15:19:03.418 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-14T15:19:03.418 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-02-14T15:19:03.423 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-14T15:19:03.434 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:19:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 -- 172.21.15.17:0/4028684636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424106510 msgr2=0x7f442410e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 --2- 172.21.15.17:0/4028684636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424106510 0x7f442410e0e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f441c009cd0 tx=0x7f441c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 -- 172.21.15.17:0/4028684636 shutdown_connections 2024-02-14T15:19:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 --2- 172.21.15.17:0/4028684636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424106510 0x7f442410e0e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 --2- 172.21.15.17:0/4028684636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4424101df0 0x7f4424105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 --2- 172.21.15.17:0/4028684636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4424101440 0x7f4424101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 -- 172.21.15.17:0/4028684636 >> 172.21.15.17:0/4028684636 conn(0x7f44240fcce0 msgr2=0x7f44240ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 -- 172.21.15.17:0/4028684636 shutdown_connections 2024-02-14T15:19:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 -- 172.21.15.17:0/4028684636 wait complete. 2024-02-14T15:19:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 Processor -- start 2024-02-14T15:19:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.537+0000 7f442af16640 1 -- start start 2024-02-14T15:19:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f442af16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424101440 0x7f442419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f442af16640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4424106510 0x7f442419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f442af16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44241976d0 0x7f4424197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f442af16640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44241981a0 con 0x7f4424101440 2024-02-14T15:19:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f442af16640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4424198310 con 0x7f4424106510 2024-02-14T15:19:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f442af16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44241984d0 con 0x7f44241976d0 2024-02-14T15:19:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424101440 0x7f442419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424101440 0x7f442419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42854/0 (socket says 172.21.15.17:42854) 2024-02-14T15:19:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 -- 172.21.15.17:0/729492236 learned_addr learned my addr 172.21.15.17:0/729492236 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4423fff640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4424106510 0x7f442419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f44294b3640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44241976d0 0x7f4424197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 -- 172.21.15.17:0/729492236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44241976d0 msgr2=0x7f4424197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44241976d0 0x7f4424197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 -- 172.21.15.17:0/729492236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4424106510 msgr2=0x7f442419db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4424106510 0x7f442419db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 -- 172.21.15.17:0/729492236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c009900 con 0x7f4424101440 2024-02-14T15:19:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4428cb2640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424101440 0x7f442419d5c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f441800d950 tx=0x7f441800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4421ffb640 1 -- 172.21.15.17:0/729492236 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418009a30 con 0x7f4424101440 2024-02-14T15:19:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4421ffb640 1 -- 172.21.15.17:0/729492236 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4418009b90 con 0x7f4424101440 2024-02-14T15:19:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f4421ffb640 1 -- 172.21.15.17:0/729492236 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418004a00 con 0x7f4424101440 2024-02-14T15:19:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.538+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44241a4420 con 0x7f4424101440 2024-02-14T15:19:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.539+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44241a4900 con 0x7f4424101440 2024-02-14T15:19:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.539+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4424102910 con 0x7f4424101440 2024-02-14T15:19:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.540+0000 7f4421ffb640 1 -- 172.21.15.17:0/729492236 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4418009d00 con 0x7f4424101440 2024-02-14T15:19:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.541+0000 7f4421ffb640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f44140a2230 0x7f44140a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.541+0000 7f4421ffb640 1 -- 172.21.15.17:0/729492236 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f44180cddb0 con 0x7f4424101440 2024-02-14T15:19:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.541+0000 7f4423fff640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f44140a2230 0x7f44140a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.542+0000 7f4423fff640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f44140a2230 0x7f44140a46f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4424198ed0 tx=0x7f441000b390 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.546+0000 7f4421ffb640 1 -- 172.21.15.17:0/729492236 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4418096680 con 0x7f4424101440 2024-02-14T15:19:03.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.819+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44241097a0 con 0x7f4424101440 2024-02-14T15:19:03.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.824+0000 7f4421ffb640 1 -- 172.21.15.17:0/729492236 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+25220 (secure 0 0 0) 0x7f4418099d40 con 0x7f4424101440 2024-02-14T15:19:03.826 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:19:03.827 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":116,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:18:15.804629+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/420003070":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/2092370502":"2024-02-15T15:03:24.930590+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2755965102":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1369411962":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/3796717329":"2024-02-15T15:03:24.930590+0000","172.21.15.17:6801/3796717329":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/3775022683":"2024-02-15T15:03:24.930590+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:19:03.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f44140a2230 msgr2=0x7f44140a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f44140a2230 0x7f44140a46f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4424198ed0 tx=0x7f441000b390 comp rx=0 tx=0).stop 2024-02-14T15:19:03.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424101440 msgr2=0x7f442419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424101440 0x7f442419d5c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f441800d950 tx=0x7f441800de10 comp rx=0 tx=0).stop 2024-02-14T15:19:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 shutdown_connections 2024-02-14T15:19:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f44140a2230 0x7f44140a46f0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4424198ed0 tx=0x7f441000b390 comp rx=0 tx=0).stop 2024-02-14T15:19:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44241976d0 0x7f4424197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4424106510 0x7f442419db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 --2- 172.21.15.17:0/729492236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4424101440 0x7f442419d5c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.828+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 >> 172.21.15.17:0/729492236 conn(0x7f44240fcce0 msgr2=0x7f44240feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:03.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.829+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 shutdown_connections 2024-02-14T15:19:03.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.830+0000 7f442af16640 1 -- 172.21.15.17:0/729492236 wait complete. 2024-02-14T15:19:03.851 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 115, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 115, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:19:03.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:19:03.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 -- 172.21.15.17:0/2959292227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c101de0 msgr2=0x7fd52c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 --2- 172.21.15.17:0/2959292227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c101de0 0x7fd52c105e20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd514009930 tx=0x7fd51402bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:03.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 -- 172.21.15.17:0/2959292227 shutdown_connections 2024-02-14T15:19:03.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 --2- 172.21.15.17:0/2959292227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c101de0 0x7fd52c105e20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 --2- 172.21.15.17:0/2959292227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c106500 0x7fd52c10e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 --2- 172.21.15.17:0/2959292227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd52c101430 0x7fd52c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 -- 172.21.15.17:0/2959292227 >> 172.21.15.17:0/2959292227 conn(0x7fd52c0fccd0 msgr2=0x7fd52c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:03.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 -- 172.21.15.17:0/2959292227 shutdown_connections 2024-02-14T15:19:03.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 -- 172.21.15.17:0/2959292227 wait complete. 2024-02-14T15:19:03.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 Processor -- start 2024-02-14T15:19:03.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.972+0000 7fd531c5c640 1 -- start start 2024-02-14T15:19:03.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd531c5c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd52c101430 0x7fd52c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd531c5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c106500 0x7fd52c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd531c5c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c1977e0 0x7fd52c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd531c5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c198290 con 0x7fd52c106500 2024-02-14T15:19:03.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd531c5c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c198400 con 0x7fd52c1977e0 2024-02-14T15:19:03.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd531c5c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1985c0 con 0x7fd52c101430 2024-02-14T15:19:03.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c106500 0x7fd52c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c106500 0x7fd52c19dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42880/0 (socket says 172.21.15.17:42880) 2024-02-14T15:19:03.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 -- 172.21.15.17:0/1302985346 learned_addr learned my addr 172.21.15.17:0/1302985346 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:03.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52bfff640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c1977e0 0x7fd52c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52b7fe640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd52c101430 0x7fd52c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 -- 172.21.15.17:0/1302985346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd52c101430 msgr2=0x7fd52c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd52c101430 0x7fd52c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 -- 172.21.15.17:0/1302985346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c1977e0 msgr2=0x7fd52c197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:03.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c1977e0 0x7fd52c197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:03.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 -- 172.21.15.17:0/1302985346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd514009560 con 0x7fd52c106500 2024-02-14T15:19:03.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd52affd640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c106500 0x7fd52c19dbd0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd514009230 tx=0x7fd51402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:03.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd528ff9640 1 -- 172.21.15.17:0/1302985346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514004350 con 0x7fd52c106500 2024-02-14T15:19:03.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd528ff9640 1 -- 172.21.15.17:0/1302985346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5140044b0 con 0x7fd52c106500 2024-02-14T15:19:03.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.973+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c1a4500 con 0x7fd52c106500 2024-02-14T15:19:03.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.974+0000 7fd528ff9640 1 -- 172.21.15.17:0/1302985346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514003da0 con 0x7fd52c106500 2024-02-14T15:19:03.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.974+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c1a4a60 con 0x7fd52c106500 2024-02-14T15:19:03.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.974+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c068500 con 0x7fd52c106500 2024-02-14T15:19:03.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.975+0000 7fd528ff9640 1 -- 172.21.15.17:0/1302985346 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd514004df0 con 0x7fd52c106500 2024-02-14T15:19:03.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.976+0000 7fd528ff9640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd5180a22b0 0x7fd5180a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:03.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.976+0000 7fd52b7fe640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd5180a22b0 0x7fd5180a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:03.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.976+0000 7fd528ff9640 1 -- 172.21.15.17:0/1302985346 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd5140f0390 con 0x7fd52c106500 2024-02-14T15:19:03.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.977+0000 7fd52b7fe640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd5180a22b0 0x7fd5180a4770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009bd0 tx=0x7fd51c005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:03.980+0000 7fd528ff9640 1 -- 172.21.15.17:0/1302985346 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5140b8d70 con 0x7fd52c106500 2024-02-14T15:19:04.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:03 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/729492236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:04.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.162+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd52c109730 con 0x7fd52c106500 2024-02-14T15:19:04.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.162+0000 7fd528ff9640 1 -- 172.21.15.17:0/1302985346 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5140bc430 con 0x7fd52c106500 2024-02-14T15:19:04.164 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:19:04.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd5180a22b0 msgr2=0x7fd5180a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd5180a22b0 0x7fd5180a4770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009bd0 tx=0x7fd51c005f90 comp rx=0 tx=0).stop 2024-02-14T15:19:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c106500 msgr2=0x7fd52c19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c106500 0x7fd52c19dbd0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd514009230 tx=0x7fd51402c680 comp rx=0 tx=0).stop 2024-02-14T15:19:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 shutdown_connections 2024-02-14T15:19:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fd5180a22b0 0x7fd5180a4770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd52c106500 0x7fd52c19dbd0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd52c1977e0 0x7fd52c197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 --2- 172.21.15.17:0/1302985346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd52c101430 0x7fd52c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 >> 172.21.15.17:0/1302985346 conn(0x7fd52c0fccd0 msgr2=0x7fd52c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 shutdown_connections 2024-02-14T15:19:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.164+0000 7fd531c5c640 1 -- 172.21.15.17:0/1302985346 wait complete. 2024-02-14T15:19:04.176 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:03 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/729492236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:04.177 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:19:04.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:03 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/729492236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:04.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 -- 172.21.15.17:0/103328267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4106500 msgr2=0x7f43b410e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 --2- 172.21.15.17:0/103328267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4106500 0x7f43b410e0d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009cd0 tx=0x7f43ac02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 -- 172.21.15.17:0/103328267 shutdown_connections 2024-02-14T15:19:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 --2- 172.21.15.17:0/103328267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4106500 0x7f43b410e0d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 --2- 172.21.15.17:0/103328267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43b4101de0 0x7f43b4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 --2- 172.21.15.17:0/103328267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43b4101430 0x7f43b4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 -- 172.21.15.17:0/103328267 >> 172.21.15.17:0/103328267 conn(0x7f43b40fccd0 msgr2=0x7f43b40ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 -- 172.21.15.17:0/103328267 shutdown_connections 2024-02-14T15:19:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 -- 172.21.15.17:0/103328267 wait complete. 2024-02-14T15:19:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 Processor -- start 2024-02-14T15:19:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.282+0000 7f43bbb93640 1 -- start start 2024-02-14T15:19:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4101430 0x7f43b419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43b4106500 0x7f43b419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43b4197670 0x7f43b4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4198140 con 0x7f43b4101430 2024-02-14T15:19:04.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41982b0 con 0x7f43b4197670 2024-02-14T15:19:04.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4198470 con 0x7f43b4106500 2024-02-14T15:19:04.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4101430 0x7f43b419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4101430 0x7f43b419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42892/0 (socket says 172.21.15.17:42892) 2024-02-14T15:19:04.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 -- 172.21.15.17:0/2077000618 learned_addr learned my addr 172.21.15.17:0/2077000618 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:04.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43ba130640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43b4197670 0x7f43b4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b912e640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43b4106500 0x7f43b419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 -- 172.21.15.17:0/2077000618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43b4106500 msgr2=0x7f43b419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43b4106500 0x7f43b419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 -- 172.21.15.17:0/2077000618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43b4197670 msgr2=0x7f43b4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43b4197670 0x7f43b4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 -- 172.21.15.17:0/2077000618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac009900 con 0x7f43b4101430 2024-02-14T15:19:04.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43b992f640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4101430 0x7f43b419d5b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d950 tx=0x7f43b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:04.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43aaffd640 1 -- 172.21.15.17:0/2077000618 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0009a30 con 0x7f43b4101430 2024-02-14T15:19:04.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43aaffd640 1 -- 172.21.15.17:0/2077000618 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b0009b90 con 0x7f43b4101430 2024-02-14T15:19:04.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b41a4410 con 0x7f43b4101430 2024-02-14T15:19:04.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43aaffd640 1 -- 172.21.15.17:0/2077000618 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0004a00 con 0x7f43b4101430 2024-02-14T15:19:04.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.283+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b41a48f0 con 0x7f43b4101430 2024-02-14T15:19:04.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.284+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b4102900 con 0x7f43b4101430 2024-02-14T15:19:04.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.285+0000 7f43aaffd640 1 -- 172.21.15.17:0/2077000618 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f43b0009d00 con 0x7f43b4101430 2024-02-14T15:19:04.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.286+0000 7f43aaffd640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43a00a2230 0x7f43a00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.286+0000 7f43b912e640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43a00a2230 0x7f43a00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.286+0000 7f43aaffd640 1 -- 172.21.15.17:0/2077000618 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f43b00ce480 con 0x7f43b4101430 2024-02-14T15:19:04.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.286+0000 7f43b912e640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43a00a2230 0x7f43a00a46f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f43b4198ec0 tx=0x7f43a4005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:04.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.290+0000 7f43aaffd640 1 -- 172.21.15.17:0/2077000618 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43b0096d50 con 0x7f43b4101430 2024-02-14T15:19:04.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.495+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43b4109760 con 0x7f43b4101430 2024-02-14T15:19:04.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.496+0000 7f43aaffd640 1 -- 172.21.15.17:0/2077000618 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43b00170a0 con 0x7f43b4101430 2024-02-14T15:19:04.498 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:19:04.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43a00a2230 msgr2=0x7f43a00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43a00a2230 0x7f43a00a46f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f43b4198ec0 tx=0x7f43a4005f90 comp rx=0 tx=0).stop 2024-02-14T15:19:04.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4101430 msgr2=0x7f43b419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4101430 0x7f43b419d5b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d950 tx=0x7f43b000de10 comp rx=0 tx=0).stop 2024-02-14T15:19:04.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 shutdown_connections 2024-02-14T15:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f43a00a2230 0x7f43a00a46f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43b4197670 0x7f43b4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43b4106500 0x7f43b419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 --2- 172.21.15.17:0/2077000618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b4101430 0x7f43b419d5b0 secure :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d950 tx=0x7f43b000de10 comp rx=0 tx=0).stop 2024-02-14T15:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 >> 172.21.15.17:0/2077000618 conn(0x7f43b40fccd0 msgr2=0x7f43b40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:04.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 shutdown_connections 2024-02-14T15:19:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.499+0000 7f43bbb93640 1 -- 172.21.15.17:0/2077000618 wait complete. 2024-02-14T15:19:04.516 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:19:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 -- 172.21.15.17:0/3869450792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90106500 msgr2=0x7f5b9010e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 --2- 172.21.15.17:0/3869450792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90106500 0x7f5b9010e0d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009cd0 tx=0x7f5b8802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 -- 172.21.15.17:0/3869450792 shutdown_connections 2024-02-14T15:19:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 --2- 172.21.15.17:0/3869450792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90106500 0x7f5b9010e0d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 --2- 172.21.15.17:0/3869450792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b90101de0 0x7f5b90105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 --2- 172.21.15.17:0/3869450792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b90101430 0x7f5b90101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 -- 172.21.15.17:0/3869450792 >> 172.21.15.17:0/3869450792 conn(0x7f5b900fccd0 msgr2=0x7f5b900ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 -- 172.21.15.17:0/3869450792 shutdown_connections 2024-02-14T15:19:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.673+0000 7f5b967bc640 1 -- 172.21.15.17:0/3869450792 wait complete. 2024-02-14T15:19:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 Processor -- start 2024-02-14T15:19:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 -- start start 2024-02-14T15:19:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b90101430 0x7f5b9019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b90106500 0x7f5b9019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90197690 0x7f5b90197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90198160 con 0x7f5b90197690 2024-02-14T15:19:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b901982d0 con 0x7f5b90106500 2024-02-14T15:19:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b967bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90198490 con 0x7f5b90101430 2024-02-14T15:19:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b8f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b90106500 0x7f5b9019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b94d59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90197690 0x7f5b90197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b8f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b90106500 0x7f5b9019db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.17:59284/0 (socket says 172.21.15.17:59284) 2024-02-14T15:19:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b8f7fe640 1 -- 172.21.15.17:0/2925181803 learned_addr learned my addr 172.21.15.17:0/2925181803 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b8ffff640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b90101430 0x7f5b9019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b94d59640 1 -- 172.21.15.17:0/2925181803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b90101430 msgr2=0x7f5b9019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b94d59640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b90101430 0x7f5b9019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b94d59640 1 -- 172.21.15.17:0/2925181803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b90106500 msgr2=0x7f5b9019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b94d59640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b90106500 0x7f5b9019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.674+0000 7f5b94d59640 1 -- 172.21.15.17:0/2925181803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b88009900 con 0x7f5b90197690 2024-02-14T15:19:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.675+0000 7f5b94d59640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90197690 0x7f5b90197b10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5b8802c630 tx=0x7f5b88034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.675+0000 7f5b8d7fa640 1 -- 172.21.15.17:0/2925181803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88032ac0 con 0x7f5b90197690 2024-02-14T15:19:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.675+0000 7f5b8d7fa640 1 -- 172.21.15.17:0/2925181803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b88032eb0 con 0x7f5b90197690 2024-02-14T15:19:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.675+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b901a43d0 con 0x7f5b90197690 2024-02-14T15:19:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.675+0000 7f5b8d7fa640 1 -- 172.21.15.17:0/2925181803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88040450 con 0x7f5b90197690 2024-02-14T15:19:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.675+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b901a4790 con 0x7f5b90197690 2024-02-14T15:19:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.676+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b90102900 con 0x7f5b90197690 2024-02-14T15:19:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.677+0000 7f5b8d7fa640 1 -- 172.21.15.17:0/2925181803 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5b8800a040 con 0x7f5b90197690 2024-02-14T15:19:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.678+0000 7f5b8d7fa640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5b7c0a2380 0x7f5b7c0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.678+0000 7f5b8ffff640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5b7c0a2380 0x7f5b7c0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.678+0000 7f5b8d7fa640 1 -- 172.21.15.17:0/2925181803 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f5b880f1440 con 0x7f5b90197690 2024-02-14T15:19:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.679+0000 7f5b8ffff640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5b7c0a2380 0x7f5b7c0a4840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b80005fd0 tx=0x7f5b80005850 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.682+0000 7f5b8d7fa640 1 -- 172.21.15.17:0/2925181803 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b880ba040 con 0x7f5b90197690 2024-02-14T15:19:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.868+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b90101de0 con 0x7f5b90197690 2024-02-14T15:19:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.868+0000 7f5b8d7fa640 1 -- 172.21.15.17:0/2925181803 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b880440a0 con 0x7f5b90197690 2024-02-14T15:19:04.870 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:19:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5b7c0a2380 msgr2=0x7f5b7c0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5b7c0a2380 0x7f5b7c0a4840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b80005fd0 tx=0x7f5b80005850 comp rx=0 tx=0).stop 2024-02-14T15:19:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90197690 msgr2=0x7f5b90197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90197690 0x7f5b90197b10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5b8802c630 tx=0x7f5b88034d40 comp rx=0 tx=0).stop 2024-02-14T15:19:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 shutdown_connections 2024-02-14T15:19:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f5b7c0a2380 0x7f5b7c0a4840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b90197690 0x7f5b90197b10 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b90106500 0x7f5b9019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 --2- 172.21.15.17:0/2925181803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b90101430 0x7f5b9019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 >> 172.21.15.17:0/2925181803 conn(0x7f5b900fccd0 msgr2=0x7f5b900ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 shutdown_connections 2024-02-14T15:19:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.870+0000 7f5b967bc640 1 -- 172.21.15.17:0/2925181803 wait complete. 2024-02-14T15:19:04.883 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:19:04.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 -- 172.21.15.17:0/1773052770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0ff990 msgr2=0x7f9dec0ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:04.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/1773052770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0ff990 0x7f9dec0ffd70 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009930 tx=0x7f9ddc02bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 -- 172.21.15.17:0/1773052770 shutdown_connections 2024-02-14T15:19:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/1773052770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dec106550 0x7f9dec10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/1773052770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dec100340 0x7f9dec105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/1773052770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec0ff990 0x7f9dec0ffd70 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 -- 172.21.15.17:0/1773052770 >> 172.21.15.17:0/1773052770 conn(0x7f9dec076630 msgr2=0x7f9dec0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 -- 172.21.15.17:0/1773052770 shutdown_connections 2024-02-14T15:19:04.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 -- 172.21.15.17:0/1773052770 wait complete. 2024-02-14T15:19:04.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.989+0000 7f9df1b0c640 1 Processor -- start 2024-02-14T15:19:04.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9df1b0c640 1 -- start start 2024-02-14T15:19:04.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9df1b0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dec0ff990 0x7f9dec19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:04.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9df1b0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dec106550 0x7f9dec19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9df1b0c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec197730 0x7f9dec197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9df1b0c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec198200 con 0x7f9dec197730 2024-02-14T15:19:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9df1b0c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec198370 con 0x7f9dec106550 2024-02-14T15:19:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9df1b0c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec198530 con 0x7f9dec0ff990 2024-02-14T15:19:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec197730 0x7f9dec197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec197730 0x7f9dec197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42940/0 (socket says 172.21.15.17:42940) 2024-02-14T15:19:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 -- 172.21.15.17:0/500337067 learned_addr learned my addr 172.21.15.17:0/500337067 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9deb7fe640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dec0ff990 0x7f9dec19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9deaffd640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dec106550 0x7f9dec19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 -- 172.21.15.17:0/500337067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dec0ff990 msgr2=0x7f9dec19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dec0ff990 0x7f9dec19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 -- 172.21.15.17:0/500337067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dec106550 msgr2=0x7f9dec19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:05.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dec106550 0x7f9dec19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 -- 172.21.15.17:0/500337067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc009560 con 0x7f9dec197730 2024-02-14T15:19:05.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.990+0000 7f9debfff640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec197730 0x7f9dec197bb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9de400eb10 tx=0x7f9de400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:05.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.991+0000 7f9de8ff9640 1 -- 172.21.15.17:0/500337067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4004b20 con 0x7f9dec197730 2024-02-14T15:19:05.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.991+0000 7f9de8ff9640 1 -- 172.21.15.17:0/500337067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9de4004c80 con 0x7f9dec197730 2024-02-14T15:19:05.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.991+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec1a44d0 con 0x7f9dec197730 2024-02-14T15:19:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.991+0000 7f9de8ff9640 1 -- 172.21.15.17:0/500337067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4010650 con 0x7f9dec197730 2024-02-14T15:19:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.991+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec1a4960 con 0x7f9dec197730 2024-02-14T15:19:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.992+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec102a40 con 0x7f9dec197730 2024-02-14T15:19:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.993+0000 7f9de8ff9640 1 -- 172.21.15.17:0/500337067 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9de4005020 con 0x7f9dec197730 2024-02-14T15:19:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.994+0000 7f9de8ff9640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9dd40a2230 0x7f9dd40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.994+0000 7f9de8ff9640 1 -- 172.21.15.17:0/500337067 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f9de4014070 con 0x7f9dec197730 2024-02-14T15:19:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.994+0000 7f9deb7fe640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9dd40a2230 0x7f9dd40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.994+0000 7f9deb7fe640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9dd40a2230 0x7f9dd40a46f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005ab0 tx=0x7f9ddc005fd0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:04.998+0000 7f9de8ff9640 1 -- 172.21.15.17:0/500337067 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9de4097240 con 0x7f9dec197730 2024-02-14T15:19:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:04 smithi050 ceph-mon[36512]: from='client.17583 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:19:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:04 smithi050 ceph-mon[36512]: pgmap v494: 129 pgs: 129 active+clean; 845 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 288 op/s 2024-02-14T15:19:05.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:04 smithi050 ceph-mon[36512]: from='client.17589 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:19:05.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1302985346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:19:05.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:04 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2077000618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:19:05.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.178+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9dec1097f0 con 0x7f9dec197730 2024-02-14T15:19:05.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.178+0000 7f9de8ff9640 1 -- 172.21.15.17:0/500337067 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v116) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9de409a900 con 0x7f9dec197730 2024-02-14T15:19:05.180 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:19:05.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9dd40a2230 msgr2=0x7f9dd40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:05.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9dd40a2230 0x7f9dd40a46f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005ab0 tx=0x7f9ddc005fd0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec197730 msgr2=0x7f9dec197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:05.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec197730 0x7f9dec197bb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9de400eb10 tx=0x7f9de400c2d0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 shutdown_connections 2024-02-14T15:19:05.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9dd40a2230 0x7f9dd40a46f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dec197730 0x7f9dec197bb0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9dec106550 0x7f9dec19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 --2- 172.21.15.17:0/500337067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dec0ff990 0x7f9dec19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 >> 172.21.15.17:0/500337067 conn(0x7f9dec076630 msgr2=0x7f9dec104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:05.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 shutdown_connections 2024-02-14T15:19:05.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.180+0000 7f9df1b0c640 1 -- 172.21.15.17:0/500337067 wait complete. 2024-02-14T15:19:05.192 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:04 smithi017 ceph-mon[32549]: from='client.17583 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:19:05.192 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:04 smithi017 ceph-mon[32549]: pgmap v494: 129 pgs: 129 active+clean; 845 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 288 op/s 2024-02-14T15:19:05.192 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:04 smithi017 ceph-mon[32549]: from='client.17589 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:19:05.192 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1302985346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:19:05.193 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:04 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2077000618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:19:05.194 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-14T15:19:05.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:04 smithi076 ceph-mon[37057]: from='client.17583 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:19:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:04 smithi076 ceph-mon[37057]: pgmap v494: 129 pgs: 129 active+clean; 845 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 288 op/s 2024-02-14T15:19:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:04 smithi076 ceph-mon[37057]: from='client.17589 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T15:19:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1302985346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:19:05.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:04 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2077000618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:19:05.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 -- 172.21.15.17:0/832950844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c105420 msgr2=0x7f122c10cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:05.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 --2- 172.21.15.17:0/832950844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c105420 0x7f122c10cff0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f121c009cd0 tx=0x7f121c02c0f0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 -- 172.21.15.17:0/832950844 shutdown_connections 2024-02-14T15:19:05.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 --2- 172.21.15.17:0/832950844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c105420 0x7f122c10cff0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 --2- 172.21.15.17:0/832950844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c100d00 0x7f122c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 --2- 172.21.15.17:0/832950844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f122c100350 0x7f122c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 -- 172.21.15.17:0/832950844 >> 172.21.15.17:0/832950844 conn(0x7f122c0772b0 msgr2=0x7f122c0776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:05.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 -- 172.21.15.17:0/832950844 shutdown_connections 2024-02-14T15:19:05.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.295+0000 7f1230823640 1 -- 172.21.15.17:0/832950844 wait complete. 2024-02-14T15:19:05.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 Processor -- start 2024-02-14T15:19:05.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 -- start start 2024-02-14T15:19:05.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c100350 0x7f122c101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:05.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f122c105420 0x7f122c102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:05.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c102860 0x7f122c1a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:05.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a6090 con 0x7f122c100350 2024-02-14T15:19:05.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a6200 con 0x7f122c105420 2024-02-14T15:19:05.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1230823640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a63c0 con 0x7f122c102860 2024-02-14T15:19:05.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f122a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c100350 0x7f122c101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f122a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c100350 0x7f122c101de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42954/0 (socket says 172.21.15.17:42954) 2024-02-14T15:19:05.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f122a59c640 1 -- 172.21.15.17:0/113380621 learned_addr learned my addr 172.21.15.17:0/113380621 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:05.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f122ad9d640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c102860 0x7f122c1a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f1229d9b640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f122c105420 0x7f122c102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.296+0000 7f122a59c640 1 -- 172.21.15.17:0/113380621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c102860 msgr2=0x7f122c1a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:05.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f122a59c640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c102860 0x7f122c1a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f122a59c640 1 -- 172.21.15.17:0/113380621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f122c105420 msgr2=0x7f122c102320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:05.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f122a59c640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f122c105420 0x7f122c102320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:05.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f122a59c640 1 -- 172.21.15.17:0/113380621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c009900 con 0x7f122c100350 2024-02-14T15:19:05.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f122a59c640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c100350 0x7f122c101de0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f122400e9f0 tx=0x7f122400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:05.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f12237fe640 1 -- 172.21.15.17:0/113380621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224009700 con 0x7f122c100350 2024-02-14T15:19:05.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f12237fe640 1 -- 172.21.15.17:0/113380621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f122400cd80 con 0x7f122c100350 2024-02-14T15:19:05.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c1a66a0 con 0x7f122c100350 2024-02-14T15:19:05.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f12237fe640 1 -- 172.21.15.17:0/113380621 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224018920 con 0x7f122c100350 2024-02-14T15:19:05.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.297+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c1a6c00 con 0x7f122c100350 2024-02-14T15:19:05.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.298+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c068500 con 0x7f122c100350 2024-02-14T15:19:05.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.299+0000 7f12237fe640 1 -- 172.21.15.17:0/113380621 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1224018a80 con 0x7f122c100350 2024-02-14T15:19:05.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.299+0000 7f12237fe640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f12180a22e0 0x7f12180a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:05.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.300+0000 7f1229d9b640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f12180a22e0 0x7f12180a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:05.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.300+0000 7f12237fe640 1 -- 172.21.15.17:0/113380621 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f1224014070 con 0x7f122c100350 2024-02-14T15:19:05.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.300+0000 7f1229d9b640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f12180a22e0 0x7f12180a47a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f122c103870 tx=0x7f1214005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:05.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.304+0000 7f12237fe640 1 -- 172.21.15.17:0/113380621 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1224052d20 con 0x7f122c100350 2024-02-14T15:19:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.590+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f122c100d00 con 0x7f122c100350 2024-02-14T15:19:05.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.604+0000 7f12237fe640 1 -- 172.21.15.17:0/113380621 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) v1 ==== 81+0+736940 (secure 0 0 0) 0x7f12240970b0 con 0x7f122c100350 2024-02-14T15:19:05.622 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:19:05.622 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:19:05.622 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 27, 2024-02-14T15:19:05.622 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.623 INFO:teuthology.orchestra.run.smithi017.stdout: "active_gid": 15681, 2024-02-14T15:19:05.623 INFO:teuthology.orchestra.run.smithi017.stdout: "active_name": "x", 2024-02-14T15:19:05.623 INFO:teuthology.orchestra.run.smithi017.stdout: "active_addrs": { 2024-02-14T15:19:05.623 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-02-14T15:19:05.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.624 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-02-14T15:19:05.624 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2024-02-14T15:19:05.624 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 877978339 2024-02-14T15:19:05.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.624 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-02-14T15:19:05.625 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2024-02-14T15:19:05.625 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 877978339 2024-02-14T15:19:05.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.625 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:05.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.625 INFO:teuthology.orchestra.run.smithi017.stdout: "active_addr": "172.21.15.17:6801/877978339", 2024-02-14T15:19:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "active_change": "2024-02-14T15:03:24.930941+0000", 2024-02-14T15:19:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "active_mgr_features": 4540701272860131327, 2024-02-14T15:19:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "available": true, 2024-02-14T15:19:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "standbys": [ 2024-02-14T15:19:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "gid": 25172, 2024-02-14T15:19:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "y", 2024-02-14T15:19:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_features": 4540701272860131327, 2024-02-14T15:19:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "available_modules": [ 2024-02-14T15:19:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "alerts", 2024-02-14T15:19:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How frequently to reexamine health status", 2024-02-14T15:19:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_destination": { 2024-02-14T15:19:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_destination", 2024-02-14T15:19:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email address to send alerts to", 2024-02-14T15:19:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_from_name": { 2024-02-14T15:19:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_from_name", 2024-02-14T15:19:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "Ceph", 2024-02-14T15:19:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email From: name", 2024-02-14T15:19:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_host": { 2024-02-14T15:19:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_host", 2024-02-14T15:19:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP server", 2024-02-14T15:19:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_password": { 2024-02-14T15:19:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_password", 2024-02-14T15:19:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.648 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.648 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.648 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.648 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.648 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.648 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Password to authenticate with", 2024-02-14T15:19:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_port": { 2024-02-14T15:19:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_port", 2024-02-14T15:19:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "465", 2024-02-14T15:19:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP port", 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_sender": { 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_sender", 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.652 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.652 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.652 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.652 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.652 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.652 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP envelope sender", 2024-02-14T15:19:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_ssl": { 2024-02-14T15:19:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_ssl", 2024-02-14T15:19:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-14T15:19:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_user": { 2024-02-14T15:19:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_user", 2024-02-14T15:19:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "User to authenticate as", 2024-02-14T15:19:05.657 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.657 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.657 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "balancer", 2024-02-14T15:19:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "active": { 2024-02-14T15:19:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "active", 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically balance PGs across cluster", 2024-02-14T15:19:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_time": { 2024-02-14T15:19:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_time", 2024-02-14T15:19:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0000", 2024-02-14T15:19:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "beginning time of day to automatically balance", 2024-02-14T15:19:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-14T15:19:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_weekday": { 2024-02-14T15:19:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_weekday", 2024-02-14T15:19:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-02-14T15:19:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-02-14T15:19:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-14T15:19:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-14T15:19:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_max_iterations": { 2024-02-14T15:19:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_max_iterations", 2024-02-14T15:19:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "25", 2024-02-14T15:19:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "250", 2024-02-14T15:19:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_metrics": { 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_metrics", 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "pgs,objects,bytes", 2024-02-14T15:19:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-14T15:19:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-14T15:19:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_step": { 2024-02-14T15:19:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_step", 2024-02-14T15:19:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.669 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.669 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.5", 2024-02-14T15:19:05.669 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0.001", 2024-02-14T15:19:05.669 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "0.999", 2024-02-14T15:19:05.669 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.669 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "aggressiveness of optimization", 2024-02-14T15:19:05.669 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: "end_time": { 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_time", 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2359", 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ending time of day to automatically balance", 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.671 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.672 INFO:teuthology.orchestra.run.smithi017.stdout: "end_weekday": { 2024-02-14T15:19:05.672 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_weekday", 2024-02-14T15:19:05.672 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:05.672 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.672 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.673 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:05.673 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-02-14T15:19:05.673 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-02-14T15:19:05.673 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.673 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-14T15:19:05.673 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-14T15:19:05.673 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.674 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.675 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.676 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.676 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.676 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.676 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.676 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.676 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.677 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.677 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.677 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.677 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.677 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.677 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.677 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.678 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.678 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.678 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.678 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.678 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.678 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.679 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.679 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.679 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.679 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.679 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.679 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.679 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.680 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.680 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.680 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.680 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.680 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.680 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.680 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.681 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.682 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.682 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.682 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.682 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.682 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.682 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.682 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: "min_score": { 2024-02-14T15:19:05.683 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_score", 2024-02-14T15:19:05.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:05.684 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.684 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.684 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:05.684 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.684 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.684 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-02-14T15:19:05.685 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "upmap", 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-compat", 2024-02-14T15:19:05.686 INFO:teuthology.orchestra.run.smithi017.stdout: "none", 2024-02-14T15:19:05.687 INFO:teuthology.orchestra.run.smithi017.stdout: "read", 2024-02-14T15:19:05.687 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap", 2024-02-14T15:19:05.687 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap-read" 2024-02-14T15:19:05.687 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.687 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Balancer mode", 2024-02-14T15:19:05.687 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.687 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_ids": { 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_ids", 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.688 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.690 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.691 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.691 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-14T15:19:05.691 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.691 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.691 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.691 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_deviation": { 2024-02-14T15:19:05.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_deviation", 2024-02-14T15:19:05.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.692 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.692 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.692 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:05.692 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:05.692 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_optimizations": { 2024-02-14T15:19:05.693 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_optimizations", 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.694 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm", 2024-02-14T15:19:05.695 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_down_multiplier": { 2024-02-14T15:19:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_down_multiplier", 2024-02-14T15:19:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.696 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.697 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_refresh_rate": { 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_refresh_rate", 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "20", 2024-02-14T15:19:05.698 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.699 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_starting_port": { 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_starting_port", 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4721", 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.700 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_ptrace": { 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_ptrace", 2024-02-14T15:19:05.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-14T15:19:05.702 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_interval": { 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_interval", 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.703 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.704 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_memory_target_ratio": { 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_memory_target_ratio", 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.7", 2024-02-14T15:19:05.705 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.706 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_log_destination": { 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm_log_destination", 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.707 INFO:teuthology.orchestra.run.smithi017.stdout: "file", 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "file,syslog", 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "syslog" 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "cgroups_split": { 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cgroups_split", 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.709 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "config_checks_enabled": { 2024-02-14T15:19:05.710 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_checks_enabled", 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-14T15:19:05.711 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "config_dashboard": { 2024-02-14T15:19:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_dashboard", 2024-02-14T15:19:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.712 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.713 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_alertmanager": { 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_alertmanager", 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-14T15:19:05.714 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_base": { 2024-02-14T15:19:05.715 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_base", 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.716 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Container image name, without the tag", 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_elasticsearch": { 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_elasticsearch", 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.717 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "elasticsearch container image", 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.718 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_grafana": { 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_grafana", 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-14T15:19:05.719 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.720 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_haproxy": { 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_haproxy", 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.721 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "HAproxy container image", 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_agent": { 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_agent", 2024-02-14T15:19:05.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger agent container image", 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.723 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_collector": { 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_collector", 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger collector container image", 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_query": { 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_query", 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.726 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger query container image", 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_keepalived": { 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_keepalived", 2024-02-14T15:19:05.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Keepalived container image", 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.728 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_loki": { 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_loki", 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-14T15:19:05.729 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Loki container image", 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.730 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_node_exporter": { 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_node_exporter", 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.731 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_nvmeof": { 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_nvmeof", 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.732 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Nvme-of container image", 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.733 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_prometheus": { 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_prometheus", 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-14T15:19:05.734 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_promtail": { 2024-02-14T15:19:05.735 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_promtail", 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.736 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Promtail container image", 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_snmp_gateway": { 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_snmp_gateway", 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.737 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SNMP Gateway container image", 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "container_init": { 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_init", 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.739 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Run podman/docker with `--init`", 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: "daemon_cache_timeout": { 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "daemon_cache_timeout", 2024-02-14T15:19:05.740 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.741 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: "default_cephadm_command_timeout": { 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_cephadm_command_timeout", 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.742 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "900", 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.743 INFO:teuthology.orchestra.run.smithi017.stdout: "default_registry": { 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_registry", 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io", 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.744 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "device_cache_timeout": { 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_cache_timeout", 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.745 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1800", 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache device inventory", 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "device_enhanced_scan": { 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_enhanced_scan", 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.747 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: "facts_cache_timeout": { 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "facts_cache_timeout", 2024-02-14T15:19:05.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache host facts data", 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.749 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "host_check_interval": { 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "host_check_interval", 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:05.750 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to perform a host check", 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "hw_monitoring": { 2024-02-14T15:19:05.751 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hw_monitoring", 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-14T15:19:05.752 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "inventory_list_all": { 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "inventory_list_all", 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.753 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.755 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.756 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "log_refresh_metadata": { 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_refresh_metadata", 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.757 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.758 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.759 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.760 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.761 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.762 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.763 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.764 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "*", 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.766 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "max_count_per_host": { 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_count_per_host", 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.767 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of daemons per service per host", 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd_draining_count": { 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_osd_draining_count", 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:05.769 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "migration_current": { 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "migration_current", 2024-02-14T15:19:05.770 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "internal - do not modify", 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.771 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.772 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "root", 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm-package", 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "root" 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "mode for remote execution of cephadm", 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.773 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "oob_default_addr": { 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "oob_default_addr", 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "169.254.1.1", 2024-02-14T15:19:05.774 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "prometheus_alerts_path": { 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus_alerts_path", 2024-02-14T15:19:05.775 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-14T15:19:05.776 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_insecure": { 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_insecure", 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.777 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.778 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_password": { 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_password", 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.779 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_url": { 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_url", 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.780 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.781 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_username": { 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_username", 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.782 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "secure_monitoring_stack": { 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "secure_monitoring_stack", 2024-02-14T15:19:05.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.784 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "service_discovery_port": { 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "service_discovery_port", 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.785 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8765", 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "cephadm service discovery port", 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.786 INFO:teuthology.orchestra.run.smithi017.stdout: "ssh_config_file": { 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssh_config_file", 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-14T15:19:05.787 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: "use_agent": { 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_agent", 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "use_repo_digest": { 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_repo_digest", 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.790 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_failed_host_check": { 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_failed_host_check", 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.791 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if the host check fails", 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.792 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_daemons": { 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_daemons", 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.793 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_hosts": { 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_hosts", 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.794 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.795 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crash", 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.796 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.797 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.800 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.801 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.802 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.803 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "retain_interval": { 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retain_interval", 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.804 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "31536000", 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_recent_interval": { 2024-02-14T15:19:05.805 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_recent_interval", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1209600", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.806 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dashboard", 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.807 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.808 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_HOST": { 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.809 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.810 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.811 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_ENABLED": { 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_ENABLED", 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.812 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.814 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "ENABLE_BROWSABLE_API": { 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.815 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.817 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.818 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.819 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.820 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.821 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.822 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.823 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-14T15:19:05.824 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.825 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.826 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-14T15:19:05.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.828 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_PASSWORD": { 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.829 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-14T15:19:05.830 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.831 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_URL": { 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_URL", 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.832 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_USERNAME": { 2024-02-14T15:19:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-14T15:19:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-14T15:19:05.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.840 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_HOST": { 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.841 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.842 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.843 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.844 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.846 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.847 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.849 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.850 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.851 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.852 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.853 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_ENABLED": { 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.855 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.856 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8", 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "45", 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ACCESS_KEY": { 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SECRET_KEY": { 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-14T15:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SSL_VERIFY": { 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-14T15:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-14T15:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "cross_origin_url": { 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cross_origin_url", 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "crt_file": { 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crt_file", 2024-02-14T15:19:05.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.875 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "debug": { 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "debug", 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.876 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable/disable debug options", 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "jwt_token_ttl": { 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "jwt_token_ttl", 2024-02-14T15:19:05.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "28800", 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.878 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.879 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.880 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.881 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.883 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.885 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.886 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.887 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "motd": { 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "motd", 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.888 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "The message of the day", 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.889 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect_resolve_ip_addr": { 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "redirect_resolve_ip_addr", 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.890 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.891 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.892 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8080", 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.893 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.894 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.895 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl_server_port": { 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl_server_port", 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8443", 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.896 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.897 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "redirect", 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect" 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.898 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-02-14T15:19:05.899 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "url_prefix": { 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url_prefix", 2024-02-14T15:19:05.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.901 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.902 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_monitoring": { 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_monitoring", 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.903 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "monitor device health metrics", 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.904 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.905 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.906 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.907 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.908 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.909 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.911 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mark_out_threshold": { 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mark_out_threshold", 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2419200", 2024-02-14T15:19:05.912 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": { 2024-02-14T15:19:05.913 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_name", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "device_health_metrics", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.914 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "retention_period": { 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retention_period", 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15552000", 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.915 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain device health metrics", 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_frequency": { 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_frequency", 2024-02-14T15:19:05.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.917 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "self_heal": { 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "self_heal", 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.918 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.919 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and check device health", 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.920 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_threshold": { 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_threshold", 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7257600", 2024-02-14T15:19:05.921 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "diskprediction_local", 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.924 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.927 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.928 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.930 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: "predict_interval": { 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predict_interval", 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-02-14T15:19:05.931 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.932 INFO:teuthology.orchestra.run.smithi017.stdout: "predictor_model": { 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predictor_model", 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "prophetstor", 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.933 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.934 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "influx", 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": false, 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "influxdb python module not found", 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "batch_size": { 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "batch_size", 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.936 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5000", 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.937 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "database": { 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "database", 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "ceph", 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.938 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": { 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hostname", 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.939 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server hostname", 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.940 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "30", 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "5", 2024-02-14T15:19:05.941 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.942 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.943 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.945 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.946 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.947 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.948 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.949 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "password": { 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "password", 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.950 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "password of InfluxDB server user", 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "port": { 2024-02-14T15:19:05.951 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "port", 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8086", 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server port", 2024-02-14T15:19:05.952 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "false", 2024-02-14T15:19:05.953 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "threads": { 2024-02-14T15:19:05.954 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threads", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "32", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.955 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "username": { 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "username", 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.956 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "username of InfluxDB server user", 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "verify_ssl": { 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "verify_ssl", 2024-02-14T15:19:05.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "true", 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.958 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "insights", 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.959 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.960 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.961 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.962 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.963 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.964 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.965 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.966 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.967 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "iostat", 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.968 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.969 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.970 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.972 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.973 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.974 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "k8sevents", 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_event_retention_days": { 2024-02-14T15:19:05.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ceph_event_retention_days", 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7", 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-14T15:19:05.977 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "config_check_secs": { 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_check_secs", 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:05.978 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "10", 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.979 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.980 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.981 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.982 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.983 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.984 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.985 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.986 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "localpool", 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "failure_domain": { 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "failure_domain", 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.988 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "host", 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "failure domain for any created local pool", 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.990 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.991 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.992 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:05.993 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:05.994 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.996 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": { 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_size", 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.997 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default min_size for any created local pool", 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:05.998 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "num_rep": { 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "num_rep", 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3", 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:05.999 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default replica count for any created local pool", 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": { 2024-02-14T15:19:06.000 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_num", 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "128", 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default pg_num for any created local pool", 2024-02-14T15:19:06.001 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "prefix": { 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prefix", 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.002 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name prefix for any created local pool", 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.003 INFO:teuthology.orchestra.run.smithi017.stdout: "subtree": { 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "subtree", 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "rack", 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.004 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mds_autoscaler", 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.005 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.006 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.007 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.008 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.009 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.010 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.011 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.012 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirroring", 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.014 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.015 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.016 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.017 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.018 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.019 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.020 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.021 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "nfs", 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.022 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.023 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.024 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.025 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.026 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.027 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.029 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-02-14T15:19:06.030 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "fail_fs": { 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "fail_fs", 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.031 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.032 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.033 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.034 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.035 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.036 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.037 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.038 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.039 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator": { 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.040 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "rook", 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "test_orchestrator" 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Orchestrator backend", 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.041 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_perf_query", 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.042 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.043 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.044 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.045 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.046 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.048 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.049 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.050 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_support", 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.051 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.052 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.053 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.054 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.055 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.056 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.058 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_autoscaler", 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.059 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.060 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.061 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.062 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.063 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.064 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.065 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.066 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.068 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "threshold": { 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threshold", 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-02-14T15:19:06.069 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1.0", 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "scaling threshold", 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "progress", 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_pg_recovery_event": { 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_pg_recovery_event", 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.071 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.072 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.073 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.074 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.075 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.076 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.077 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.078 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.079 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "max_completed_events": { 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_completed_events", 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "50", 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.082 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "number of past completed events to remember", 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long the module is going to sleep", 2024-02-14T15:19:06.084 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus", 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.085 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cache", 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.086 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "exclude_perf_counters": { 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "exclude_perf_counters", 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.087 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.088 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.089 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.090 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.091 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.093 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.094 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.095 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools": { 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools", 2024-02-14T15:19:06.096 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.097 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300", 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.098 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_interval": { 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_interval", 2024-02-14T15:19:06.099 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15.0", 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.100 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.101 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:06.102 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "9283", 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-14T15:19:06.103 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "stale_cache_strategy": { 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stale_cache_strategy", 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "log", 2024-02-14T15:19:06.104 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-02-14T15:19:06.105 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "default", 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "default", 2024-02-14T15:19:06.106 INFO:teuthology.orchestra.run.smithi017.stdout: "error" 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-02-14T15:19:06.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.108 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.109 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.110 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.112 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.114 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.115 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.116 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_snap_create": { 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_snap_create", 2024-02-14T15:19:06.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.118 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "mirror_snapshot_schedule": { 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirror_snapshot_schedule", 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.119 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "trash_purge_schedule": { 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "trash_purge_schedule", 2024-02-14T15:19:06.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.121 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "restful", 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.122 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_auth": { 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_auth", 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.123 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-02-14T15:19:06.124 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.125 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.126 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.127 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.128 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.129 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.129 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.129 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.129 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.129 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.129 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.130 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.131 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.132 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.133 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.134 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.135 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rgw", 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.137 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.138 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.139 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.140 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.141 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.142 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.144 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rook", 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.145 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "drive_group_interval": { 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "drive_group_interval", 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300.0", 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.146 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.147 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.148 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.150 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.151 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.152 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.153 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "storage_class": { 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "storage_class", 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "local", 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.155 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "selftest", 2024-02-14T15:19:06.156 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.157 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.158 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.159 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.160 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.161 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.162 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "roption1": { 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption1", 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "roption2": { 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption2", 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "xyz", 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption1": { 2024-02-14T15:19:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption1", 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption2": { 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption2", 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption3": { 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption3", 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption4": { 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption4", 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption5": { 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption5", 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption6": { 2024-02-14T15:19:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption6", 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption7": { 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption7", 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "42", 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "testkey": { 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testkey", 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "testlkey": { 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testlkey", 2024-02-14T15:19:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.181 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "testnewline": { 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testnewline", 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.182 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.184 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-02-14T15:19:06.184 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.184 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.184 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.184 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_m_granularity": { 2024-02-14T15:19:06.184 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_m_granularity", 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow minute scheduled snapshots", 2024-02-14T15:19:06.185 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "dump_on_update": { 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dump_on_update", 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.186 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "dump database to debug log on update", 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.187 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.188 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.189 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.190 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.191 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.192 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.193 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.194 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.195 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stats", 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.196 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.197 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.199 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.200 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.201 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.202 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "status", 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.204 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.205 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.207 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.208 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.209 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.211 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.212 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telegraf", 2024-02-14T15:19:06.212 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.212 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "address": { 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "address", 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.213 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.214 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15", 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.215 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.216 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.217 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.219 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.220 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.222 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.223 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telemetry", 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_basic": { 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_basic", 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.224 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.225 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_crash": { 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_crash", 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-14T15:19:06.226 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_device": { 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_device", 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.227 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_ident": { 2024-02-14T15:19:06.228 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_ident", 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-14T15:19:06.229 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_perf": { 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_perf", 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.230 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "contact": { 2024-02-14T15:19:06.231 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "contact", 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.232 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "description": { 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "description", 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "device_url": { 2024-02-14T15:19:06.234 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_url", 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.235 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.236 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.237 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "24", 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "8", 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.238 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "last_opt_revision": { 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "last_opt_revision", 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1", 2024-02-14T15:19:06.239 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard": { 2024-02-14T15:19:06.240 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard", 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.241 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard_description": { 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard_description", 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.242 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.243 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.244 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.246 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.247 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.248 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.249 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.250 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "organization": { 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "organization", 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.251 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "proxy": { 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "proxy", 2024-02-14T15:19:06.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.253 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "url": { 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url", 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.254 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "test_orchestrator", 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.256 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.257 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.258 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.260 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.261 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.262 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.264 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.265 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.266 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.267 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.268 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.269 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.270 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_clones": { 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_clones", 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4", 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.272 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "periodic_async_work": { 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "periodic_async_work", 2024-02-14T15:19:06.273 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Periodically check for async work", 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.274 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_delay": { 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_delay", 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.275 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix", 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "discovery_interval": { 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "discovery_interval", 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.277 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "100", 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "identifier": { 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "identifier", 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.279 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.280 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.282 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.283 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.284 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.285 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.286 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.287 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.288 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_host": { 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_host", 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.289 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.290 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_port": { 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_port", 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10051", 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.291 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_sender": { 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_sender", 2024-02-14T15:19:06.292 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.293 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: "gid": 25288, 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "z", 2024-02-14T15:19:06.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_features": 4540701272860131327, 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "available_modules": [ 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "alerts", 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.295 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How frequently to reexamine health status", 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.296 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.297 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.298 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.299 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.300 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.301 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.302 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.303 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_destination": { 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_destination", 2024-02-14T15:19:06.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email address to send alerts to", 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.305 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_from_name": { 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_from_name", 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "Ceph", 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.306 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email From: name", 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_host": { 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_host", 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.307 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP server", 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.308 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_password": { 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_password", 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.309 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Password to authenticate with", 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_port": { 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_port", 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.310 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "465", 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP port", 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_sender": { 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_sender", 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.312 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP envelope sender", 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_ssl": { 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_ssl", 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.313 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_user": { 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_user", 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.315 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "User to authenticate as", 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "balancer", 2024-02-14T15:19:06.316 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "active": { 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "active", 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.317 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically balance PGs across cluster", 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_time": { 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_time", 2024-02-14T15:19:06.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0000", 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "beginning time of day to automatically balance", 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-14T15:19:06.319 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_weekday": { 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_weekday", 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.320 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_max_iterations": { 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_max_iterations", 2024-02-14T15:19:06.321 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "25", 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "250", 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.322 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_metrics": { 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_metrics", 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "pgs,objects,bytes", 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.323 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_step": { 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_step", 2024-02-14T15:19:06.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.5", 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0.001", 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "0.999", 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "aggressiveness of optimization", 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.325 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "end_time": { 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_time", 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2359", 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.326 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ending time of day to automatically balance", 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "end_weekday": { 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_weekday", 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.327 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.328 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.329 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.330 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.331 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.333 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.334 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.335 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "min_score": { 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_score", 2024-02-14T15:19:06.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.337 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "upmap", 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.338 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-compat", 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "none", 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "read", 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap", 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap-read" 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Balancer mode", 2024-02-14T15:19:06.339 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_ids": { 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_ids", 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.340 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.341 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.342 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_deviation": { 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_deviation", 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.343 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.344 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_optimizations": { 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_optimizations", 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-14T15:19:06.345 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm", 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.346 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_down_multiplier": { 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_down_multiplier", 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.347 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_refresh_rate": { 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_refresh_rate", 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.348 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "20", 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_starting_port": { 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_starting_port", 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4721", 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.350 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_ptrace": { 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_ptrace", 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.351 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.352 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_interval": { 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_interval", 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.353 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_memory_target_ratio": { 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_memory_target_ratio", 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.354 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.7", 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_log_destination": { 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm_log_destination", 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.356 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: "file", 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: "file,syslog", 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: "syslog" 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "cgroups_split": { 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cgroups_split", 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.358 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "config_checks_enabled": { 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_checks_enabled", 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.359 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "config_dashboard": { 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_dashboard", 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.361 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_alertmanager": { 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_alertmanager", 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.362 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_base": { 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_base", 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.364 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Container image name, without the tag", 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_elasticsearch": { 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_elasticsearch", 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.365 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "elasticsearch container image", 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.366 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_grafana": { 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_grafana", 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.367 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_haproxy": { 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_haproxy", 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.368 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "HAproxy container image", 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.369 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_agent": { 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_agent", 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger agent container image", 2024-02-14T15:19:06.370 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_collector": { 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_collector", 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.371 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger collector container image", 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_query": { 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_query", 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.373 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger query container image", 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_keepalived": { 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_keepalived", 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.374 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Keepalived container image", 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_loki": { 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_loki", 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.376 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Loki container image", 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_node_exporter": { 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_node_exporter", 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.377 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_nvmeof": { 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_nvmeof", 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.379 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Nvme-of container image", 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_prometheus": { 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_prometheus", 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.380 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.381 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_promtail": { 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_promtail", 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.382 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Promtail container image", 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_snmp_gateway": { 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_snmp_gateway", 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SNMP Gateway container image", 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.384 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "container_init": { 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_init", 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.385 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Run podman/docker with `--init`", 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "daemon_cache_timeout": { 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "daemon_cache_timeout", 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.386 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "default_cephadm_command_timeout": { 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_cephadm_command_timeout", 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "900", 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.388 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "default_registry": { 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_registry", 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.389 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io", 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.390 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "device_cache_timeout": { 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_cache_timeout", 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1800", 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.391 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache device inventory", 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: "device_enhanced_scan": { 2024-02-14T15:19:06.392 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_enhanced_scan", 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-14T15:19:06.393 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "facts_cache_timeout": { 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "facts_cache_timeout", 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.394 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache host facts data", 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "host_check_interval": { 2024-02-14T15:19:06.395 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "host_check_interval", 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to perform a host check", 2024-02-14T15:19:06.396 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "hw_monitoring": { 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hw_monitoring", 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.397 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "inventory_list_all": { 2024-02-14T15:19:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "inventory_list_all", 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-14T15:19:06.399 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.400 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.401 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "log_refresh_metadata": { 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_refresh_metadata", 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.402 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.404 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.405 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.406 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.408 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.409 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-14T15:19:06.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "*", 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.411 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: "max_count_per_host": { 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_count_per_host", 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.412 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of daemons per service per host", 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd_draining_count": { 2024-02-14T15:19:06.413 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_osd_draining_count", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.414 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "migration_current": { 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "migration_current", 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.415 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "internal - do not modify", 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-02-14T15:19:06.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "root", 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm-package", 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: "root" 2024-02-14T15:19:06.417 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "mode for remote execution of cephadm", 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "oob_default_addr": { 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "oob_default_addr", 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.418 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "169.254.1.1", 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "prometheus_alerts_path": { 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus_alerts_path", 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_insecure": { 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_insecure", 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.421 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_password": { 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_password", 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.423 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_url": { 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_url", 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.424 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_username": { 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_username", 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.426 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "secure_monitoring_stack": { 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "secure_monitoring_stack", 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.427 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.428 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "service_discovery_port": { 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "service_discovery_port", 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8765", 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.429 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "cephadm service discovery port", 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "ssh_config_file": { 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssh_config_file", 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.431 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "use_agent": { 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_agent", 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.432 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "use_repo_digest": { 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_repo_digest", 2024-02-14T15:19:06.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.434 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_failed_host_check": { 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_failed_host_check", 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.435 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if the host check fails", 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_daemons": { 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_daemons", 2024-02-14T15:19:06.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.437 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_hosts": { 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_hosts", 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.438 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crash", 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.440 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.441 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.442 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.443 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.444 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.445 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.446 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.447 INFO:teuthology.orchestra.run.smithi017.stdout: "retain_interval": { 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retain_interval", 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "31536000", 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.448 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_recent_interval": { 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_recent_interval", 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.449 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1209600", 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dashboard", 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-14T15:19:06.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.452 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_HOST": { 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.453 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-14T15:19:06.454 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.455 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_ENABLED": { 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_ENABLED", 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.456 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-14T15:19:06.457 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.458 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "ENABLE_BROWSABLE_API": { 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.459 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-14T15:19:06.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.461 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.462 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-14T15:19:06.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.464 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.465 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-14T15:19:06.466 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.467 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.468 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-14T15:19:06.469 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.470 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.471 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.472 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_PASSWORD": { 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.473 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.474 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_URL": { 2024-02-14T15:19:06.475 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_URL", 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.476 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_USERNAME": { 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.477 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-14T15:19:06.478 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.479 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.480 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-14T15:19:06.481 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.482 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.483 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_HOST": { 2024-02-14T15:19:06.484 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.485 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.486 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-14T15:19:06.487 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.488 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.489 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.490 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.491 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.492 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.493 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.494 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.495 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.496 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.497 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_ENABLED": { 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.498 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-14T15:19:06.499 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.500 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.501 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.502 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8", 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.503 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.504 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "45", 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.505 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ACCESS_KEY": { 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.506 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.507 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.508 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SECRET_KEY": { 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.509 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SSL_VERIFY": { 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.510 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.511 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.512 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.513 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.514 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.515 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "cross_origin_url": { 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cross_origin_url", 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.516 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.517 INFO:teuthology.orchestra.run.smithi017.stdout: "crt_file": { 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crt_file", 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.518 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "debug": { 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "debug", 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.519 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable/disable debug options", 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.520 INFO:teuthology.orchestra.run.smithi017.stdout: "jwt_token_ttl": { 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "jwt_token_ttl", 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "28800", 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.521 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.522 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.523 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.524 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.525 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.526 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.527 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.528 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.529 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.530 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "motd": { 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "motd", 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.531 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "The message of the day", 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect_resolve_ip_addr": { 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "redirect_resolve_ip_addr", 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.533 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.534 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.535 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8080", 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.536 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.537 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.538 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl_server_port": { 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl_server_port", 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8443", 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.539 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.540 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "redirect", 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect" 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.541 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-02-14T15:19:06.542 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "url_prefix": { 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url_prefix", 2024-02-14T15:19:06.543 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.544 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.545 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_monitoring": { 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_monitoring", 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "monitor device health metrics", 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.547 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.548 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.550 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.551 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.552 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.553 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mark_out_threshold": { 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mark_out_threshold", 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2419200", 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.555 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": { 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_name", 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.556 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "device_health_metrics", 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "retention_period": { 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retention_period", 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15552000", 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.558 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain device health metrics", 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_frequency": { 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_frequency", 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.559 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "self_heal": { 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "self_heal", 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.561 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.562 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and check device health", 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.563 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_threshold": { 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_threshold", 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7257600", 2024-02-14T15:19:06.564 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "diskprediction_local", 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.566 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.567 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.568 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.569 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.570 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.571 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.572 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "predict_interval": { 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predict_interval", 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.574 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "predictor_model": { 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predictor_model", 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.575 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "prophetstor", 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.576 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.577 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "influx", 2024-02-14T15:19:06.578 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": false, 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "influxdb python module not found", 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "batch_size": { 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "batch_size", 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5000", 2024-02-14T15:19:06.579 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.580 INFO:teuthology.orchestra.run.smithi017.stdout: "database": { 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "database", 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "ceph", 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.581 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": { 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hostname", 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.582 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server hostname", 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "30", 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "5", 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.584 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.585 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.586 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.587 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.588 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.589 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.590 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.591 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "password": { 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "password", 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.593 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "password of InfluxDB server user", 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "port": { 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "port", 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.594 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8086", 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server port", 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.595 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "false", 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.596 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "threads": { 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threads", 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.597 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "32", 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.598 INFO:teuthology.orchestra.run.smithi017.stdout: "username": { 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "username", 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "username of InfluxDB server user", 2024-02-14T15:19:06.599 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "verify_ssl": { 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "verify_ssl", 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.600 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "true", 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "insights", 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.602 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.603 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.604 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.605 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.606 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.607 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.609 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "iostat", 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.610 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.611 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.612 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.613 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.614 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.615 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.616 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.617 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "k8sevents", 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_event_retention_days": { 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ceph_event_retention_days", 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7", 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.619 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "config_check_secs": { 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_check_secs", 2024-02-14T15:19:06.620 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "10", 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.621 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.622 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.623 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.624 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.626 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.627 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.628 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "localpool", 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "failure_domain": { 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "failure_domain", 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.630 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "host", 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "failure domain for any created local pool", 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.631 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.632 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.633 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.634 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.635 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.636 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.637 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.638 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": { 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_size", 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.639 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default min_size for any created local pool", 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "num_rep": { 2024-02-14T15:19:06.640 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "num_rep", 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3", 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default replica count for any created local pool", 2024-02-14T15:19:06.641 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": { 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_num", 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.642 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "128", 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default pg_num for any created local pool", 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "prefix": { 2024-02-14T15:19:06.643 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prefix", 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.644 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name prefix for any created local pool", 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "subtree": { 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "subtree", 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.645 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "rack", 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mds_autoscaler", 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.647 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.648 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.650 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.651 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.652 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.652 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.652 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.652 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.652 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.652 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.652 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.653 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.654 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirroring", 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.655 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.656 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.657 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.658 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.659 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.660 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.661 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.662 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "nfs", 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.664 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.665 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.666 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.667 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.668 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.669 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.670 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: "fail_fs": { 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "fail_fs", 2024-02-14T15:19:06.672 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.673 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.674 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.675 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.676 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.678 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.679 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.680 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator": { 2024-02-14T15:19:06.681 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "rook", 2024-02-14T15:19:06.682 INFO:teuthology.orchestra.run.smithi017.stdout: "test_orchestrator" 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Orchestrator backend", 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_perf_query", 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.684 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.685 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.686 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.687 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.688 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.689 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.690 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_support", 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.693 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.694 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.695 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.697 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.698 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.699 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_autoscaler", 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.700 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.701 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.702 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.703 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.704 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.705 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.706 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.707 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.708 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "threshold": { 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threshold", 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1.0", 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.710 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "scaling threshold", 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.711 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "progress", 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_pg_recovery_event": { 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_pg_recovery_event", 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.712 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-02-14T15:19:06.713 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.714 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.715 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.716 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.717 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.719 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.720 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.721 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "max_completed_events": { 2024-02-14T15:19:06.722 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_completed_events", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "50", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "number of past completed events to remember", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.723 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:06.724 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long the module is going to sleep", 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus", 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cache", 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.726 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "exclude_perf_counters": { 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "exclude_perf_counters", 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.728 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.729 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.730 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.732 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.734 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.735 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools": { 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools", 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.737 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.738 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300", 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.739 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_interval": { 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_interval", 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15.0", 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.740 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.741 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "9283", 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.743 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "stale_cache_strategy": { 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stale_cache_strategy", 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.744 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "log", 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.745 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "default", 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.746 INFO:teuthology.orchestra.run.smithi017.stdout: "default", 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "error" 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-02-14T15:19:06.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.748 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.749 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.750 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.751 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.752 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.754 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.755 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.756 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_snap_create": { 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_snap_create", 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.757 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.758 INFO:teuthology.orchestra.run.smithi017.stdout: "mirror_snapshot_schedule": { 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirror_snapshot_schedule", 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.759 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "trash_purge_schedule": { 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "trash_purge_schedule", 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "restful", 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_auth": { 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_auth", 2024-02-14T15:19:06.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.763 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.764 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.765 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.766 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.767 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.768 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.769 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.770 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.771 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.774 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.775 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.776 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rgw", 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.777 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.778 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.779 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.780 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.781 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.782 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.783 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.784 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rook", 2024-02-14T15:19:06.785 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "drive_group_interval": { 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "drive_group_interval", 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.786 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300.0", 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.787 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.788 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.789 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.790 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.791 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.792 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.793 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.794 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "storage_class": { 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "storage_class", 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "local", 2024-02-14T15:19:06.795 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "selftest", 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.797 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.798 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.799 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.800 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.801 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.802 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.803 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.804 INFO:teuthology.orchestra.run.smithi017.stdout: "roption1": { 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption1", 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.805 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "roption2": { 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption2", 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.806 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "xyz", 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.807 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption1": { 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption1", 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.808 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption2": { 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption2", 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.809 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.810 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption3": { 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption3", 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.811 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption4": { 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption4", 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.812 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption5": { 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption5", 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.814 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption6": { 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption6", 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.815 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption7": { 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption7", 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "42", 2024-02-14T15:19:06.817 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "testkey": { 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testkey", 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.818 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "testlkey": { 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testlkey", 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.820 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "testnewline": { 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testnewline", 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.821 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_m_granularity": { 2024-02-14T15:19:06.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_m_granularity", 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow minute scheduled snapshots", 2024-02-14T15:19:06.824 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "dump_on_update": { 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dump_on_update", 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.825 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "dump database to debug log on update", 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.826 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.827 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.829 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.830 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.831 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.832 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.833 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stats", 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.834 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.835 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.836 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.837 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.839 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.840 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.841 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.842 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "status", 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.843 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.844 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.845 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.847 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.848 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.849 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telegraf", 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "address": { 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "address", 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.851 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15", 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.853 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.854 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.855 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.857 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.858 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.859 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.860 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.861 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telemetry", 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_basic": { 2024-02-14T15:19:06.862 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_basic", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.863 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_crash": { 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_crash", 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.864 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_device": { 2024-02-14T15:19:06.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_device", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.866 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_ident": { 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_ident", 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.867 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_perf": { 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_perf", 2024-02-14T15:19:06.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "contact": { 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "contact", 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.870 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "description": { 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "description", 2024-02-14T15:19:06.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.872 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "device_url": { 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_url", 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-14T15:19:06.873 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-02-14T15:19:06.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.875 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "24", 2024-02-14T15:19:06.876 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "8", 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "last_opt_revision": { 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "last_opt_revision", 2024-02-14T15:19:06.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1", 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.878 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard": { 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard", 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.879 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard_description": { 2024-02-14T15:19:06.880 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard_description", 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.881 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.882 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.883 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.885 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.886 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.887 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.888 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "organization": { 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "organization", 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.890 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "proxy": { 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "proxy", 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.891 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.892 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "url": { 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url", 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.893 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "test_orchestrator", 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.895 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.896 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.897 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.898 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.899 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.900 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.901 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.903 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.904 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.905 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.906 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.907 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.908 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.909 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.910 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_clones": { 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_clones", 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4", 2024-02-14T15:19:06.911 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.912 INFO:teuthology.orchestra.run.smithi017.stdout: "periodic_async_work": { 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "periodic_async_work", 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Periodically check for async work", 2024-02-14T15:19:06.913 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_delay": { 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_delay", 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.914 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix", 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "discovery_interval": { 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "discovery_interval", 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.916 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "100", 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "identifier": { 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "identifier", 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.918 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.919 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.921 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.922 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.923 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.924 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.925 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.926 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.927 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_host": { 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_host", 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.928 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_port": { 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_port", 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10051", 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.930 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_sender": { 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_sender", 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.931 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: "modules": [ 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: "dashboard", 2024-02-14T15:19:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: "iostat", 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: "nfs", 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: "restful", 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_schedule" 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: "available_modules": [ 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "alerts", 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.934 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.935 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How frequently to reexamine health status", 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.936 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.937 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.938 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.939 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.940 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.941 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.942 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.943 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_destination": { 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_destination", 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.944 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email address to send alerts to", 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.945 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_from_name": { 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_from_name", 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "Ceph", 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Email From: name", 2024-02-14T15:19:06.946 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_host": { 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_host", 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.947 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP server", 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_password": { 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_password", 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Password to authenticate with", 2024-02-14T15:19:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_port": { 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_port", 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.950 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "465", 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP port", 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_sender": { 2024-02-14T15:19:06.951 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_sender", 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SMTP envelope sender", 2024-02-14T15:19:06.952 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_ssl": { 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_ssl", 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.953 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.954 INFO:teuthology.orchestra.run.smithi017.stdout: "smtp_user": { 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "smtp_user", 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "User to authenticate as", 2024-02-14T15:19:06.955 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "balancer", 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.956 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "active": { 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "active", 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically balance PGs across cluster", 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_time": { 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_time", 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.958 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0000", 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "beginning time of day to automatically balance", 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "begin_weekday": { 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "begin_weekday", 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-02-14T15:19:06.960 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_max_iterations": { 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_max_iterations", 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.961 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "25", 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "250", 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.962 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_metrics": { 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_metrics", 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "pgs,objects,bytes", 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.963 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_compat_step": { 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crush_compat_step", 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.964 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.5", 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0.001", 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "0.999", 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "aggressiveness of optimization", 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.965 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "end_time": { 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_time", 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2359", 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.966 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ending time of day to automatically balance", 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "end_weekday": { 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "end_weekday", 2024-02-14T15:19:06.967 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "0", 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "6", 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-14T15:19:06.968 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.969 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.970 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.971 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.972 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.973 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.974 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.975 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.976 INFO:teuthology.orchestra.run.smithi017.stdout: "min_score": { 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_score", 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.977 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.978 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "upmap", 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-compat", 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "none", 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "read", 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap", 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap-read" 2024-02-14T15:19:06.979 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Balancer mode", 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_ids": { 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_ids", 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.980 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.982 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_deviation": { 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_deviation", 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.983 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "upmap_max_optimizations": { 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "upmap_max_optimizations", 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.985 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm", 2024-02-14T15:19:06.986 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_down_multiplier": { 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_down_multiplier", 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-02-14T15:19:06.987 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_refresh_rate": { 2024-02-14T15:19:06.988 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_refresh_rate", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "20", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.989 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "agent_starting_port": { 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "agent_starting_port", 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4721", 2024-02-14T15:19:06.990 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_ptrace": { 2024-02-14T15:19:06.991 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_ptrace", 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-14T15:19:06.992 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_interval": { 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_interval", 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.993 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "autotune_memory_target_ratio": { 2024-02-14T15:19:06.994 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "autotune_memory_target_ratio", 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0.7", 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-14T15:19:06.995 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_log_destination": { 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephadm_log_destination", 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:06.996 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: "file", 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: "file,syslog", 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: "syslog" 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-14T15:19:06.997 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "cgroups_split": { 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cgroups_split", 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:06.998 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:06.999 INFO:teuthology.orchestra.run.smithi017.stdout: "config_checks_enabled": { 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_checks_enabled", 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.000 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: "config_dashboard": { 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_dashboard", 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.001 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.002 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_alertmanager": { 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_alertmanager", 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-14T15:19:07.003 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.004 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_base": { 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_base", 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.005 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Container image name, without the tag", 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_elasticsearch": { 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_elasticsearch", 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.006 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "elasticsearch container image", 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.007 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_grafana": { 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_grafana", 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.008 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.009 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_haproxy": { 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_haproxy", 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.010 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "HAproxy container image", 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_agent": { 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_agent", 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.011 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger agent container image", 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.012 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_collector": { 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_collector", 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-14T15:19:07.013 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger collector container image", 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_jaeger_query": { 2024-02-14T15:19:07.014 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_jaeger_query", 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.015 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Jaeger query container image", 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_keepalived": { 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_keepalived", 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.016 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Keepalived container image", 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.017 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_loki": { 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_loki", 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-14T15:19:07.018 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Loki container image", 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_node_exporter": { 2024-02-14T15:19:07.019 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_node_exporter", 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.020 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_nvmeof": { 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_nvmeof", 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.021 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Nvme-of container image", 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.022 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_prometheus": { 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_prometheus", 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-14T15:19:07.023 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Prometheus container image", 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.024 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_promtail": { 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_promtail", 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.025 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Promtail container image", 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image_snmp_gateway": { 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_image_snmp_gateway", 2024-02-14T15:19:07.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "SNMP Gateway container image", 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.027 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "container_init": { 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "container_init", 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.028 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Run podman/docker with `--init`", 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.029 INFO:teuthology.orchestra.run.smithi017.stdout: "daemon_cache_timeout": { 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "daemon_cache_timeout", 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.030 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "default_cephadm_command_timeout": { 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_cephadm_command_timeout", 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.031 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "900", 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "default_registry": { 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "default_registry", 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "docker.io", 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.033 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "device_cache_timeout": { 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_cache_timeout", 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.034 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1800", 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache device inventory", 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "device_enhanced_scan": { 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_enhanced_scan", 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.036 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "facts_cache_timeout": { 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "facts_cache_timeout", 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.037 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "seconds to cache host facts data", 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "host_check_interval": { 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "host_check_interval", 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.039 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to perform a host check", 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "hw_monitoring": { 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hw_monitoring", 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.040 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.041 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: "inventory_list_all": { 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "inventory_list_all", 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.042 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.043 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.044 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "log_refresh_metadata": { 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_refresh_metadata", 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.046 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.047 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.048 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.049 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.050 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.051 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.052 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.053 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.054 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "*", 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.055 INFO:teuthology.orchestra.run.smithi017.stdout: "max_count_per_host": { 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_count_per_host", 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.056 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of daemons per service per host", 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd_draining_count": { 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_osd_draining_count", 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.057 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.058 INFO:teuthology.orchestra.run.smithi017.stdout: "migration_current": { 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "migration_current", 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.059 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "internal - do not modify", 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "mode": { 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mode", 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.060 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "root", 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm-package", 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: "root" 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.061 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "mode for remote execution of cephadm", 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "oob_default_addr": { 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "oob_default_addr", 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.062 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "169.254.1.1", 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.063 INFO:teuthology.orchestra.run.smithi017.stdout: "prometheus_alerts_path": { 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus_alerts_path", 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.064 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_insecure": { 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_insecure", 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.065 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_password": { 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_password", 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.067 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_url": { 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_url", 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.068 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "registry_username": { 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "registry_username", 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.070 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "secure_monitoring_stack": { 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "secure_monitoring_stack", 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.071 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.072 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "service_discovery_port": { 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "service_discovery_port", 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8765", 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.073 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "cephadm service discovery port", 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "ssh_config_file": { 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssh_config_file", 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.074 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.075 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "use_agent": { 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_agent", 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.076 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "use_repo_digest": { 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "use_repo_digest", 2024-02-14T15:19:07.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.078 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_failed_host_check": { 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_failed_host_check", 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.079 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if the host check fails", 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_daemons": { 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_daemons", 2024-02-14T15:19:07.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.081 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_on_stray_hosts": { 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_on_stray_hosts", 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.082 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crash", 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.084 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.085 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.087 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.088 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.089 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.090 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.091 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "retain_interval": { 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retain_interval", 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "31536000", 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.092 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_recent_interval": { 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_recent_interval", 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.093 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1209600", 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.094 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dashboard", 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-14T15:19:07.095 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.096 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_HOST": { 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.097 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-14T15:19:07.098 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.099 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_ENABLED": { 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_ENABLED", 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.100 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-14T15:19:07.101 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.102 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: "ENABLE_BROWSABLE_API": { 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.103 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.104 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.104 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.104 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.104 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.104 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.104 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.104 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.106 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.107 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.108 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.109 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.110 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.111 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.113 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.114 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.116 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.117 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_PASSWORD": { 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.119 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.120 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_URL": { 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_URL", 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.122 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_API_USERNAME": { 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.123 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.125 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.126 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.127 INFO:teuthology.orchestra.run.smithi017.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.128 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.129 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.130 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_HOST": { 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.131 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.132 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.133 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.135 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.136 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.137 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.138 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-14T15:19:07.139 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.140 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.141 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.142 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.143 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_ENABLED": { 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.144 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-14T15:19:07.145 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.146 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:07.147 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-14T15:19:07.148 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.149 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "45", 2024-02-14T15:19:07.150 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ACCESS_KEY": { 2024-02-14T15:19:07.151 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.152 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.153 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "admin", 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.154 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SECRET_KEY": { 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.155 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "RGW_API_SSL_VERIFY": { 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.156 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.158 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.159 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.161 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: "cross_origin_url": { 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cross_origin_url", 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.162 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "crt_file": { 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "crt_file", 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.164 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "debug": { 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "debug", 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.165 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Enable/disable debug options", 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "jwt_token_ttl": { 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "jwt_token_ttl", 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "28800", 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.167 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.168 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.170 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.171 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.172 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.173 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.174 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.175 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.176 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "motd": { 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "motd", 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.177 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "The message of the day", 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.178 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect_resolve_ip_addr": { 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "redirect_resolve_ip_addr", 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.179 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.180 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8080", 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.182 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.183 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl_server_port": { 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl_server_port", 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8443", 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.185 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.186 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "redirect", 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "redirect" 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.187 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.188 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "url_prefix": { 2024-02-14T15:19:07.189 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url_prefix", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.190 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.191 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_monitoring": { 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_monitoring", 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.192 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "monitor device health metrics", 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.193 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.194 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.195 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.196 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.198 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.199 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.200 INFO:teuthology.orchestra.run.smithi017.stdout: "mark_out_threshold": { 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mark_out_threshold", 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "2419200", 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-14T15:19:07.201 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": { 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pool_name", 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.202 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "device_health_metrics", 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "retention_period": { 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "retention_period", 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15552000", 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.204 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long to retain device health metrics", 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_frequency": { 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_frequency", 2024-02-14T15:19:07.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-14T15:19:07.206 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: "self_heal": { 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "self_heal", 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.207 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.208 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.209 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how frequently to wake up and check device health", 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "warn_threshold": { 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "warn_threshold", 2024-02-14T15:19:07.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7257600", 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-14T15:19:07.211 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "diskprediction_local", 2024-02-14T15:19:07.212 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.213 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.214 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.215 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.216 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.217 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.218 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.219 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "predict_interval": { 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predict_interval", 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "86400", 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.221 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "predictor_model": { 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "predictor_model", 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.222 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "prophetstor", 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "600", 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.224 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "influx", 2024-02-14T15:19:07.225 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": false, 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "influxdb python module not found", 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "batch_size": { 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "batch_size", 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5000", 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.226 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "database": { 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "database", 2024-02-14T15:19:07.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "ceph", 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.228 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": { 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "hostname", 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.229 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server hostname", 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.230 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "30", 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "5", 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.231 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.232 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.232 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.232 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.232 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.232 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.232 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.233 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.234 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.236 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.237 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.238 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.239 INFO:teuthology.orchestra.run.smithi017.stdout: "password": { 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "password", 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.240 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "password of InfluxDB server user", 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "port": { 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "port", 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.241 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "8086", 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "InfluxDB server port", 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "ssl": { 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ssl", 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "false", 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.243 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "threads": { 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threads", 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.244 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "32", 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.245 INFO:teuthology.orchestra.run.smithi017.stdout: "username": { 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "username", 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.246 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "username of InfluxDB server user", 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "verify_ssl": { 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "verify_ssl", 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.247 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "true", 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "insights", 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.249 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.250 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.251 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.252 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.253 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.254 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.255 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "iostat", 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.257 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.258 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.260 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.261 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.262 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.263 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.264 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "k8sevents", 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.265 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_event_retention_days": { 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "ceph_event_retention_days", 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "7", 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.266 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "config_check_secs": { 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "config_check_secs", 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.267 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "10", 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.269 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.270 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.271 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "localpool", 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "failure_domain": { 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "failure_domain", 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "host", 2024-02-14T15:19:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "failure domain for any created local pool", 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": { 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "min_size", 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default min_size for any created local pool", 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "num_rep": { 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "num_rep", 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3", 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default replica count for any created local pool", 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": { 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_num", 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "128", 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "default pg_num for any created local pool", 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: "prefix": { 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prefix", 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "name prefix for any created local pool", 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "subtree": { 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "subtree", 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "rack", 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mds_autoscaler", 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirroring", 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "nfs", 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.314 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.315 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.316 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.317 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "fail_fs": { 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "fail_fs", 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.319 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.320 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.321 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.322 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.323 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.325 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.327 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator": { 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "orchestrator", 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.328 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm", 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "rook", 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "test_orchestrator" 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.329 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Orchestrator backend", 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_perf_query", 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.330 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.331 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.332 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.333 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.334 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.335 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.336 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.337 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.338 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "osd_support", 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.339 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.340 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.341 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.342 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.343 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.344 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.345 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "pg_autoscaler", 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.347 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.348 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.349 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.350 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.351 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.352 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.353 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.355 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: "threshold": { 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "threshold", 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:07.356 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "3.0", 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1.0", 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "scaling threshold", 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.357 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "progress", 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_pg_recovery_event": { 2024-02-14T15:19:07.358 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_pg_recovery_event", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.359 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.360 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.361 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.362 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.363 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.364 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.366 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.367 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.368 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "max_completed_events": { 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_completed_events", 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.369 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "50", 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "number of past completed events to remember", 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.370 INFO:teuthology.orchestra.run.smithi017.stdout: "sleep_interval": { 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "sleep_interval", 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "5", 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.371 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "how long the module is going to sleep", 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.372 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "prometheus", 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cache", 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.373 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "exclude_perf_counters": { 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "exclude_perf_counters", 2024-02-14T15:19:07.374 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-14T15:19:07.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.376 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.377 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.378 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.380 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.381 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.382 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.383 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools": { 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools", 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.384 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.385 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300", 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.386 INFO:teuthology.orchestra.run.smithi017.stdout: "scrape_interval": { 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "scrape_interval", 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15.0", 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.387 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.388 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "::", 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "9283", 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.390 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "stale_cache_strategy": { 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stale_cache_strategy", 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "log", 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.392 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_behaviour": { 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_behaviour", 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "default", 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.393 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: "default", 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: "error" 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.394 INFO:teuthology.orchestra.run.smithi017.stdout: "standby_error_status_code": { 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "standby_error_status_code", 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "500", 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "400", 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "599", 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.395 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.396 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.397 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.398 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.399 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.400 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.401 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.403 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_snap_create": { 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_snap_create", 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.404 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10", 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "mirror_snapshot_schedule": { 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mirror_snapshot_schedule", 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.406 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "trash_purge_schedule": { 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "trash_purge_schedule", 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.407 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "restful", 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_auth": { 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enable_auth", 2024-02-14T15:19:07.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.410 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "key_file": { 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "key_file", 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.411 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.413 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.414 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.415 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.416 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.417 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.418 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.419 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "server_addr": { 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_addr", 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.420 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "server_port": { 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "server_port", 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.421 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.422 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rgw", 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.423 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.424 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.425 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.426 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.427 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.428 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.429 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.430 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rook", 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: "drive_group_interval": { 2024-02-14T15:19:07.431 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "drive_group_interval", 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "300.0", 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-14T15:19:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.433 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.434 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.435 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.437 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.438 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.439 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "storage_class": { 2024-02-14T15:19:07.440 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "storage_class", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "local", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.441 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "selftest", 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.442 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.443 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.444 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.445 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.446 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.447 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.449 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "roption1": { 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption1", 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.450 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "roption2": { 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "roption2", 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "xyz", 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.452 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption1": { 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption1", 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.453 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption2": { 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption2", 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.455 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption3": { 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption3", 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "float", 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.456 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption4": { 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption4", 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.458 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption5": { 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption5", 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.459 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.460 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption6": { 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption6", 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.461 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "rwoption7": { 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rwoption7", 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.462 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "1", 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "42", 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.463 INFO:teuthology.orchestra.run.smithi017.stdout: "testkey": { 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testkey", 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.464 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "testlkey": { 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testlkey", 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.465 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.466 INFO:teuthology.orchestra.run.smithi017.stdout: "testnewline": { 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "testnewline", 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.467 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-02-14T15:19:07.468 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_m_granularity": { 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "allow_m_granularity", 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.469 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "allow minute scheduled snapshots", 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "dump_on_update": { 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "dump_on_update", 2024-02-14T15:19:07.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "dump database to debug log on update", 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.471 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.472 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.473 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.474 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.476 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.477 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.478 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "stats", 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.480 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.481 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.483 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.484 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.485 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.486 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.487 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "status", 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.488 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.489 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.490 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.491 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.492 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.493 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.494 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.495 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telegraf", 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "address": { 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "address", 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.497 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.498 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "15", 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.499 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.500 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.501 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.503 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.504 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.505 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.506 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "telemetry", 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_basic": { 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_basic", 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.508 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.509 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_crash": { 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_crash", 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.510 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_device": { 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_device", 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.511 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.512 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "True", 2024-02-14T15:19:07.512 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.512 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.512 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.512 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-14T15:19:07.514 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.514 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_ident": { 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_ident", 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.515 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "channel_perf": { 2024-02-14T15:19:07.516 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "channel_perf", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.517 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "contact": { 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "contact", 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.518 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "description": { 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "description", 2024-02-14T15:19:07.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.520 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "device_url": { 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_url", 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.521 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: "enabled": { 2024-02-14T15:19:07.522 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "enabled", 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.523 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "24", 2024-02-14T15:19:07.524 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "8", 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "last_opt_revision": { 2024-02-14T15:19:07.525 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "last_opt_revision", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "1", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.526 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard": { 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard", 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.527 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "leaderboard_description": { 2024-02-14T15:19:07.528 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "leaderboard_description", 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.530 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.531 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.532 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.534 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.535 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.536 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "organization": { 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "organization", 2024-02-14T15:19:07.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.538 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "proxy": { 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "proxy", 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.539 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "url": { 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "url", 2024-02-14T15:19:07.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.541 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "test_orchestrator", 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.542 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.543 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.544 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.545 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.546 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.547 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.548 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.549 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.550 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.551 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.552 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.553 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.554 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.555 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.556 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.557 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.558 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "max_concurrent_clones": { 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "max_concurrent_clones", 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.559 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "4", 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.560 INFO:teuthology.orchestra.run.smithi017.stdout: "periodic_async_work": { 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "periodic_async_work", 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Periodically check for async work", 2024-02-14T15:19:07.561 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: "snapshot_clone_delay": { 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snapshot_clone_delay", 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.562 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "0", 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.563 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix", 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: "can_run": true, 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: "error_string": "", 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: "module_options": { 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: "discovery_interval": { 2024-02-14T15:19:07.564 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "discovery_interval", 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "uint", 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "100", 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.565 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "identifier": { 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "identifier", 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.566 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "interval": { 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "interval", 2024-02-14T15:19:07.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "secs", 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "60", 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.568 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "log_level": { 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_level", 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.569 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.570 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster": { 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster", 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.571 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.572 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_cluster_level": { 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_cluster_level", 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "info", 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.573 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [ 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "", 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "critical", 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "debug", 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "error", 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "info", 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "warning" 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": { 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "log_to_file", 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "bool", 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-02-14T15:19:07.575 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "False", 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_host": { 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_host", 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "", 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.577 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_port": { 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_port", 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "int", 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.578 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "10051", 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "zabbix_sender": { 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "zabbix_sender", 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "str", 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "level": "advanced", 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 0, 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "min": "", 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "max": "", 2024-02-14T15:19:07.580 INFO:teuthology.orchestra.run.smithi017.stdout: "enum_allowed": [], 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: "desc": "", 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: "long_desc": "", 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: "tags": [], 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: "see_also": [] 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.581 INFO:teuthology.orchestra.run.smithi017.stdout: "services": { 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "dashboard": "https://172.21.15.17:8443/" 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "always_on_modules": { 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus": [ 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-02-14T15:19:07.582 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific": [ 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-02-14T15:19:07.583 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-02-14T15:19:07.584 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy": [ 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-02-14T15:19:07.585 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "reef": [ 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-02-14T15:19:07.586 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "squid": [ 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "balancer", 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "crash", 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "devicehealth", 2024-02-14T15:19:07.587 INFO:teuthology.orchestra.run.smithi017.stdout: "orchestrator", 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscaler", 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: "progress", 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_support", 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: "status", 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: "telemetry", 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: "volumes" 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.588 INFO:teuthology.orchestra.run.smithi017.stdout: "last_failure_osd_epoch": 94, 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: "active_clients": [ 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "devicehealth", 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1892580379 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.589 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "libcephsqlite", 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 608188641 2024-02-14T15:19:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd_support", 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-02-14T15:19:07.591 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3347488929 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-02-14T15:19:07.592 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3709948856 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "snap_schedule", 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.593 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 770003468 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "volumes", 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-02-14T15:19:07.594 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-02-14T15:19:07.595 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:0", 2024-02-14T15:19:07.595 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2845936929 2024-02-14T15:19:07.595 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.595 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.595 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:07.595 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-02-14T15:19:07.595 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:19:07.597 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:05 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2925181803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:19:07.597 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:05 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/500337067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:19:07.597 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:05 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/113380621' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-14T15:19:07.597 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:06 smithi076 ceph-mon[37057]: pgmap v495: 129 pgs: 129 active+clean; 845 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 253 op/s 2024-02-14T15:19:07.597 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:05 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2925181803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:19:07.597 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:05 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/500337067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:19:07.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:05 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/113380621' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-14T15:19:07.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:06 smithi050 ceph-mon[36512]: pgmap v495: 129 pgs: 129 active+clean; 845 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 253 op/s 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f12180a22e0 msgr2=0x7f12180a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f12180a22e0 0x7f12180a47a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f122c103870 tx=0x7f1214005f90 comp rx=0 tx=0).stop 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c100350 msgr2=0x7f122c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c100350 0x7f122c101de0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f122400e9f0 tx=0x7f122400eeb0 comp rx=0 tx=0).stop 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 shutdown_connections 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f12180a22e0 0x7f12180a47a0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f122c103870 tx=0x7f1214005f90 comp rx=0 tx=0).stop 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c102860 0x7f122c1a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f122c105420 0x7f122c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 --2- 172.21.15.17:0/113380621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f122c100350 0x7f122c101de0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 >> 172.21.15.17:0/113380621 conn(0x7f122c0772b0 msgr2=0x7f122c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.613+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 shutdown_connections 2024-02-14T15:19:07.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:05.614+0000 7f1230823640 1 -- 172.21.15.17:0/113380621 wait complete. 2024-02-14T15:19:07.606 DEBUG:tasks.check_counter:Getting stats from a 2024-02-14T15:19:07.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-14T15:19:07.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:05 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2925181803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:19:07.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:05 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/500337067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:19:07.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:05 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/113380621' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-14T15:19:07.607 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:06 smithi017 ceph-mon[32549]: pgmap v495: 129 pgs: 129 active+clean; 845 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 253 op/s 2024-02-14T15:19:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:19:09.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:08 smithi050 ceph-mon[36512]: pgmap v496: 129 pgs: 129 active+clean; 847 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.4 MiB/s wr, 343 op/s 2024-02-14T15:19:09.207 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:08 smithi017 ceph-mon[32549]: pgmap v496: 129 pgs: 129 active+clean; 847 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.4 MiB/s wr, 343 op/s 2024-02-14T15:19:09.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:08 smithi076 ceph-mon[37057]: pgmap v496: 129 pgs: 129 active+clean; 847 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.4 MiB/s wr, 343 op/s 2024-02-14T15:19:10.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:09 smithi017 ceph-mon[32549]: osdmap e117: 12 total, 12 up, 12 in 2024-02-14T15:19:10.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:09 smithi050 ceph-mon[36512]: osdmap e117: 12 total, 12 up, 12 in 2024-02-14T15:19:10.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:09 smithi076 ceph-mon[37057]: osdmap e117: 12 total, 12 up, 12 in 2024-02-14T15:19:10.296 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:19:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 0, 2024-02-14T15:19:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 2, 2024-02-14T15:19:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 0, 2024-02-14T15:19:10.297 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 0, 2024-02-14T15:19:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-02-14T15:19:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 0, 2024-02-14T15:19:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.001252260, 2024-02-14T15:19:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.000000000, 2024-02-14T15:19:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.000241136, 2024-02-14T15:19:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:10.298 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:10.299 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-14T15:19:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 0 2024-02-14T15:19:10.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 309, 2024-02-14T15:19:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 297, 2024-02-14T15:19:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1731008, 2024-02-14T15:19:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 178496, 2024-02-14T15:19:10.301 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 2, 2024-02-14T15:19:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2024-02-14T15:19:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.050581101, 2024-02-14T15:19:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.015953968, 2024-02-14T15:19:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.039350980, 2024-02-14T15:19:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.005237221, 2024-02-14T15:19:10.302 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 296, 2024-02-14T15:19:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.011811746, 2024-02-14T15:19:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000039904 2024-02-14T15:19:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:10.303 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1731008, 2024-02-14T15:19:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 178496 2024-02-14T15:19:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.304 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 7, 2024-02-14T15:19:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 229, 2024-02-14T15:19:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 3552, 2024-02-14T15:19:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 273008, 2024-02-14T15:19:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2024-02-14T15:19:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-02-14T15:19:10.305 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.015403371, 2024-02-14T15:19:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.010236490, 2024-02-14T15:19:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.002565102, 2024-02-14T15:19:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 227, 2024-02-14T15:19:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.013994163, 2024-02-14T15:19:10.306 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000061648 2024-02-14T15:19:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:10.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-02-14T15:19:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 273008 2024-02-14T15:19:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-02-14T15:19:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 0, 2024-02-14T15:19:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-02-14T15:19:10.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-02-14T15:19:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:10.309 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:10.310 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:10.311 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:10.312 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:10.313 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-02-14T15:19:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 148754, 2024-02-14T15:19:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 31, 2024-02-14T15:19:10.314 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-02-14T15:19:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-02-14T15:19:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:10.315 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 0, 2024-02-14T15:19:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 0, 2024-02-14T15:19:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:10.316 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-02-14T15:19:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 928, 2024-02-14T15:19:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 2, 2024-02-14T15:19:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-02-14T15:19:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-02-14T15:19:10.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-02-14T15:19:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-02-14T15:19:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-02-14T15:19:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 312, 2024-02-14T15:19:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1687410, 2024-02-14T15:19:10.318 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 312, 2024-02-14T15:19:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-02-14T15:19:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-02-14T15:19:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 312, 2024-02-14T15:19:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1687410, 2024-02-14T15:19:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-02-14T15:19:10.319 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:10.320 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:10.320 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:19:10.943 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-02-14T15:19:10.944 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-02-14T15:19:10.944 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-02-14T15:19:10.944 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-02-14T15:19:10.944 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-02-14T15:19:10.944 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-02-14T15:19:10.945 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-02-14T15:19:10.945 DEBUG:tasks.check_counter:Getting stats from d 2024-02-14T15:19:10.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-14T15:19:11.198 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:19:11.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:10 smithi076 ceph-mon[37057]: pgmap v498: 129 pgs: 129 active+clean; 847 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.4 MiB/s wr, 260 op/s 2024-02-14T15:19:11.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:10 smithi017 ceph-mon[32549]: pgmap v498: 129 pgs: 129 active+clean; 847 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.4 MiB/s wr, 260 op/s 2024-02-14T15:19:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:10 smithi050 ceph-mon[36512]: pgmap v498: 129 pgs: 129 active+clean; 847 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.4 MiB/s wr, 260 op/s 2024-02-14T15:19:13.023 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:19:13.024 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:13.024 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 0, 2024-02-14T15:19:13.024 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 2, 2024-02-14T15:19:13.025 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 0, 2024-02-14T15:19:13.025 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 0, 2024-02-14T15:19:13.025 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-02-14T15:19:13.025 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 0, 2024-02-14T15:19:13.025 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.001273528, 2024-02-14T15:19:13.025 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.000000000, 2024-02-14T15:19:13.026 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.000295036, 2024-02-14T15:19:13.026 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:13.026 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:13.026 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:13.026 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:13.026 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:13.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.027 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:13.027 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:13.027 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:13.027 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:13.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.028 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-14T15:19:13.028 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 0 2024-02-14T15:19:13.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.028 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:13.028 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 3, 2024-02-14T15:19:13.029 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 232, 2024-02-14T15:19:13.029 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 480, 2024-02-14T15:19:13.029 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 273440, 2024-02-14T15:19:13.029 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2024-02-14T15:19:13.029 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-02-14T15:19:13.029 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.015981481, 2024-02-14T15:19:13.030 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.011367355, 2024-02-14T15:19:13.030 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.002043903, 2024-02-14T15:19:13.030 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:13.030 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:13.030 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 229, 2024-02-14T15:19:13.031 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.012623693, 2024-02-14T15:19:13.031 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000055125 2024-02-14T15:19:13.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.031 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:13.031 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:13.031 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:13.032 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:13.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.032 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-14T15:19:13.032 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 273440 2024-02-14T15:19:13.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.033 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:13.033 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 319, 2024-02-14T15:19:13.033 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 305, 2024-02-14T15:19:13.034 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1735712, 2024-02-14T15:19:13.034 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 182688, 2024-02-14T15:19:13.034 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 2, 2024-02-14T15:19:13.035 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2024-02-14T15:19:13.035 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.050794616, 2024-02-14T15:19:13.035 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.016570618, 2024-02-14T15:19:13.036 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.038916879, 2024-02-14T15:19:13.036 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.005355061, 2024-02-14T15:19:13.037 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:13.037 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 305, 2024-02-14T15:19:13.037 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.013578289, 2024-02-14T15:19:13.037 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000044518 2024-02-14T15:19:13.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.038 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:13.038 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:13.038 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:13.039 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:13.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.039 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1735712, 2024-02-14T15:19:13.040 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 182688 2024-02-14T15:19:13.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.040 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-02-14T15:19:13.041 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 0, 2024-02-14T15:19:13.041 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-02-14T15:19:13.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.041 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-02-14T15:19:13.041 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:13.042 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:13.042 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:13.042 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:13.042 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:13.042 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:13.042 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:13.042 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:13.043 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:13.043 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:13.043 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:13.043 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:13.043 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:13.043 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:13.044 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:13.044 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:13.044 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:13.044 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:13.044 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:13.044 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:13.045 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:13.045 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:13.045 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:13.045 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:13.045 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:13.045 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:13.046 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:13.046 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:13.046 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:13.046 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:13.046 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:13.046 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-02-14T15:19:13.046 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:13.047 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:13.047 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:13.047 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:13.047 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 148010, 2024-02-14T15:19:13.047 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 30, 2024-02-14T15:19:13.047 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:13.048 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:13.048 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-02-14T15:19:13.048 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-02-14T15:19:13.048 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:13.048 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:13.048 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:13.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:13.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 0, 2024-02-14T15:19:13.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 0, 2024-02-14T15:19:13.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:13.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:13.049 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:13.050 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-02-14T15:19:13.050 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 928, 2024-02-14T15:19:13.050 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 2, 2024-02-14T15:19:13.050 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:13.050 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-02-14T15:19:13.050 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:13.051 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-02-14T15:19:13.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:13.051 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:13.051 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-02-14T15:19:13.051 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-02-14T15:19:13.051 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-02-14T15:19:13.052 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 318, 2024-02-14T15:19:13.052 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1688190, 2024-02-14T15:19:13.052 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:13.052 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 318, 2024-02-14T15:19:13.052 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-02-14T15:19:13.052 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-02-14T15:19:13.053 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 318, 2024-02-14T15:19:13.053 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1688190, 2024-02-14T15:19:13.053 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-02-14T15:19:13.053 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:13.053 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:13.053 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:13.054 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:13.054 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:13.054 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:19:13.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:12 smithi076 ceph-mon[37057]: pgmap v499: 129 pgs: 129 active+clean; 847 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 178 op/s 2024-02-14T15:19:13.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:12 smithi017 ceph-mon[32549]: pgmap v499: 129 pgs: 129 active+clean; 847 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 178 op/s 2024-02-14T15:19:13.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:12 smithi050 ceph-mon[36512]: pgmap v499: 129 pgs: 129 active+clean; 847 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 178 op/s 2024-02-14T15:19:13.750 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-14T15:19:13.750 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-14T15:19:13.750 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-02-14T15:19:13.750 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-02-14T15:19:13.751 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-02-14T15:19:13.751 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-02-14T15:19:13.751 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-02-14T15:19:13.751 DEBUG:tasks.check_counter:Getting stats from g 2024-02-14T15:19:13.751 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-14T15:19:14.144 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:19:15.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:14 smithi076 ceph-mon[37057]: pgmap v500: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-02-14T15:19:15.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:14 smithi017 ceph-mon[32549]: pgmap v500: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-02-14T15:19:15.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:14 smithi050 ceph-mon[36512]: pgmap v500: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-02-14T15:19:16.110 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:19:16.110 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 4, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 4, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 3072, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 448, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 0, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.001630782, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.000042804, 2024-02-14T15:19:16.111 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.000687683, 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3, 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000766457, 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000255485 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:16.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 448 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 3, 2024-02-14T15:19:16.113 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 230, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 480, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 273088, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.015677498, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.011089522, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.002136230, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:16.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 227, 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.017831688, 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000078553 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-14T15:19:16.115 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 273088 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 311, 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 299, 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1731552, 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 179744, 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 2, 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2024-02-14T15:19:16.116 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.050093611, 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.016092234, 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.039044533, 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.005169243, 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 298, 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.010734110, 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000036020 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1731552, 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 179744 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 0, 2024-02-14T15:19:16.118 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:16.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:16.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:16.121 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:16.122 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 148748, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 32, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:16.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 928, 2024-02-14T15:19:16.124 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 2, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-02-14T15:19:16.125 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 314, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1687670, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 314, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 314, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1687670, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:16.126 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:16.127 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:16.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:16.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:16.127 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:19:16.792 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-14T15:19:16.792 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-14T15:19:16.792 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-02-14T15:19:16.792 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-02-14T15:19:16.792 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-02-14T15:19:16.792 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-02-14T15:19:16.792 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-02-14T15:19:16.792 DEBUG:tasks.check_counter:Getting stats from j 2024-02-14T15:19:16.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-14T15:19:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:16 smithi017 ceph-mon[32549]: pgmap v501: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-02-14T15:19:17.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:16 smithi017 ceph-mon[32549]: osdmap e118: 12 total, 12 up, 12 in 2024-02-14T15:19:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T15:19:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:16 smithi050 ceph-mon[36512]: pgmap v501: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-02-14T15:19:17.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:16 smithi050 ceph-mon[36512]: osdmap e118: 12 total, 12 up, 12 in 2024-02-14T15:19:17.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:16 smithi076 ceph-mon[37057]: pgmap v501: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.4 MiB/s wr, 136 op/s 2024-02-14T15:19:17.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:16 smithi076 ceph-mon[37057]: osdmap e118: 12 total, 12 up, 12 in 2024-02-14T15:19:18.743 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-02-14T15:19:18.743 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:18.743 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 4, 2024-02-14T15:19:18.743 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 4, 2024-02-14T15:19:18.743 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 3072, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 448, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 0, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.001601435, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.000043865, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.000619275, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:18.744 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3, 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000718719, 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000239573 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.745 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 448 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 317, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 305, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1733184, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 183488, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 2, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2024-02-14T15:19:18.746 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.049683492, 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.016425674, 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.037820141, 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.005682459, 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 304, 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.010639050, 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000034996 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.747 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 1733184, 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 183488 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:18.748 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 3, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 234, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 480, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 273792, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.015620690, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.010705524, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.002282757, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:18.749 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 231, 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.016323847, 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000070666 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:18.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 273792 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 0, 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:18.753 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:18.754 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 148961, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 29, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:18.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 928, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 2, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:18.757 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 320, 2024-02-14T15:19:18.758 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 1688450, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 320, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 320, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 1688450, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2024-02-14T15:19:18.759 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2024-02-14T15:19:18.760 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:18.760 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-02-14T15:19:18.760 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-02-14T15:19:19.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:18 smithi017 ceph-mon[32549]: pgmap v503: 129 pgs: 129 active+clean; 844 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 4.1 MiB/s wr, 136 op/s 2024-02-14T15:19:19.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:18 smithi050 ceph-mon[36512]: pgmap v503: 129 pgs: 129 active+clean; 844 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 4.1 MiB/s wr, 136 op/s 2024-02-14T15:19:19.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:18 smithi076 ceph-mon[37057]: pgmap v503: 129 pgs: 129 active+clean; 844 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 4.1 MiB/s wr, 136 op/s 2024-02-14T15:19:19.389 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-14T15:19:19.389 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-14T15:19:19.389 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-02-14T15:19:19.389 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-02-14T15:19:19.389 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-02-14T15:19:19.390 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-02-14T15:19:19.390 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-02-14T15:19:19.390 DEBUG:tasks.check_counter:Getting stats from b 2024-02-14T15:19:19.390 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-14T15:19:19.661 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T15:19:21.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:20 smithi050 ceph-mon[36512]: pgmap v504: 129 pgs: 129 active+clean; 844 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.4 MiB/s wr, 112 op/s 2024-02-14T15:19:21.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:20 smithi076 ceph-mon[37057]: pgmap v504: 129 pgs: 129 active+clean; 844 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.4 MiB/s wr, 112 op/s 2024-02-14T15:19:21.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:20 smithi017 ceph-mon[32549]: pgmap v504: 129 pgs: 129 active+clean; 844 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.4 MiB/s wr, 112 op/s 2024-02-14T15:19:21.616 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-02-14T15:19:21.616 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:21.616 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 3202, 2024-02-14T15:19:21.616 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 3661, 2024-02-14T15:19:21.616 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 2811603, 2024-02-14T15:19:21.616 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 7163852, 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 174, 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 4, 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.338942473, 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.096715951, 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.167784509, 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.061969141, 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:21.617 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3662, 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.448136502, 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000122374 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3728, 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000639814, 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000171 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.618 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1894512, 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 202624 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 409729, 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 207195, 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 193028416, 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 34878309, 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 15, 2024-02-14T15:19:21.619 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 5, 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 17.091120404, 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 6.090355808, 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 15.354666145, 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.056322743, 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 207194, 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 3.657945621, 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000017654 2024-02-14T15:19:21.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 614276, 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.074834434, 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000121 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 43456, 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 573408 2024-02-14T15:19:21.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 50954, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 51575, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 13371083, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 17029080, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 31, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 9, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 2.999894370, 2024-02-14T15:19:21.622 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 1.231019074, 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 2.192470557, 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.162206149, 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 51570, 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.689882601, 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000032768 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:21.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 49403, 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.025514243, 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000516 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 550224, 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 186240 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-02-14T15:19:21.624 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 4037, 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.582205754, 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000391926 2024-02-14T15:19:21.625 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 894, 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.098841504, 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000110560 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 44594, 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 3421, 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3421, 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 22.095335748, 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.006458735 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.627 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 41173, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 4, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 1, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 481, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 196, 2024-02-14T15:19:21.628 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 76, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 109, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 11, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 88, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 170, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 34, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 34, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 6, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 47263, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 3589, 2024-02-14T15:19:21.629 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 41171, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 95, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 72, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 1, 2024-02-14T15:19:21.630 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 170, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 2, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 22, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 60, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 670, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 211580315, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 6, 2024-02-14T15:19:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 60, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 40, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 5, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 35, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 20, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 2, 2024-02-14T15:19:21.632 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 209, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 1836, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 90, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 1301, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 735, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 161, 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 2198 2024-02-14T15:19:21.633 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 57, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 298, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 194, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 216, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 88, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 15, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-02-14T15:19:21.634 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 497, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 482, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-02-14T15:19:21.635 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 5, 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 1, 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-02-14T15:19:21.636 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 3075, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 3028, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 3028, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 47, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 104, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 75, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 74, 2024-02-14T15:19:21.637 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 74, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 1, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 1, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 1, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 13844239, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 13993304, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 0, 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-02-14T15:19:21.638 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1431, 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 4.069588401, 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002843877 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 0 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 188, 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 742, 2024-02-14T15:19:21.639 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 554, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 165, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 206, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 41, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 196, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 1289, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 1093, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 31, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 615, 2024-02-14T15:19:21.640 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 584, 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 64748, 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 215420 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 46202, 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 4, 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 44594, 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 159, 2024-02-14T15:19:21.641 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 164, 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 563, 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 2.154224835, 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003826331 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 288, 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.441186439, 2024-02-14T15:19:21.642 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001531897 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.643 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1095, 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 14.897546880, 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.013605065 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-02-14T15:19:21.644 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.645 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-02-14T15:19:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-02-14T15:19:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 16, 2024-02-14T15:19:21.648 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.002514084, 2024-02-14T15:19:21.648 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000157130 2024-02-14T15:19:21.648 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.648 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-02-14T15:19:21.648 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 20, 2024-02-14T15:19:21.648 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.025223412, 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001261170 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 16, 2024-02-14T15:19:21.649 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.159113189, 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.072444574 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2, 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.001256104, 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000628052 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-02-14T15:19:21.650 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 762, 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.637297075, 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002148683 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 126, 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.757255093, 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.006009961 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.651 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 10, 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.160522402, 2024-02-14T15:19:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.016052240 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 5, 2024-02-14T15:19:21.654 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.013315059, 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002663011 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 12, 2024-02-14T15:19:21.655 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.003234028, 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000269502 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.656 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 9, 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.023782642, 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002642515 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.657 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 497, 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.818864506, 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001647614 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.658 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 2, 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 7, 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 5, 2024-02-14T15:19:21.659 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 2, 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 443, 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 221, 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 943, 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-02-14T15:19:21.660 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:21.661 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:21.662 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:21.663 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:21.664 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 378566, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 300, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 88, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 1, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:21.665 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9744, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 204, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-02-14T15:19:21.666 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 899732, 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 9853, 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-02-14T15:19:21.667 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 4657, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 11941130, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 19, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 4657, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 4657, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 46.656837879, 2024-02-14T15:19:21.668 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.010018646 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 4657, 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 6688 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 4657, 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 122, 2024-02-14T15:19:21.669 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 4535, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 447, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 645, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 52, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 1478, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 284, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-02-14T15:19:21.670 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 1480, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 70, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 996, 2024-02-14T15:19:21.671 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-02-14T15:19:21.672 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-02-14T15:19:21.673 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 118, 2024-02-14T15:19:21.674 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 34, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 12, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 12, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 0, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 1155, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 10, 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 71 2024-02-14T15:19:21.675 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 1, 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 1 2024-02-14T15:19:21.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 16, 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 8, 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 964, 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 482, 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-02-14T15:19:21.677 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 463881, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 168123610, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 463881, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:21.678 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 463881, 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 168123610, 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 4657, 2024-02-14T15:19:21.680 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 11993632, 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 4657, 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 11993632, 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.681 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:21.682 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 4657, 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 4657, 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 4657, 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 4657, 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 482, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 52570, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 482, 2024-02-14T15:19:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 47, 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 52570, 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x55e47e05a640": { 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 7573, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 3075, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 9799000, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 3075, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:21.686 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 1431, 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 9791427, 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:21.687 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-02-14T15:19:22.319 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-02-14T15:19:22.319 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-02-14T15:19:22.319 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=57 2024-02-14T15:19:22.319 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=298 2024-02-14T15:19:22.320 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-02-14T15:19:22.320 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-02-14T15:19:22.320 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-02-14T15:19:22.320 DEBUG:tasks.check_counter:Getting stats from e 2024-02-14T15:19:22.320 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-14T15:19:22.594 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T15:19:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:22 smithi050 ceph-mon[36512]: pgmap v505: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.7 MiB/s wr, 115 op/s 2024-02-14T15:19:23.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:22 smithi076 ceph-mon[37057]: pgmap v505: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.7 MiB/s wr, 115 op/s 2024-02-14T15:19:23.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:22 smithi017 ceph-mon[32549]: pgmap v505: 129 pgs: 129 active+clean; 848 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.7 MiB/s wr, 115 op/s 2024-02-14T15:19:24.545 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-02-14T15:19:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 1886, 2024-02-14T15:19:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 1888, 2024-02-14T15:19:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 570921714, 2024-02-14T15:19:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 586610, 2024-02-14T15:19:24.546 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 8, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 4, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.371673808, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.039602796, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.280262634, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.038568040, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1886, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.043339318, 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000022979 2024-02-14T15:19:24.547 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.548 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2772, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2744, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 753240857, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 937210, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 6, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 5, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.560318093, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.057797668, 2024-02-14T15:19:24.549 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.440905432, 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.055848699, 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2744, 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.069681065, 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000025393 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.550 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1828928, 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 189872 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 1911, 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2141, 2024-02-14T15:19:24.551 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 310953332, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 1153235, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 8, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 6, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.293520751, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.052670665, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.174588646, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.049163084, 2024-02-14T15:19:24.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2138, 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.072976468, 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000034133 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.553 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 559536 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.554 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 9045, 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.279801432, 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000030934 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-02-14T15:19:24.555 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 0, 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 0, 2024-02-14T15:19:24.556 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 0, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 0, 2024-02-14T15:19:24.557 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 64919, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 29950, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 34969, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 0, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 22134, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 0, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 0, 2024-02-14T15:19:24.558 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 6, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-02-14T15:19:24.559 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-02-14T15:19:24.560 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 0, 2024-02-14T15:19:24.561 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 0, 2024-02-14T15:19:24.562 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 0, 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 0, 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 0 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 0, 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 0, 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 0, 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 0, 2024-02-14T15:19:24.563 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-02-14T15:19:24.564 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-02-14T15:19:24.565 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 0, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 284202, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-02-14T15:19:24.566 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 0, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 149, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 25, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 1048206507, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 1635771072, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 1635761909, 2024-02-14T15:19:24.567 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 284202 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-02-14T15:19:24.568 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 3521, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 64257, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 60736, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 209, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 389, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 180, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 64919, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 64963, 2024-02-14T15:19:24.569 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 44, 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 0, 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 0, 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 0, 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 118684, 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 215420 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 0, 2024-02-14T15:19:24.570 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 0, 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 0, 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 0, 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 0, 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.571 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.573 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.574 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.575 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.576 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-02-14T15:19:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.577 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.578 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.580 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.581 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.582 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-02-14T15:19:24.583 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.584 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.585 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.586 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-02-14T15:19:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 2, 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 5, 2024-02-14T15:19:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 3, 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 2, 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 0, 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 0, 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 3789, 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-02-14T15:19:24.589 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:24.590 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:24.591 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:24.592 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 1247429, 2024-02-14T15:19:24.593 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 4104, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 88, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 1, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:24.594 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9744, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 204, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 54776456, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 437162, 2024-02-14T15:19:24.595 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 6198, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 21, 2024-02-14T15:19:24.596 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 3, 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 6198, 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6198, 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 6.841830627, 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001103877 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6198, 2024-02-14T15:19:24.597 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 6199 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 6198, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 6198, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 0, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 3953, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 0, 2024-02-14T15:19:24.598 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 2243, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-02-14T15:19:24.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 1, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-02-14T15:19:24.600 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-02-14T15:19:24.601 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-02-14T15:19:24.602 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 118, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 29, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 11, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 11, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 0, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-02-14T15:19:24.603 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 0, 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 2, 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 0 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 0, 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-02-14T15:19:24.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 0 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 0, 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 0, 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 0, 2024-02-14T15:19:24.605 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 0, 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 6565, 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1634503380, 2024-02-14T15:19:24.606 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 6565, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 6565, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1634503380, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.607 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:24.608 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 6198, 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 1631607229, 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 6198, 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1631607229, 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.609 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:24.610 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 6198, 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 6198, 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 6198, 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 6198, 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:24.612 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 0, 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 0, 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x55a4757c6960": { 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:24.614 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 0, 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 0, 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:24.615 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-02-14T15:19:25.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:24 smithi050 ceph-mon[36512]: pgmap v506: 129 pgs: 129 active+clean; 850 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.7 MiB/s wr, 116 op/s 2024-02-14T15:19:25.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:24 smithi076 ceph-mon[37057]: pgmap v506: 129 pgs: 129 active+clean; 850 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.7 MiB/s wr, 116 op/s 2024-02-14T15:19:25.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:24 smithi017 ceph-mon[32549]: pgmap v506: 129 pgs: 129 active+clean; 850 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.7 MiB/s wr, 116 op/s 2024-02-14T15:19:25.485 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-14T15:19:25.485 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-14T15:19:25.485 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-02-14T15:19:25.485 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-02-14T15:19:25.485 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-02-14T15:19:25.485 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-02-14T15:19:25.486 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-02-14T15:19:25.486 DEBUG:tasks.check_counter:Getting stats from h 2024-02-14T15:19:25.486 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-14T15:19:25.741 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T15:19:27.135 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:26 smithi050 ceph-mon[36512]: pgmap v507: 129 pgs: 129 active+clean; 850 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.7 MiB/s wr, 116 op/s 2024-02-14T15:19:27.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:26 smithi076 ceph-mon[37057]: pgmap v507: 129 pgs: 129 active+clean; 850 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.7 MiB/s wr, 116 op/s 2024-02-14T15:19:27.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:26 smithi017 ceph-mon[32549]: pgmap v507: 129 pgs: 129 active+clean; 850 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.7 MiB/s wr, 116 op/s 2024-02-14T15:19:27.552 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-02-14T15:19:27.552 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:27.552 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-02-14T15:19:27.553 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-02-14T15:19:27.553 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-02-14T15:19:27.553 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-02-14T15:19:27.553 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-02-14T15:19:27.553 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-02-14T15:19:27.553 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.001420065, 2024-02-14T15:19:27.554 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-02-14T15:19:27.554 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000423685, 2024-02-14T15:19:27.554 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:27.554 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:27.554 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:27.554 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:27.555 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:27.555 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.555 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:27.555 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:27.555 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:27.555 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:27.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.556 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-14T15:19:27.556 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-02-14T15:19:27.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.556 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:27.556 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 72, 2024-02-14T15:19:27.557 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 71, 2024-02-14T15:19:27.557 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 159760, 2024-02-14T15:19:27.557 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 41408, 2024-02-14T15:19:27.557 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-02-14T15:19:27.557 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-02-14T15:19:27.557 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.008987821, 2024-02-14T15:19:27.557 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.002417751, 2024-02-14T15:19:27.558 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.004863738, 2024-02-14T15:19:27.558 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.001655825, 2024-02-14T15:19:27.558 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:27.558 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 70, 2024-02-14T15:19:27.558 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.002929390, 2024-02-14T15:19:27.558 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000041848 2024-02-14T15:19:27.559 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.559 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:27.559 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:27.559 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:27.559 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:27.559 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 159760, 2024-02-14T15:19:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 41408 2024-02-14T15:19:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 5, 2024-02-14T15:19:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 56, 2024-02-14T15:19:27.560 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3232, 2024-02-14T15:19:27.561 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 87312, 2024-02-14T15:19:27.561 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 3, 2024-02-14T15:19:27.561 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-02-14T15:19:27.561 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.004370125, 2024-02-14T15:19:27.561 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.001834176, 2024-02-14T15:19:27.561 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.002071401, 2024-02-14T15:19:27.562 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:27.562 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:27.562 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 55, 2024-02-14T15:19:27.562 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.002734526, 2024-02-14T15:19:27.562 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000049718 2024-02-14T15:19:27.562 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.563 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:27.563 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:27.563 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:27.563 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:27.563 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.563 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-02-14T15:19:27.564 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 87312 2024-02-14T15:19:27.564 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.564 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-02-14T15:19:27.564 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-02-14T15:19:27.564 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-02-14T15:19:27.564 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.564 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-02-14T15:19:27.565 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:27.565 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:27.565 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:27.565 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:27.565 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:27.565 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:27.567 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:27.567 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:27.567 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:27.567 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:27.567 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:27.567 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:27.568 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:27.568 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:27.568 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:27.568 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:27.568 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:27.568 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:27.569 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-02-14T15:19:27.570 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:27.570 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:27.570 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:27.570 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:27.570 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 149871, 2024-02-14T15:19:27.570 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 29, 2024-02-14T15:19:27.570 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:27.571 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:27.571 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-02-14T15:19:27.571 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-02-14T15:19:27.571 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:27.571 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:27.571 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:27.572 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:27.572 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-02-14T15:19:27.572 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-02-14T15:19:27.572 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:27.572 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:27.572 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:27.573 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-02-14T15:19:27.573 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-02-14T15:19:27.573 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-02-14T15:19:27.573 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:27.573 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-02-14T15:19:27.573 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:27.573 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-02-14T15:19:27.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:27.574 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:27.574 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:27.574 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-02-14T15:19:27.574 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:27.574 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 73, 2024-02-14T15:19:27.574 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 149617, 2024-02-14T15:19:27.575 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:27.575 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 73, 2024-02-14T15:19:27.575 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:27.575 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:27.575 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 73, 2024-02-14T15:19:27.575 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 149617, 2024-02-14T15:19:27.576 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:27.576 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:27.576 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:27.576 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:27.576 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:27.576 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:27.577 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-02-14T15:19:28.191 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2024-02-14T15:19:28.191 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2024-02-14T15:19:28.191 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.h 2024-02-14T15:19:28.192 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.h 2024-02-14T15:19:28.192 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-02-14T15:19:28.192 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-02-14T15:19:28.192 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-02-14T15:19:28.192 DEBUG:tasks.check_counter:Getting stats from k 2024-02-14T15:19:28.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-14T15:19:28.449 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T15:19:29.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:28 smithi050 ceph-mon[36512]: pgmap v508: 129 pgs: 129 active+clean; 861 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 5.1 MiB/s wr, 151 op/s 2024-02-14T15:19:29.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:28 smithi076 ceph-mon[37057]: pgmap v508: 129 pgs: 129 active+clean; 861 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 5.1 MiB/s wr, 151 op/s 2024-02-14T15:19:29.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:28 smithi017 ceph-mon[32549]: pgmap v508: 129 pgs: 129 active+clean; 861 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 5.1 MiB/s wr, 151 op/s 2024-02-14T15:19:30.485 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-02-14T15:19:30.485 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:30.485 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 4, 2024-02-14T15:19:30.485 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 4, 2024-02-14T15:19:30.485 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3072, 2024-02-14T15:19:30.485 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 448, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.001605247, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000040748, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000705421, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3, 2024-02-14T15:19:30.486 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000783667, 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000261222 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-02-14T15:19:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 448 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 320, 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 308, 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1734000, 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 185360, 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-02-14T15:19:30.488 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.053040821, 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.016023006, 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.040565762, 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.006426609, 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 307, 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.012503655, 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000040728 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.489 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:30.490 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:30.490 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:30.490 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:30.490 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.490 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1734000, 2024-02-14T15:19:30.490 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 185360 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 3, 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 236, 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 480, 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 274144, 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 6, 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-02-14T15:19:30.491 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.013940510, 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.009047749, 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.002352044, 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 233, 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.019372933, 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000083145 2024-02-14T15:19:30.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 274144 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.493 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:30.494 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:30.495 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:30.496 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:30.497 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 147970, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 29, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-02-14T15:19:30.498 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:30.499 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-02-14T15:19:30.500 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 323, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1688840, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 323, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-02-14T15:19:30.501 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 323, 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1688840, 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-02-14T15:19:30.502 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-02-14T15:19:31.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:30 smithi050 ceph-mon[36512]: pgmap v509: 129 pgs: 129 active+clean; 861 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.7 MiB/s wr, 73 op/s 2024-02-14T15:19:31.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:30 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:19:31.221 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-02-14T15:19:31.221 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-02-14T15:19:31.221 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-02-14T15:19:31.221 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-02-14T15:19:31.222 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-02-14T15:19:31.222 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-02-14T15:19:31.222 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-02-14T15:19:31.222 DEBUG:tasks.check_counter:Getting stats from c 2024-02-14T15:19:31.222 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-14T15:19:31.248 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:30 smithi076 ceph-mon[37057]: pgmap v509: 129 pgs: 129 active+clean; 861 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.7 MiB/s wr, 73 op/s 2024-02-14T15:19:31.249 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:30 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:19:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:30 smithi017 ceph-mon[32549]: pgmap v509: 129 pgs: 129 active+clean; 861 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 2.7 MiB/s wr, 73 op/s 2024-02-14T15:19:31.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:30 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:19:31.694 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T15:19:32.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:32 smithi076 ceph-mon[37057]: pgmap v510: 129 pgs: 129 active+clean; 855 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.4 MiB/s wr, 103 op/s 2024-02-14T15:19:33.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:32 smithi050 ceph-mon[36512]: pgmap v510: 129 pgs: 129 active+clean; 855 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.4 MiB/s wr, 103 op/s 2024-02-14T15:19:33.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:32 smithi017 ceph-mon[32549]: pgmap v510: 129 pgs: 129 active+clean; 855 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.4 MiB/s wr, 103 op/s 2024-02-14T15:19:34.560 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-02-14T15:19:34.560 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:34.560 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 74351, 2024-02-14T15:19:34.560 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 74520, 2024-02-14T15:19:34.561 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 20423232, 2024-02-14T15:19:34.561 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 494554506, 2024-02-14T15:19:34.561 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 14, 2024-02-14T15:19:34.561 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-02-14T15:19:34.561 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 6.577195649, 2024-02-14T15:19:34.561 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 2.458700528, 2024-02-14T15:19:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 1.378383838, 2024-02-14T15:19:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 2.230525414, 2024-02-14T15:19:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 74519, 2024-02-14T15:19:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 6.300100137, 2024-02-14T15:19:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000084543 2024-02-14T15:19:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.563 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:34.563 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 15, 2024-02-14T15:19:34.563 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000001877, 2024-02-14T15:19:34.563 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000125 2024-02-14T15:19:34.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.563 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1838864, 2024-02-14T15:19:34.564 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 656880 2024-02-14T15:19:34.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.564 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:34.564 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 698963, 2024-02-14T15:19:34.564 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 996743, 2024-02-14T15:19:34.564 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 199028803, 2024-02-14T15:19:34.564 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 1554368644, 2024-02-14T15:19:34.565 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 19, 2024-02-14T15:19:34.565 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-02-14T15:19:34.565 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 35.287009964, 2024-02-14T15:19:34.565 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 18.040707377, 2024-02-14T15:19:34.565 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 20.697364307, 2024-02-14T15:19:34.565 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 3.369621446, 2024-02-14T15:19:34.566 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:34.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 996741, 2024-02-14T15:19:34.566 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 32.736154582, 2024-02-14T15:19:34.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000032843 2024-02-14T15:19:34.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.566 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:34.567 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 834558, 2024-02-14T15:19:34.567 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.150129095, 2024-02-14T15:19:34.567 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000179 2024-02-14T15:19:34.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.567 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 7616, 2024-02-14T15:19:34.567 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 144000 2024-02-14T15:19:34.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.568 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:34.568 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 470023, 2024-02-14T15:19:34.568 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 871566, 2024-02-14T15:19:34.568 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 88111937, 2024-02-14T15:19:34.568 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 796796788, 2024-02-14T15:19:34.568 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 12, 2024-02-14T15:19:34.568 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 6, 2024-02-14T15:19:34.569 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 29.983328739, 2024-02-14T15:19:34.569 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 13.717157119, 2024-02-14T15:19:34.569 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 15.177338140, 2024-02-14T15:19:34.569 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 2.205040106, 2024-02-14T15:19:34.569 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:34.569 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 871564, 2024-02-14T15:19:34.570 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 22.719660538, 2024-02-14T15:19:34.570 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000026067 2024-02-14T15:19:34.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.570 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:34.570 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1205435, 2024-02-14T15:19:34.570 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.151415727, 2024-02-14T15:19:34.570 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000125 2024-02-14T15:19:34.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.571 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-14T15:19:34.571 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-02-14T15:19:34.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.571 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-02-14T15:19:34.571 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-02-14T15:19:34.571 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-02-14T15:19:34.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.572 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-02-14T15:19:34.572 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:34.572 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:34.572 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 187318, 2024-02-14T15:19:34.572 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 158.079647925, 2024-02-14T15:19:34.573 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000843910 2024-02-14T15:19:34.573 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.573 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-02-14T15:19:34.573 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:34.573 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:34.573 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 102346, 2024-02-14T15:19:34.574 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 23.521855345, 2024-02-14T15:19:34.574 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000229826 2024-02-14T15:19:34.574 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.574 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-02-14T15:19:34.574 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 471369, 2024-02-14T15:19:34.574 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 471034, 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 471034, 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1608.038071240, 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003413847 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 335, 2024-02-14T15:19:34.575 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-02-14T15:19:34.576 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-02-14T15:19:34.576 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 412, 2024-02-14T15:19:34.576 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 29, 2024-02-14T15:19:34.576 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 28, 2024-02-14T15:19:34.576 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 7218, 2024-02-14T15:19:34.576 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 2261, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 4691, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 266, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 3876, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 60, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 742, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 742, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 10, 2024-02-14T15:19:34.577 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 685611, 2024-02-14T15:19:34.578 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 424861, 2024-02-14T15:19:34.578 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 333, 2024-02-14T15:19:34.578 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 16, 2024-02-14T15:19:34.578 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-02-14T15:19:34.578 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-02-14T15:19:34.578 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 4284, 2024-02-14T15:19:34.579 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-02-14T15:19:34.579 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-02-14T15:19:34.579 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 3, 2024-02-14T15:19:34.579 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 45, 2024-02-14T15:19:34.579 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 1, 2024-02-14T15:19:34.579 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 170, 2024-02-14T15:19:34.579 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-02-14T15:19:34.580 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 96, 2024-02-14T15:19:34.580 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-02-14T15:19:34.580 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 66, 2024-02-14T15:19:34.580 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 422432, 2024-02-14T15:19:34.580 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 2, 2024-02-14T15:19:34.580 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 96, 2024-02-14T15:19:34.580 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-02-14T15:19:34.581 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-02-14T15:19:34.581 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-02-14T15:19:34.581 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 12, 2024-02-14T15:19:34.581 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-02-14T15:19:34.581 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 12, 2024-02-14T15:19:34.581 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 84, 2024-02-14T15:19:34.581 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 3, 2024-02-14T15:19:34.582 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 22381, 2024-02-14T15:19:34.582 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 224219, 2024-02-14T15:19:34.582 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-02-14T15:19:34.582 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 4018, 2024-02-14T15:19:34.582 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-14T15:19:34.582 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 113460, 2024-02-14T15:19:34.582 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 54357, 2024-02-14T15:19:34.583 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 17900, 2024-02-14T15:19:34.583 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 327937 2024-02-14T15:19:34.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.583 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-02-14T15:19:34.583 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 180, 2024-02-14T15:19:34.583 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 38, 2024-02-14T15:19:34.584 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 25, 2024-02-14T15:19:34.584 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 30, 2024-02-14T15:19:34.584 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 414, 2024-02-14T15:19:34.584 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 173, 2024-02-14T15:19:34.584 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-02-14T15:19:34.584 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-02-14T15:19:34.584 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 61443, 2024-02-14T15:19:34.585 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 61270, 2024-02-14T15:19:34.585 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-02-14T15:19:34.585 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-02-14T15:19:34.585 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-02-14T15:19:34.585 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-02-14T15:19:34.585 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-02-14T15:19:34.585 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-02-14T15:19:34.586 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-02-14T15:19:34.586 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-02-14T15:19:34.586 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 3, 2024-02-14T15:19:34.586 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-02-14T15:19:34.586 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 58, 2024-02-14T15:19:34.586 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-02-14T15:19:34.586 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-02-14T15:19:34.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.587 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-02-14T15:19:34.587 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-02-14T15:19:34.587 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 285491, 2024-02-14T15:19:34.587 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 207137, 2024-02-14T15:19:34.587 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 196641, 2024-02-14T15:19:34.587 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 88850, 2024-02-14T15:19:34.588 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-02-14T15:19:34.588 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 10496, 2024-02-14T15:19:34.588 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 467, 2024-02-14T15:19:34.588 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 339, 2024-02-14T15:19:34.588 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 316, 2024-02-14T15:19:34.588 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 151, 2024-02-14T15:19:34.589 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 8, 2024-02-14T15:19:34.589 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-02-14T15:19:34.589 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 23, 2024-02-14T15:19:34.589 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 1048206507, 2024-02-14T15:19:34.589 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 1642651996, 2024-02-14T15:19:34.589 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-02-14T15:19:34.589 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-02-14T15:19:34.590 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 96846, 2024-02-14T15:19:34.590 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 308.988575027, 2024-02-14T15:19:34.590 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003190514 2024-02-14T15:19:34.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.590 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-02-14T15:19:34.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.590 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-02-14T15:19:34.591 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 3108, 2024-02-14T15:19:34.591 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 64406, 2024-02-14T15:19:34.591 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 61298, 2024-02-14T15:19:34.591 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 205, 2024-02-14T15:19:34.591 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 394, 2024-02-14T15:19:34.591 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 189, 2024-02-14T15:19:34.591 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 7266, 2024-02-14T15:19:34.592 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 131630, 2024-02-14T15:19:34.592 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 124364, 2024-02-14T15:19:34.592 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 739, 2024-02-14T15:19:34.592 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 67650, 2024-02-14T15:19:34.592 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 66911, 2024-02-14T15:19:34.592 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 94252, 2024-02-14T15:19:34.593 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 207228 2024-02-14T15:19:34.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.593 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-02-14T15:19:34.593 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 714581, 2024-02-14T15:19:34.593 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 342, 2024-02-14T15:19:34.593 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 471369, 2024-02-14T15:19:34.593 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 8476, 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 681, 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 62047, 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 219.786239392, 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003542254 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-02-14T15:19:34.594 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1033, 2024-02-14T15:19:34.595 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 16.411470499, 2024-02-14T15:19:34.595 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.015887193 2024-02-14T15:19:34.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.595 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-02-14T15:19:34.595 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.595 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.595 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.596 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-02-14T15:19:34.596 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.596 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.596 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.597 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-02-14T15:19:34.597 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 145281, 2024-02-14T15:19:34.597 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 400.985091373, 2024-02-14T15:19:34.597 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002760065 2024-02-14T15:19:34.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.597 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-02-14T15:19:34.597 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.599 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-02-14T15:19:34.599 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.599 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.599 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.599 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-02-14T15:19:34.599 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.600 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.600 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.600 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-02-14T15:19:34.600 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.600 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.600 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.601 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-02-14T15:19:34.601 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.601 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.601 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.602 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-02-14T15:19:34.602 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 99, 2024-02-14T15:19:34.602 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.915590889, 2024-02-14T15:19:34.602 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.009248392 2024-02-14T15:19:34.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.602 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-02-14T15:19:34.602 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.603 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.603 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.603 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-02-14T15:19:34.603 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.603 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.603 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.604 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-02-14T15:19:34.604 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 9, 2024-02-14T15:19:34.604 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.019473880, 2024-02-14T15:19:34.604 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002163764 2024-02-14T15:19:34.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.604 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-02-14T15:19:34.605 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 186215, 2024-02-14T15:19:34.605 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 660.986275743, 2024-02-14T15:19:34.605 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003549586 2024-02-14T15:19:34.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.605 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-02-14T15:19:34.605 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 12860, 2024-02-14T15:19:34.605 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 49.656881106, 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003861343 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.606 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.002129821, 2024-02-14T15:19:34.607 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002129821 2024-02-14T15:19:34.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.608 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-02-14T15:19:34.608 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.608 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.608 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.608 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-02-14T15:19:34.609 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.609 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.609 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.609 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-02-14T15:19:34.609 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 66, 2024-02-14T15:19:34.609 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 2.034210500, 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.030821371 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.610 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1980, 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 5.736097355, 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002897018 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.611 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.612 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-02-14T15:19:34.612 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.612 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.612 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.612 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-02-14T15:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-02-14T15:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 61443, 2024-02-14T15:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 251.504610682, 2024-02-14T15:19:34.614 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004093299 2024-02-14T15:19:34.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.614 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-02-14T15:19:34.614 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-02-14T15:19:34.614 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-02-14T15:19:34.614 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.614 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 5, 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 3, 2024-02-14T15:19:34.615 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-02-14T15:19:34.616 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-02-14T15:19:34.616 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 55897, 2024-02-14T15:19:34.616 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 27948, 2024-02-14T15:19:34.616 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 904, 2024-02-14T15:19:34.616 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-02-14T15:19:34.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.616 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:34.617 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:34.618 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:34.618 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:34.618 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:34.618 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:34.618 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:34.618 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:34.618 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:34.619 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:34.620 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:34.620 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:34.620 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:34.620 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:34.620 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:34.620 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:34.620 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:34.621 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:34.621 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-02-14T15:19:34.621 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:34.621 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:34.621 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:34.621 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:34.621 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 352981, 2024-02-14T15:19:34.622 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 1636, 2024-02-14T15:19:34.622 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:34.622 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:34.622 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-02-14T15:19:34.622 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-02-14T15:19:34.622 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:34.622 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 9744, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 204, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:34.623 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-02-14T15:19:34.624 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 13377176, 2024-02-14T15:19:34.624 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 150451, 2024-02-14T15:19:34.624 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:34.624 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-02-14T15:19:34.624 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:34.624 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-02-14T15:19:34.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-02-14T15:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-02-14T15:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-02-14T15:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 225655, 2024-02-14T15:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 1654095398, 2024-02-14T15:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 28, 2024-02-14T15:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 225655, 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 225655, 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 902.933600389, 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004001389 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-02-14T15:19:34.626 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 225655, 2024-02-14T15:19:34.627 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 228135 2024-02-14T15:19:34.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.627 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 225655, 2024-02-14T15:19:34.627 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 1714, 2024-02-14T15:19:34.627 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 223941, 2024-02-14T15:19:34.627 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-02-14T15:19:34.627 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 140, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 133, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 1618, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 98464, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 723, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-02-14T15:19:34.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 29, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 123343, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 96, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 210, 2024-02-14T15:19:34.629 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-02-14T15:19:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-02-14T15:19:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-02-14T15:19:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-02-14T15:19:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-02-14T15:19:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-02-14T15:19:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-02-14T15:19:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-02-14T15:19:34.631 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-02-14T15:19:34.631 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-02-14T15:19:34.631 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-02-14T15:19:34.631 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-02-14T15:19:34.631 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-02-14T15:19:34.631 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-02-14T15:19:34.631 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-02-14T15:19:34.632 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-02-14T15:19:34.632 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-02-14T15:19:34.632 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-02-14T15:19:34.632 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-02-14T15:19:34.632 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-02-14T15:19:34.632 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-02-14T15:19:34.632 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 118, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 29, 2024-02-14T15:19:34.633 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 12, 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 2465, 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 914 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.634 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 1 2024-02-14T15:19:34.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 16, 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 8, 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 122540, 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 61270, 2024-02-14T15:19:34.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 8088, 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 1243333, 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 205421559, 2024-02-14T15:19:34.637 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 1243333, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 1243291, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 205413471, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.638 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.639 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.639 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-02-14T15:19:34.639 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:34.640 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:34.640 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:34.640 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:34.640 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:34.640 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:34.640 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:34.640 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 225655, 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 1660782380, 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 225655, 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1660782380, 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.641 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:34.642 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 225655, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 225655, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 225655, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 225655, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.643 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 61270, 2024-02-14T15:19:34.644 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 6686982, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 61270, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 1618, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 6686982, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.645 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x564dbfc60640": { 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 1377820, 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:34.646 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 285491, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1638457692, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 285491, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 96846, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1637079872, 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:34.647 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:34.648 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:34.648 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:34.648 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.648 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:34.648 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-02-14T15:19:35.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:34 smithi050 ceph-mon[36512]: pgmap v511: 129 pgs: 129 active+clean; 845 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.2 MiB/s wr, 114 op/s 2024-02-14T15:19:35.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:34 smithi076 ceph-mon[37057]: pgmap v511: 129 pgs: 129 active+clean; 845 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.2 MiB/s wr, 114 op/s 2024-02-14T15:19:35.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:34 smithi017 ceph-mon[32549]: pgmap v511: 129 pgs: 129 active+clean; 845 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.2 MiB/s wr, 114 op/s 2024-02-14T15:19:35.689 INFO:tasks.check_counter:Daemon mds.c mds.exported=3 2024-02-14T15:19:35.689 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-02-14T15:19:35.689 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=180 2024-02-14T15:19:35.689 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=38 2024-02-14T15:19:35.689 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=2 2024-02-14T15:19:35.689 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-02-14T15:19:35.689 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-02-14T15:19:35.689 DEBUG:tasks.check_counter:Getting stats from f 2024-02-14T15:19:35.690 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-14T15:19:35.909 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:35 smithi076 ceph-mon[37057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:19:35.940 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T15:19:36.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:35 smithi050 ceph-mon[36512]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:19:36.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:35 smithi017 ceph-mon[32549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-14T15:19:37.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:36 smithi050 ceph-mon[36512]: pgmap v512: 129 pgs: 129 active+clean; 845 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 2.8 MiB/s wr, 101 op/s 2024-02-14T15:19:37.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:36 smithi076 ceph-mon[37057]: pgmap v512: 129 pgs: 129 active+clean; 845 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 2.8 MiB/s wr, 101 op/s 2024-02-14T15:19:37.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:36 smithi017 ceph-mon[32549]: pgmap v512: 129 pgs: 129 active+clean; 845 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 2.8 MiB/s wr, 101 op/s 2024-02-14T15:19:38.442 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-02-14T15:19:38.443 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:38.443 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 3246, 2024-02-14T15:19:38.443 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 3248, 2024-02-14T15:19:38.443 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1985508, 2024-02-14T15:19:38.443 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 1009513, 2024-02-14T15:19:38.443 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 6, 2024-02-14T15:19:38.443 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 2, 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.203967961, 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.062163639, 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.050987942, 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.076559211, 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3246, 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.081226244, 2024-02-14T15:19:38.444 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000025023 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-02-14T15:19:38.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 9, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 240, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 4108, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 654588, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012282956, 2024-02-14T15:19:38.446 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.007276728, 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.003002000, 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000085727, 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 238, 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.016063571, 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000067493 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.447 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 653904 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:38.448 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 371, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 343, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1835164, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 191607, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 4, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 2, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.050954510, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.015569817, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.039146244, 2024-02-14T15:19:38.449 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.006205211, 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 342, 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.013508127, 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000039497 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.450 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1834896, 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 191296 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-02-14T15:19:38.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5475, 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.119547860, 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000021835 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.452 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-02-14T15:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 0, 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 0, 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-02-14T15:19:38.454 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 0, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 162, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 0, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 162, 2024-02-14T15:19:38.455 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 162, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 0, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 0, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 3, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 0, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 0, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-02-14T15:19:38.456 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-02-14T15:19:38.457 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-02-14T15:19:38.458 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 0, 2024-02-14T15:19:38.459 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 0, 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 0 2024-02-14T15:19:38.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 0, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-02-14T15:19:38.461 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-02-14T15:19:38.462 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-02-14T15:19:38.463 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 696, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-02-14T15:19:38.464 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 5263861, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 5261300, 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-02-14T15:19:38.465 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 696 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 39, 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 160, 2024-02-14T15:19:38.466 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 121, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 13, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 13, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 0, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 162, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 162, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 0, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 0, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 0, 2024-02-14T15:19:38.467 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 49384, 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 215420 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 0, 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 0, 2024-02-14T15:19:38.468 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 0, 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.469 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-02-14T15:19:38.470 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.471 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.473 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-02-14T15:19:38.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.475 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.475 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.475 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-02-14T15:19:38.475 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.475 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.476 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.476 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-02-14T15:19:38.476 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.476 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.476 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-02-14T15:19:38.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.478 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.481 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-02-14T15:19:38.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.484 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.485 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 4, 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 2, 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 3, 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 0, 2024-02-14T15:19:38.487 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 0, 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 5048, 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:38.488 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:38.489 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:38.490 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:38.491 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 183321, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 82, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:38.492 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 9744, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 204, 2024-02-14T15:19:38.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 226877, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 1536, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:38.494 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 3249, 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 21, 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 3, 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 3249, 2024-02-14T15:19:38.495 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3249, 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1.742944901, 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000536455 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3249, 2024-02-14T15:19:38.496 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3250 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 3249, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 3249, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 0, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 2090, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 0, 2024-02-14T15:19:38.497 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 1157, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-02-14T15:19:38.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 1, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-02-14T15:19:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-02-14T15:19:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-02-14T15:19:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 118, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 29, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 4, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 6, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 2, 2024-02-14T15:19:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 0, 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 2, 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 0 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 0, 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-02-14T15:19:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-02-14T15:19:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 3622, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 3476830, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 3622, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3622, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 3476830, 2024-02-14T15:19:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 3249, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 1069562, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3249, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1069562, 2024-02-14T15:19:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-02-14T15:19:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 3249, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 3249, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3249, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 3249, 2024-02-14T15:19:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-02-14T15:19:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x5634f7aa8960": { 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-02-14T15:19:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-02-14T15:19:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:38.515 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-02-14T15:19:39.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:38 smithi050 ceph-mon[36512]: pgmap v513: 129 pgs: 129 active+clean; 849 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.1 MiB/s wr, 119 op/s 2024-02-14T15:19:39.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:38 smithi076 ceph-mon[37057]: pgmap v513: 129 pgs: 129 active+clean; 849 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.1 MiB/s wr, 119 op/s 2024-02-14T15:19:39.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:38 smithi017 ceph-mon[32549]: pgmap v513: 129 pgs: 129 active+clean; 849 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 3.1 MiB/s wr, 119 op/s 2024-02-14T15:19:39.453 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-02-14T15:19:39.453 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-02-14T15:19:39.453 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-02-14T15:19:39.453 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-02-14T15:19:39.453 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-02-14T15:19:39.453 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-02-14T15:19:39.454 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-02-14T15:19:39.454 DEBUG:tasks.check_counter:Getting stats from i 2024-02-14T15:19:39.454 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-14T15:19:39.748 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T15:19:41.137 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:40 smithi076 ceph-mon[37057]: pgmap v514: 129 pgs: 129 active+clean; 849 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-02-14T15:19:41.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:40 smithi050 ceph-mon[36512]: pgmap v514: 129 pgs: 129 active+clean; 849 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-02-14T15:19:41.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:40 smithi017 ceph-mon[32549]: pgmap v514: 129 pgs: 129 active+clean; 849 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-02-14T15:19:41.525 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-02-14T15:19:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 2415, 2024-02-14T15:19:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 1896, 2024-02-14T15:19:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 698618, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 1862071, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 12, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.166802086, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.062221414, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.108812418, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.013812294, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1894, 2024-02-14T15:19:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.090982751, 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000048037 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3573, 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000638414, 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000178 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 4128, 2024-02-14T15:19:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 18416 2024-02-14T15:19:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 440416, 2024-02-14T15:19:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 242452, 2024-02-14T15:19:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 198534141, 2024-02-14T15:19:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 37296280, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 26, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 17.145842398, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 7.586647577, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 13.495969944, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.002809834, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 242449, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3.036551998, 2024-02-14T15:19:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000012524 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 640368, 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.081036845, 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000126 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 133776 2024-02-14T15:19:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 657, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 820, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1928620, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 751978, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 9, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 6, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.076167250, 2024-02-14T15:19:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.026303355, 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.046996808, 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.012109450, 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 820, 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.053156976, 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000064825 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 14, 2024-02-14T15:19:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000002168, 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000154 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1849792, 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 632752 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 524, 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.276525074, 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000527719 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 383, 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.018141743, 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000047367 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 41358, 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 637, 2024-02-14T15:19:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 637, 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 4.458690611, 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.006999514 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 40721, 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-02-14T15:19:41.537 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 11, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 172, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 66, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 106, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 78, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 3, 2024-02-14T15:19:41.538 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 4, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 4, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 42561, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 986, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 40717, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 106, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 44, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-02-14T15:19:41.539 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 1, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 23, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 66, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 422432, 2024-02-14T15:19:41.540 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 2, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-02-14T15:19:41.541 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 12, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 544, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 20, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 106, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 74, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 54, 2024-02-14T15:19:41.542 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 648 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 14, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 370, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 236, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 276, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 20, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-02-14T15:19:41.543 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 121, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 121, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-02-14T15:19:41.544 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-02-14T15:19:41.545 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 706, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 706, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 2, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-02-14T15:19:41.546 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 5287265, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 216, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.854769748, 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003957267 2024-02-14T15:19:41.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 166, 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 287, 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 121, 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 148, 2024-02-14T15:19:41.548 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 191, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 43, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 172, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 293, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 121, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 4, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 139, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 135, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 58020, 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 207228 2024-02-14T15:19:41.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 42085, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 41358, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 32, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 187, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 121, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 2.590337034, 2024-02-14T15:19:41.550 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.021407744 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 4, 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.001980440, 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000495110 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-02-14T15:19:41.552 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 169, 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.239951089, 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001419828 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-02-14T15:19:41.553 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000221397, 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000110698 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-02-14T15:19:41.554 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-02-14T15:19:41.555 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.556 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.556 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.556 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 176, 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.165798175, 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000942035 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 44, 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.825147204, 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.018753345 2024-02-14T15:19:41.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.560 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.561 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.562 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.564 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.565 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 121, 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.635255272, 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.005250043 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 4, 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 2, 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 360, 2024-02-14T15:19:41.567 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 180, 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 964, 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:41.568 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:41.569 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:41.570 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-02-14T15:19:41.571 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 382377, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 281, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:41.572 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 9744, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 204, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:41.573 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 721241, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 5104, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-02-14T15:19:41.574 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 696, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 1138227, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 28, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 696, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 696, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3.419893760, 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004913640 2024-02-14T15:19:41.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 696, 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 795 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 696, 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 49, 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 647, 2024-02-14T15:19:41.576 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 10, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 49, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 265, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 46, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-02-14T15:19:41.577 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 254, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 20, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-02-14T15:19:41.578 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-02-14T15:19:41.579 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-02-14T15:19:41.580 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 118, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 29, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-02-14T15:19:41.581 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 15, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 3, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 119, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 32 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-02-14T15:19:41.582 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 16, 2024-02-14T15:19:41.583 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 8, 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 242, 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 121, 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:41.584 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 443484, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 161889577, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 443484, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 443484, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 161889577, 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:41.585 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:41.586 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 696, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 1151416, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 696, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1151416, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.587 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:41.588 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 696, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 696, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 696, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 696, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.589 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 121, 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 13189, 2024-02-14T15:19:41.590 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 121, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 49, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 13189, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.591 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x56300583a640": { 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 706, 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1092961, 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:41.592 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 706, 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 216, 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1092961, 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.593 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:41.594 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-02-14T15:19:42.340 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-02-14T15:19:42.340 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-02-14T15:19:42.341 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=14 2024-02-14T15:19:42.341 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=370 2024-02-14T15:19:42.341 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=2 2024-02-14T15:19:42.341 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-02-14T15:19:42.341 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-02-14T15:19:42.341 DEBUG:tasks.check_counter:Getting stats from l 2024-02-14T15:19:42.342 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 shell --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-14T15:19:42.583 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/mon.c/config 2024-02-14T15:19:42.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:42 smithi076 ceph-mon[37057]: pgmap v515: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-02-14T15:19:43.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:42 smithi050 ceph-mon[36512]: pgmap v515: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-02-14T15:19:43.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:42 smithi017 ceph-mon[32549]: pgmap v515: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 80 op/s 2024-02-14T15:19:44.528 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-02-14T15:19:44.528 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-02-14T15:19:44.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 107, 2024-02-14T15:19:44.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 104, 2024-02-14T15:19:44.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 633944, 2024-02-14T15:19:44.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 42246, 2024-02-14T15:19:44.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-02-14T15:19:44.529 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 1, 2024-02-14T15:19:44.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012081443, 2024-02-14T15:19:44.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.003381242, 2024-02-14T15:19:44.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.007197198, 2024-02-14T15:19:44.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.002000624, 2024-02-14T15:19:44.530 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:44.530 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 103, 2024-02-14T15:19:44.531 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.004179977, 2024-02-14T15:19:44.531 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000040582 2024-02-14T15:19:44.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.531 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:44.531 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.531 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.532 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 177600, 2024-02-14T15:19:44.532 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 26384 2024-02-14T15:19:44.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-02-14T15:19:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 348, 2024-02-14T15:19:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 379, 2024-02-14T15:19:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1604422, 2024-02-14T15:19:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 246474, 2024-02-14T15:19:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-02-14T15:19:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-02-14T15:19:44.534 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.029579857, 2024-02-14T15:19:44.534 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.009384790, 2024-02-14T15:19:44.534 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.009223348, 2024-02-14T15:19:44.534 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.008710096, 2024-02-14T15:19:44.534 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:44.534 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 377, 2024-02-14T15:19:44.535 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.063310423, 2024-02-14T15:19:44.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000167932 2024-02-14T15:19:44.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.535 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:44.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.536 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.536 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.536 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-14T15:19:44.536 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 138496 2024-02-14T15:19:44.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.537 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-02-14T15:19:44.537 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 116, 2024-02-14T15:19:44.537 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 116, 2024-02-14T15:19:44.537 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 31267, 2024-02-14T15:19:44.537 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 35295, 2024-02-14T15:19:44.537 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-02-14T15:19:44.538 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-02-14T15:19:44.538 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.010729364, 2024-02-14T15:19:44.538 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.002504765, 2024-02-14T15:19:44.538 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.003898242, 2024-02-14T15:19:44.538 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.003540868, 2024-02-14T15:19:44.538 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-02-14T15:19:44.538 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 115, 2024-02-14T15:19:44.539 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.009298606, 2024-02-14T15:19:44.539 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000080857 2024-02-14T15:19:44.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.539 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-02-14T15:19:44.539 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.539 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.540 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.540 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-02-14T15:19:44.540 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 448 2024-02-14T15:19:44.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.541 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-02-14T15:19:44.541 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-02-14T15:19:44.541 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-02-14T15:19:44.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.541 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-02-14T15:19:44.541 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:44.542 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:44.542 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 849, 2024-02-14T15:19:44.542 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.022431125, 2024-02-14T15:19:44.542 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000026420 2024-02-14T15:19:44.542 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.543 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-02-14T15:19:44.543 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-02-14T15:19:44.543 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-02-14T15:19:44.543 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.543 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.543 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.543 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.544 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-02-14T15:19:44.544 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 0, 2024-02-14T15:19:44.544 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 0, 2024-02-14T15:19:44.544 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-02-14T15:19:44.544 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.545 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.545 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.545 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-02-14T15:19:44.545 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-02-14T15:19:44.545 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-02-14T15:19:44.545 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-02-14T15:19:44.546 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 0, 2024-02-14T15:19:44.546 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-02-14T15:19:44.546 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-02-14T15:19:44.546 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 159, 2024-02-14T15:19:44.546 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 79, 2024-02-14T15:19:44.546 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 80, 2024-02-14T15:19:44.547 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-02-14T15:19:44.547 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 47, 2024-02-14T15:19:44.547 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-02-14T15:19:44.547 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 0, 2024-02-14T15:19:44.547 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 0, 2024-02-14T15:19:44.547 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 5, 2024-02-14T15:19:44.547 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 0, 2024-02-14T15:19:44.548 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 0, 2024-02-14T15:19:44.548 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-02-14T15:19:44.548 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 0, 2024-02-14T15:19:44.548 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-02-14T15:19:44.548 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-02-14T15:19:44.548 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-02-14T15:19:44.549 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-02-14T15:19:44.549 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-02-14T15:19:44.549 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-02-14T15:19:44.549 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-02-14T15:19:44.549 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-02-14T15:19:44.549 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-02-14T15:19:44.550 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-02-14T15:19:44.550 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-02-14T15:19:44.550 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-02-14T15:19:44.550 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 670, 2024-02-14T15:19:44.550 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 213719848, 2024-02-14T15:19:44.550 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 6, 2024-02-14T15:19:44.550 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-02-14T15:19:44.551 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-02-14T15:19:44.551 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-02-14T15:19:44.551 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-02-14T15:19:44.551 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-02-14T15:19:44.551 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-02-14T15:19:44.551 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-02-14T15:19:44.552 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-02-14T15:19:44.552 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-02-14T15:19:44.552 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 0, 2024-02-14T15:19:44.552 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-02-14T15:19:44.552 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-02-14T15:19:44.552 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-14T15:19:44.553 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-14T15:19:44.553 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 0, 2024-02-14T15:19:44.553 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-02-14T15:19:44.553 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-02-14T15:19:44.553 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 0 2024-02-14T15:19:44.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.553 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-02-14T15:19:44.554 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-02-14T15:19:44.554 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-02-14T15:19:44.554 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-02-14T15:19:44.554 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 0, 2024-02-14T15:19:44.554 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-02-14T15:19:44.554 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-02-14T15:19:44.554 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-02-14T15:19:44.555 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-02-14T15:19:44.555 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-02-14T15:19:44.555 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-02-14T15:19:44.555 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-02-14T15:19:44.555 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-02-14T15:19:44.555 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-02-14T15:19:44.555 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-02-14T15:19:44.556 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-02-14T15:19:44.556 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-02-14T15:19:44.556 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-02-14T15:19:44.556 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-02-14T15:19:44.556 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-02-14T15:19:44.556 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-02-14T15:19:44.557 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-02-14T15:19:44.557 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-02-14T15:19:44.557 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-02-14T15:19:44.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.557 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-02-14T15:19:44.557 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-02-14T15:19:44.557 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 0, 2024-02-14T15:19:44.558 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-02-14T15:19:44.558 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-02-14T15:19:44.558 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 983, 2024-02-14T15:19:44.558 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-02-14T15:19:44.558 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-02-14T15:19:44.558 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 0, 2024-02-14T15:19:44.559 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-02-14T15:19:44.559 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-02-14T15:19:44.559 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-02-14T15:19:44.559 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 9, 2024-02-14T15:19:44.559 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-02-14T15:19:44.559 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-02-14T15:19:44.560 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 13844239, 2024-02-14T15:19:44.560 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 14082413, 2024-02-14T15:19:44.560 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 14080069, 2024-02-14T15:19:44.560 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-02-14T15:19:44.560 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.560 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.560 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.561 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 983 2024-02-14T15:19:44.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.561 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-02-14T15:19:44.561 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 61, 2024-02-14T15:19:44.561 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 158, 2024-02-14T15:19:44.562 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 97, 2024-02-14T15:19:44.562 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 22, 2024-02-14T15:19:44.562 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 22, 2024-02-14T15:19:44.562 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 0, 2024-02-14T15:19:44.562 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 159, 2024-02-14T15:19:44.562 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 159, 2024-02-14T15:19:44.562 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 0, 2024-02-14T15:19:44.563 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 0, 2024-02-14T15:19:44.563 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 0, 2024-02-14T15:19:44.563 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-02-14T15:19:44.563 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 48028, 2024-02-14T15:19:44.563 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 215420 2024-02-14T15:19:44.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.564 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-02-14T15:19:44.564 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 0, 2024-02-14T15:19:44.564 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-02-14T15:19:44.564 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 0, 2024-02-14T15:19:44.564 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 0, 2024-02-14T15:19:44.564 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-02-14T15:19:44.565 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-02-14T15:19:44.565 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.565 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.565 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.565 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-02-14T15:19:44.565 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.566 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.566 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-02-14T15:19:44.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.566 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.566 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.567 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-02-14T15:19:44.567 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.567 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.567 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.567 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-02-14T15:19:44.568 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.568 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.568 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.568 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-02-14T15:19:44.568 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.569 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.569 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.569 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-02-14T15:19:44.569 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.569 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.569 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.570 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-02-14T15:19:44.570 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.570 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.570 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.570 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-02-14T15:19:44.571 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.571 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.571 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.571 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-02-14T15:19:44.571 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.571 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.572 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.572 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-02-14T15:19:44.572 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.572 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.572 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.573 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-02-14T15:19:44.573 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.573 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.573 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.573 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-02-14T15:19:44.574 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.574 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.574 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.574 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-02-14T15:19:44.574 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.574 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.575 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-02-14T15:19:44.576 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.576 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.576 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.576 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-02-14T15:19:44.576 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.577 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.577 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.577 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-02-14T15:19:44.577 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.577 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.577 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.578 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-02-14T15:19:44.578 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.578 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.578 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.578 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.579 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.580 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-02-14T15:19:44.580 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.580 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.580 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.581 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-02-14T15:19:44.581 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.581 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.581 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.581 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-02-14T15:19:44.581 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.582 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.582 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.582 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-02-14T15:19:44.582 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.582 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.583 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-02-14T15:19:44.584 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.584 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.584 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.584 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-02-14T15:19:44.584 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.585 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.585 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.585 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-02-14T15:19:44.585 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.585 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.585 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.587 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-02-14T15:19:44.587 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-02-14T15:19:44.587 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 2, 2024-02-14T15:19:44.587 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-02-14T15:19:44.587 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 0, 2024-02-14T15:19:44.587 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-02-14T15:19:44.588 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 0, 2024-02-14T15:19:44.588 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 0, 2024-02-14T15:19:44.588 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 2345, 2024-02-14T15:19:44.588 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-02-14T15:19:44.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.588 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-02-14T15:19:44.588 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-02-14T15:19:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-14T15:19:44.590 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-02-14T15:19:44.590 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-02-14T15:19:44.590 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-02-14T15:19:44.590 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-14T15:19:44.590 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-02-14T15:19:44.590 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-02-14T15:19:44.590 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-02-14T15:19:44.591 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-02-14T15:19:44.592 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-02-14T15:19:44.592 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-02-14T15:19:44.592 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-14T15:19:44.592 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-02-14T15:19:44.592 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-02-14T15:19:44.592 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 198756, 2024-02-14T15:19:44.593 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 110, 2024-02-14T15:19:44.594 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-02-14T15:19:44.594 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-02-14T15:19:44.594 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-02-14T15:19:44.594 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-02-14T15:19:44.594 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-02-14T15:19:44.594 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-02-14T15:19:44.594 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-02-14T15:19:44.595 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-02-14T15:19:44.595 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 9744, 2024-02-14T15:19:44.595 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 204, 2024-02-14T15:19:44.595 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-02-14T15:19:44.595 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-02-14T15:19:44.595 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-02-14T15:19:44.595 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 326962, 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 3471, 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.596 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 510, 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 26, 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 4, 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 510, 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-02-14T15:19:44.597 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 510, 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1.688947612, 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003311661 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 510, 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 511 2024-02-14T15:19:44.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.599 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 510, 2024-02-14T15:19:44.599 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 510, 2024-02-14T15:19:44.599 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 0, 2024-02-14T15:19:44.599 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-02-14T15:19:44.599 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-02-14T15:19:44.599 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 286, 2024-02-14T15:19:44.599 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 0, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 221, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 0, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 0, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-02-14T15:19:44.600 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 0, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 2, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 0, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-02-14T15:19:44.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-02-14T15:19:44.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-02-14T15:19:44.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-02-14T15:19:44.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-02-14T15:19:44.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-02-14T15:19:44.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-02-14T15:19:44.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-02-14T15:19:44.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-02-14T15:19:44.603 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-02-14T15:19:44.604 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-02-14T15:19:44.604 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-02-14T15:19:44.604 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-02-14T15:19:44.604 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-02-14T15:19:44.604 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-02-14T15:19:44.604 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-02-14T15:19:44.604 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 118, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 4, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 6, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 6, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-02-14T15:19:44.605 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 0, 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 2, 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 0 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 0, 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-02-14T15:19:44.606 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-02-14T15:19:44.607 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-14T15:19:44.608 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 567, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 2212681, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 567, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:44.609 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 567, 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 2212681, 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.611 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-02-14T15:19:44.611 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:44.611 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-02-14T15:19:44.611 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:44.611 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:44.611 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:44.611 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 510, 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 1942164, 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 510, 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1942164, 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.612 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.613 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.613 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.613 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-02-14T15:19:44.613 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:44.613 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-02-14T15:19:44.613 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 510, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 510, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 510, 2024-02-14T15:19:44.614 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 510, 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-02-14T15:19:44.615 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:44.616 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-02-14T15:19:44.617 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x561eae928960": { 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-02-14T15:19:44.618 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-02-14T15:19:44.619 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.620 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-02-14T15:19:44.620 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-02-14T15:19:45.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:44 smithi050 ceph-mon[36512]: pgmap v516: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 764 KiB/s wr, 56 op/s 2024-02-14T15:19:45.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:44 smithi076 ceph-mon[37057]: pgmap v516: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 764 KiB/s wr, 56 op/s 2024-02-14T15:19:45.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:44 smithi017 ceph-mon[32549]: pgmap v516: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 764 KiB/s wr, 56 op/s 2024-02-14T15:19:45.362 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-02-14T15:19:45.362 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-02-14T15:19:45.362 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-02-14T15:19:45.362 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-02-14T15:19:45.362 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-02-14T15:19:45.362 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-02-14T15:19:45.363 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-02-14T15:19:45.363 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-14T15:19:45.373 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-14T15:19:45.373 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:19:45.373 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-14T15:19:45.390 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:19:45.391 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-14T15:19:45.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:19:45.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.591+0000 7fe1f8096640 1 -- 172.21.15.17:0/742660393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101df0 msgr2=0x7fe1f0105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.591+0000 7fe1f8096640 1 --2- 172.21.15.17:0/742660393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101df0 0x7fe1f0105e30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009930 tx=0x7fe1e002bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- 172.21.15.17:0/742660393 shutdown_connections 2024-02-14T15:19:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 --2- 172.21.15.17:0/742660393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101df0 0x7fe1f0105e30 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 --2- 172.21.15.17:0/742660393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1f0106510 0x7fe1f010e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 --2- 172.21.15.17:0/742660393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1f0101440 0x7fe1f0101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- 172.21.15.17:0/742660393 >> 172.21.15.17:0/742660393 conn(0x7fe1f00fcce0 msgr2=0x7fe1f00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- 172.21.15.17:0/742660393 shutdown_connections 2024-02-14T15:19:45.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- 172.21.15.17:0/742660393 wait complete. 2024-02-14T15:19:45.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 Processor -- start 2024-02-14T15:19:45.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- start start 2024-02-14T15:19:45.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101440 0x7fe1f019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1f0106510 0x7fe1f019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1f01976d0 0x7fe1f0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01981a0 con 0x7fe1f0101440 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0198310 con 0x7fe1f0106510 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.592+0000 7fe1f8096640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01984d0 con 0x7fe1f01976d0 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101440 0x7fe1f019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101440 0x7fe1f019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37760/0 (socket says 172.21.15.17:37760) 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 -- 172.21.15.17:0/2673655256 learned_addr learned my addr 172.21.15.17:0/2673655256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f6633640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1f01976d0 0x7fe1f0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5631640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1f0106510 0x7fe1f019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:45.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 -- 172.21.15.17:0/2673655256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1f01976d0 msgr2=0x7fe1f0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1f01976d0 0x7fe1f0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 -- 172.21.15.17:0/2673655256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1f0106510 msgr2=0x7fe1f019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1f0106510 0x7fe1f019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 -- 172.21.15.17:0/2673655256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e0009560 con 0x7fe1f0101440 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5631640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1f0106510 0x7fe1f019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f5e32640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101440 0x7fe1f019d610 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00d950 tx=0x7fe1ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1e6ffd640 1 -- 172.21.15.17:0/2673655256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec009c30 con 0x7fe1f0101440 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1e6ffd640 1 -- 172.21.15.17:0/2673655256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1ec009d90 con 0x7fe1f0101440 2024-02-14T15:19:45.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f01a4470 con 0x7fe1f0101440 2024-02-14T15:19:45.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1e6ffd640 1 -- 172.21.15.17:0/2673655256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec00f7e0 con 0x7fe1f0101440 2024-02-14T15:19:45.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.593+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f01a4950 con 0x7fe1f0101440 2024-02-14T15:19:45.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.594+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f0102910 con 0x7fe1f0101440 2024-02-14T15:19:45.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.595+0000 7fe1e6ffd640 1 -- 172.21.15.17:0/2673655256 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe1ec010ad0 con 0x7fe1f0101440 2024-02-14T15:19:45.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.596+0000 7fe1e6ffd640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe1dc0a2230 0x7fe1dc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.596+0000 7fe1f5631640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe1dc0a2230 0x7fe1dc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:45.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.596+0000 7fe1e6ffd640 1 -- 172.21.15.17:0/2673655256 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe1ec0ce130 con 0x7fe1f0101440 2024-02-14T15:19:45.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.596+0000 7fe1f5631640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe1dc0a2230 0x7fe1dc0a46f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009900 tx=0x7fe1e0005fd0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:45.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.601+0000 7fe1e6ffd640 1 -- 172.21.15.17:0/2673655256 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1ec0969a0 con 0x7fe1f0101440 2024-02-14T15:19:45.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.782+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1f0109770 con 0x7fe1f0101440 2024-02-14T15:19:45.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.784+0000 7fe1e6ffd640 1 -- 172.21.15.17:0/2673655256 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+25220 (secure 0 0 0) 0x7fe1ec09a060 con 0x7fe1f0101440 2024-02-14T15:19:45.785 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:19:45.785 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":118,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:19:15.817399+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/420003070":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/2092370502":"2024-02-15T15:03:24.930590+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2755965102":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1369411962":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/3796717329":"2024-02-15T15:03:24.930590+0000","172.21.15.17:6801/3796717329":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/3775022683":"2024-02-15T15:03:24.930590+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:19:45.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.785+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe1dc0a2230 msgr2=0x7fe1dc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.785+0000 7fe1f8096640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe1dc0a2230 0x7fe1dc0a46f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009900 tx=0x7fe1e0005fd0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.785+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101440 msgr2=0x7fe1f019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.785+0000 7fe1f8096640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101440 0x7fe1f019d610 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00d950 tx=0x7fe1ec00de10 comp rx=0 tx=0).stop 2024-02-14T15:19:45.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 shutdown_connections 2024-02-14T15:19:45.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fe1dc0a2230 0x7fe1dc0a46f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1f01976d0 0x7fe1f0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1f0106510 0x7fe1f019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 --2- 172.21.15.17:0/2673655256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f0101440 0x7fe1f019d610 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 >> 172.21.15.17:0/2673655256 conn(0x7fe1f00fcce0 msgr2=0x7fe1f00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:45.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 shutdown_connections 2024-02-14T15:19:45.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.786+0000 7fe1f8096640 1 -- 172.21.15.17:0/2673655256 wait complete. 2024-02-14T15:19:45.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T14:56:08.919729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-14T15:00:47.196882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-14T15:00:47.810743+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-14T15:00:51.802704+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-14T15:19:45.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-14T15:19:45.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- 172.21.15.17:0/2702071843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c100cf0 msgr2=0x7f202c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 --2- 172.21.15.17:0/2702071843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c100cf0 0x7f202c104d30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f201c009930 tx=0x7f201c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:45.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- 172.21.15.17:0/2702071843 shutdown_connections 2024-02-14T15:19:45.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 --2- 172.21.15.17:0/2702071843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c100cf0 0x7f202c104d30 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 --2- 172.21.15.17:0/2702071843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c105410 0x7f202c10cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 --2- 172.21.15.17:0/2702071843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f202c100340 0x7f202c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- 172.21.15.17:0/2702071843 >> 172.21.15.17:0/2702071843 conn(0x7f202c0772e0 msgr2=0x7f202c0776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- 172.21.15.17:0/2702071843 shutdown_connections 2024-02-14T15:19:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- 172.21.15.17:0/2702071843 wait complete. 2024-02-14T15:19:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 Processor -- start 2024-02-14T15:19:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- start start 2024-02-14T15:19:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c100340 0x7f202c10d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c105410 0x7f202c10de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f202c10e340 0x7f202c1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1a60b0 con 0x7f202c105410 2024-02-14T15:19:45.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1a6220 con 0x7f202c10e340 2024-02-14T15:19:45.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.905+0000 7f203458c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1a63e0 con 0x7f202c100340 2024-02-14T15:19:45.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c105410 0x7f202c10de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:45.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c105410 0x7f202c10de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37768/0 (socket says 172.21.15.17:37768) 2024-02-14T15:19:45.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 -- 172.21.15.17:0/408155919 learned_addr learned my addr 172.21.15.17:0/408155919 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:45.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2032328640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c100340 0x7f202c10d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:45.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 -- 172.21.15.17:0/408155919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c100340 msgr2=0x7f202c10d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c100340 0x7f202c10d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 -- 172.21.15.17:0/408155919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f202c10e340 msgr2=0x7f202c1a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:45.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f202c10e340 0x7f202c1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:45.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 -- 172.21.15.17:0/408155919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c009560 con 0x7f202c105410 2024-02-14T15:19:45.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f2031b27640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c105410 0x7f202c10de00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f201c009230 tx=0x7f201c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:45.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f20237fe640 1 -- 172.21.15.17:0/408155919 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c02cbc0 con 0x7f202c105410 2024-02-14T15:19:45.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c1a6660 con 0x7f202c105410 2024-02-14T15:19:45.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f20237fe640 1 -- 172.21.15.17:0/408155919 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f201c0052d0 con 0x7f202c105410 2024-02-14T15:19:45.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f20237fe640 1 -- 172.21.15.17:0/408155919 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c003e30 con 0x7f202c105410 2024-02-14T15:19:45.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.906+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c1a6b70 con 0x7f202c105410 2024-02-14T15:19:45.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.907+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c0680b0 con 0x7f202c105410 2024-02-14T15:19:45.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.908+0000 7f20237fe640 1 -- 172.21.15.17:0/408155919 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f201c0035f0 con 0x7f202c105410 2024-02-14T15:19:45.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.909+0000 7f20237fe640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20180a2260 0x7f20180a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:45.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.909+0000 7f20237fe640 1 -- 172.21.15.17:0/408155919 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f201c0f08f0 con 0x7f202c105410 2024-02-14T15:19:45.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.909+0000 7f2032328640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20180a2260 0x7f20180a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:45.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.909+0000 7f2032328640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20180a2260 0x7f20180a4720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2028005fd0 tx=0x7f2028009470 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:45.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:45.913+0000 7f20237fe640 1 -- 172.21.15.17:0/408155919 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f201c0b9270 con 0x7f202c105410 2024-02-14T15:19:46.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.094+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f202c108640 con 0x7f202c105410 2024-02-14T15:19:46.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.094+0000 7f20237fe640 1 -- 172.21.15.17:0/408155919 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f201c0bc930 con 0x7f202c105410 2024-02-14T15:19:46.095 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-02-14T15:19:46.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20180a2260 msgr2=0x7f20180a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20180a2260 0x7f20180a4720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2028005fd0 tx=0x7f2028009470 comp rx=0 tx=0).stop 2024-02-14T15:19:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c105410 msgr2=0x7f202c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c105410 0x7f202c10de00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f201c009230 tx=0x7f201c004270 comp rx=0 tx=0).stop 2024-02-14T15:19:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 shutdown_connections 2024-02-14T15:19:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f20180a2260 0x7f20180a4720 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f202c105410 0x7f202c10de00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f202c10e340 0x7f202c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 --2- 172.21.15.17:0/408155919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c100340 0x7f202c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 >> 172.21.15.17:0/408155919 conn(0x7f202c0772e0 msgr2=0x7f202c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:46.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 shutdown_connections 2024-02-14T15:19:46.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.096+0000 7f203458c640 1 -- 172.21.15.17:0/408155919 wait complete. 2024-02-14T15:19:46.106 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:45 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2673655256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:46.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-14T15:19:46.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:45 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2673655256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 -- 172.21.15.17:0/2861316756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c100330 msgr2=0x7f314c100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 --2- 172.21.15.17:0/2861316756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c100330 0x7f314c100710 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f313c009930 tx=0x7f313c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 -- 172.21.15.17:0/2861316756 shutdown_connections 2024-02-14T15:19:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 --2- 172.21.15.17:0/2861316756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f314c105400 0x7f314c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 --2- 172.21.15.17:0/2861316756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c100ce0 0x7f314c104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 --2- 172.21.15.17:0/2861316756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c100330 0x7f314c100710 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 -- 172.21.15.17:0/2861316756 >> 172.21.15.17:0/2861316756 conn(0x7f314c077250 msgr2=0x7f314c077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 -- 172.21.15.17:0/2861316756 shutdown_connections 2024-02-14T15:19:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 -- 172.21.15.17:0/2861316756 wait complete. 2024-02-14T15:19:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 Processor -- start 2024-02-14T15:19:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.214+0000 7f31525d0640 1 -- start start 2024-02-14T15:19:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f31525d0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c100330 0x7f314c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f31525d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c105400 0x7f314c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f31525d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f314c1976c0 0x7f314c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f31525d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c198190 con 0x7f314c105400 2024-02-14T15:19:46.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f31525d0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c198300 con 0x7f314c100330 2024-02-14T15:19:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f31525d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c1984c0 con 0x7f314c1976c0 2024-02-14T15:19:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c105400 0x7f314c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c105400 0x7f314c19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37788/0 (socket says 172.21.15.17:37788) 2024-02-14T15:19:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 -- 172.21.15.17:0/3883115643 learned_addr learned my addr 172.21.15.17:0/3883115643 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314bfff640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c100330 0x7f314c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 -- 172.21.15.17:0/3883115643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f314c1976c0 msgr2=0x7f314c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f3150b6d640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f314c1976c0 0x7f314c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f314c1976c0 0x7f314c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 -- 172.21.15.17:0/3883115643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c100330 msgr2=0x7f314c19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c100330 0x7f314c19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 -- 172.21.15.17:0/3883115643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c009560 con 0x7f314c105400 2024-02-14T15:19:46.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314bfff640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c100330 0x7f314c19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:19:46.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.215+0000 7f314b7fe640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c105400 0x7f314c19db40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f314000c970 tx=0x7f314000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:46.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.216+0000 7f31497fa640 1 -- 172.21.15.17:0/3883115643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140007bb0 con 0x7f314c105400 2024-02-14T15:19:46.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.216+0000 7f31497fa640 1 -- 172.21.15.17:0/3883115643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3140007d10 con 0x7f314c105400 2024-02-14T15:19:46.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.216+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c1a4460 con 0x7f314c105400 2024-02-14T15:19:46.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.216+0000 7f31497fa640 1 -- 172.21.15.17:0/3883115643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314000f450 con 0x7f314c105400 2024-02-14T15:19:46.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.216+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c1a4940 con 0x7f314c105400 2024-02-14T15:19:46.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.217+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c101800 con 0x7f314c105400 2024-02-14T15:19:46.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.218+0000 7f31497fa640 1 -- 172.21.15.17:0/3883115643 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3140004f20 con 0x7f314c105400 2024-02-14T15:19:46.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.218+0000 7f31497fa640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f31340a2230 0x7f31340a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.218+0000 7f314bfff640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f31340a2230 0x7f31340a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.218+0000 7f31497fa640 1 -- 172.21.15.17:0/3883115643 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f31400ce4c0 con 0x7f314c105400 2024-02-14T15:19:46.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.219+0000 7f314bfff640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f31340a2230 0x7f31340a46f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f313c006010 tx=0x7f313c004960 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:46.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.223+0000 7f31497fa640 1 -- 172.21.15.17:0/3883115643 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3140096e30 con 0x7f314c105400 2024-02-14T15:19:46.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:45 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2673655256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.401+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f314c108630 con 0x7f314c105400 2024-02-14T15:19:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.402+0000 7f31497fa640 1 -- 172.21.15.17:0/3883115643 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7f314001d020 con 0x7f314c105400 2024-02-14T15:19:46.403 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:19:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f31340a2230 msgr2=0x7f31340a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f31340a2230 0x7f31340a46f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f313c006010 tx=0x7f313c004960 comp rx=0 tx=0).stop 2024-02-14T15:19:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c105400 msgr2=0x7f314c19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c105400 0x7f314c19db40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f314000c970 tx=0x7f314000ce30 comp rx=0 tx=0).stop 2024-02-14T15:19:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 shutdown_connections 2024-02-14T15:19:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f31340a2230 0x7f31340a46f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f314c105400 0x7f314c19db40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f314c1976c0 0x7f314c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 --2- 172.21.15.17:0/3883115643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c100330 0x7f314c19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 >> 172.21.15.17:0/3883115643 conn(0x7f314c077250 msgr2=0x7f314c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 shutdown_connections 2024-02-14T15:19:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.404+0000 7f31525d0640 1 -- 172.21.15.17:0/3883115643 wait complete. 2024-02-14T15:19:46.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-14T15:19:46.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 -- 172.21.15.17:0/3947741371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594100ce0 msgr2=0x7f9594104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 --2- 172.21.15.17:0/3947741371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594100ce0 0x7f9594104d20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9588009930 tx=0x7f958802bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:46.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 -- 172.21.15.17:0/3947741371 shutdown_connections 2024-02-14T15:19:46.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 --2- 172.21.15.17:0/3947741371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594100ce0 0x7f9594104d20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 --2- 172.21.15.17:0/3947741371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594105400 0x7f959410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 --2- 172.21.15.17:0/3947741371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9594100330 0x7f9594100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 -- 172.21.15.17:0/3947741371 >> 172.21.15.17:0/3947741371 conn(0x7f95940fbb90 msgr2=0x7f95940fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 -- 172.21.15.17:0/3947741371 shutdown_connections 2024-02-14T15:19:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 -- 172.21.15.17:0/3947741371 wait complete. 2024-02-14T15:19:46.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 Processor -- start 2024-02-14T15:19:46.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.521+0000 7f9599ed4640 1 -- start start 2024-02-14T15:19:46.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9594100330 0x7f959410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594105400 0x7f959410ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f959410e330 0x7f95941a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941a6140 con 0x7f9594105400 2024-02-14T15:19:46.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941a62b0 con 0x7f9594100330 2024-02-14T15:19:46.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941a6470 con 0x7f959410e330 2024-02-14T15:19:46.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594105400 0x7f959410ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594105400 0x7f959410ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37812/0 (socket says 172.21.15.17:37812) 2024-02-14T15:19:46.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 -- 172.21.15.17:0/521918784 learned_addr learned my addr 172.21.15.17:0/521918784 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:46.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f95937fe640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9594100330 0x7f959410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 -- 172.21.15.17:0/521918784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f959410e330 msgr2=0x7f95941a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f959410e330 0x7f95941a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 -- 172.21.15.17:0/521918784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9594100330 msgr2=0x7f959410d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9594100330 0x7f959410d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 -- 172.21.15.17:0/521918784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588009560 con 0x7f9594105400 2024-02-14T15:19:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9592ffd640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594105400 0x7f959410ddf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9588000c00 tx=0x7f9588004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:46.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9590ff9640 1 -- 172.21.15.17:0/521918784 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588004df0 con 0x7f9594105400 2024-02-14T15:19:46.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95941a66f0 con 0x7f9594105400 2024-02-14T15:19:46.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9590ff9640 1 -- 172.21.15.17:0/521918784 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9588005430 con 0x7f9594105400 2024-02-14T15:19:46.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9590ff9640 1 -- 172.21.15.17:0/521918784 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958803e450 con 0x7f9594105400 2024-02-14T15:19:46.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.522+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95941a6c00 con 0x7f9594105400 2024-02-14T15:19:46.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.523+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9594068500 con 0x7f9594105400 2024-02-14T15:19:46.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.524+0000 7f9590ff9640 1 -- 172.21.15.17:0/521918784 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9588003680 con 0x7f9594105400 2024-02-14T15:19:46.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.525+0000 7f9590ff9640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f957c0a2230 0x7f957c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.525+0000 7f95937fe640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f957c0a2230 0x7f957c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.525+0000 7f9590ff9640 1 -- 172.21.15.17:0/521918784 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f95880f0f00 con 0x7f9594105400 2024-02-14T15:19:46.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.526+0000 7f95937fe640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f957c0a2230 0x7f957c0a46f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f95840058c0 tx=0x7f958400a2d0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:46.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.529+0000 7f9590ff9640 1 -- 172.21.15.17:0/521918784 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95880b9880 con 0x7f9594105400 2024-02-14T15:19:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.711+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9594108690 con 0x7f9594105400 2024-02-14T15:19:46.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.711+0000 7f9590ff9640 1 -- 172.21.15.17:0/521918784 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95880bcf40 con 0x7f9594105400 2024-02-14T15:19:46.712 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-02-14T15:19:46.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f957c0a2230 msgr2=0x7f957c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f957c0a2230 0x7f957c0a46f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f95840058c0 tx=0x7f958400a2d0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594105400 msgr2=0x7f959410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594105400 0x7f959410ddf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9588000c00 tx=0x7f9588004270 comp rx=0 tx=0).stop 2024-02-14T15:19:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 shutdown_connections 2024-02-14T15:19:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f957c0a2230 0x7f957c0a46f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9594105400 0x7f959410ddf0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f959410e330 0x7f95941a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 --2- 172.21.15.17:0/521918784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9594100330 0x7f959410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 >> 172.21.15.17:0/521918784 conn(0x7f95940fbb90 msgr2=0x7f95940fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 shutdown_connections 2024-02-14T15:19:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.713+0000 7f9599ed4640 1 -- 172.21.15.17:0/521918784 wait complete. 2024-02-14T15:19:46.725 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-14T15:19:46.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 -- 172.21.15.17:0/698690413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c101dd0 msgr2=0x7f780c105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 --2- 172.21.15.17:0/698690413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c101dd0 0x7f780c105e10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7800009930 tx=0x7f780002bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 -- 172.21.15.17:0/698690413 shutdown_connections 2024-02-14T15:19:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 --2- 172.21.15.17:0/698690413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c101dd0 0x7f780c105e10 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 --2- 172.21.15.17:0/698690413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f780c1064f0 0x7f780c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 --2- 172.21.15.17:0/698690413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780c101420 0x7f780c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 -- 172.21.15.17:0/698690413 >> 172.21.15.17:0/698690413 conn(0x7f780c0fcca0 msgr2=0x7f780c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:46.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 -- 172.21.15.17:0/698690413 shutdown_connections 2024-02-14T15:19:46.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 -- 172.21.15.17:0/698690413 wait complete. 2024-02-14T15:19:46.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 Processor -- start 2024-02-14T15:19:46.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.830+0000 7f7812f13640 1 -- start start 2024-02-14T15:19:46.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7812f13640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780c101420 0x7f780c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7812f13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f780c1064f0 0x7f780c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7812f13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c1976f0 0x7f780c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7812f13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1981c0 con 0x7f780c1976f0 2024-02-14T15:19:46.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7812f13640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c198330 con 0x7f780c101420 2024-02-14T15:19:46.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7812f13640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1984f0 con 0x7f780c1064f0 2024-02-14T15:19:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c1976f0 0x7f780c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c1976f0 0x7f780c197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37822/0 (socket says 172.21.15.17:37822) 2024-02-14T15:19:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 -- 172.21.15.17:0/205758791 learned_addr learned my addr 172.21.15.17:0/205758791 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7810caf640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780c101420 0x7f780c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f780bfff640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f780c1064f0 0x7f780c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 -- 172.21.15.17:0/205758791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f780c1064f0 msgr2=0x7f780c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f780c1064f0 0x7f780c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 -- 172.21.15.17:0/205758791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780c101420 msgr2=0x7f780c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:46.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780c101420 0x7f780c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:46.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 -- 172.21.15.17:0/205758791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7800009560 con 0x7f780c1976f0 2024-02-14T15:19:46.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f780bfff640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f780c1064f0 0x7f780c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:19:46.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f78114b0640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c1976f0 0x7f780c197b70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00d750 tx=0x7f77fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:46.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7809ffb640 1 -- 172.21.15.17:0/205758791 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc009bd0 con 0x7f780c1976f0 2024-02-14T15:19:46.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7809ffb640 1 -- 172.21.15.17:0/205758791 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77fc00de60 con 0x7f780c1976f0 2024-02-14T15:19:46.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7809ffb640 1 -- 172.21.15.17:0/205758791 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc004e70 con 0x7f780c1976f0 2024-02-14T15:19:46.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.831+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c1a4490 con 0x7f780c1976f0 2024-02-14T15:19:46.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.832+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c1a4970 con 0x7f780c1976f0 2024-02-14T15:19:46.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.833+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c1028f0 con 0x7f780c1976f0 2024-02-14T15:19:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.834+0000 7f7809ffb640 1 -- 172.21.15.17:0/205758791 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f77fc005020 con 0x7f780c1976f0 2024-02-14T15:19:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.834+0000 7f7809ffb640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f77f40a2230 0x7f77f40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.834+0000 7f7810caf640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f77f40a2230 0x7f77f40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.834+0000 7f7809ffb640 1 -- 172.21.15.17:0/205758791 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f77fc0cdd60 con 0x7f780c1976f0 2024-02-14T15:19:46.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.835+0000 7f7810caf640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f77f40a2230 0x7f77f40a46f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7804005fd0 tx=0x7f7804009680 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:46.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:46.839+0000 7f7809ffb640 1 -- 172.21.15.17:0/205758791 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f77fc09a160 con 0x7f780c1976f0 2024-02-14T15:19:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:46 smithi017 ceph-mon[32549]: pgmap v517: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 447 KiB/s wr, 31 op/s 2024-02-14T15:19:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/408155919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3883115643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.016 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:46 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/521918784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.022+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f780c109720 con 0x7f780c1976f0 2024-02-14T15:19:47.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.022+0000 7f7809ffb640 1 -- 172.21.15.17:0/205758791 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v118) v1 ==== 103+0+11 (secure 0 0 0) 0x7f77fc017020 con 0x7f780c1976f0 2024-02-14T15:19:47.023 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-02-14T15:19:47.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f77f40a2230 msgr2=0x7f77f40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:47.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f77f40a2230 0x7f77f40a46f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7804005fd0 tx=0x7f7804009680 comp rx=0 tx=0).stop 2024-02-14T15:19:47.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c1976f0 msgr2=0x7f780c197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:47.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c1976f0 0x7f780c197b70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00d750 tx=0x7f77fc00dc10 comp rx=0 tx=0).stop 2024-02-14T15:19:47.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 shutdown_connections 2024-02-14T15:19:47.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f77f40a2230 0x7f77f40a46f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f780c1976f0 0x7f780c197b70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f780c1064f0 0x7f780c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 --2- 172.21.15.17:0/205758791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780c101420 0x7f780c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 >> 172.21.15.17:0/205758791 conn(0x7f780c0fcca0 msgr2=0x7f780c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:47.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 shutdown_connections 2024-02-14T15:19:47.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.024+0000 7f7812f13640 1 -- 172.21.15.17:0/205758791 wait complete. 2024-02-14T15:19:47.036 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-14T15:19:47.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 -- 172.21.15.17:0/4077163997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7850100440 msgr2=0x7f7850100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:47.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 --2- 172.21.15.17:0/4077163997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7850100440 0x7f7850100820 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7840009930 tx=0x7f784002bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:47.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 -- 172.21.15.17:0/4077163997 shutdown_connections 2024-02-14T15:19:47.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 --2- 172.21.15.17:0/4077163997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78501026c0 0x7f785010d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 --2- 172.21.15.17:0/4077163997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7850101d20 0x7f7850102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 --2- 172.21.15.17:0/4077163997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7850100440 0x7f7850100820 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 -- 172.21.15.17:0/4077163997 >> 172.21.15.17:0/4077163997 conn(0x7f7850077310 msgr2=0x7f7850077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:47.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 -- 172.21.15.17:0/4077163997 shutdown_connections 2024-02-14T15:19:47.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.139+0000 7f7856a40640 1 -- 172.21.15.17:0/4077163997 wait complete. 2024-02-14T15:19:47.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 Processor -- start 2024-02-14T15:19:47.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 -- start start 2024-02-14T15:19:47.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7850100440 0x7f785019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:47.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7850101d20 0x7f785019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:47.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78501026c0 0x7f7850197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:47.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7850197f20 con 0x7f78501026c0 2024-02-14T15:19:47.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7850198090 con 0x7f7850101d20 2024-02-14T15:19:47.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7856a40640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7850198250 con 0x7f7850100440 2024-02-14T15:19:47.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7854fdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78501026c0 0x7f7850197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:47.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7854fdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78501026c0 0x7f7850197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37846/0 (socket says 172.21.15.17:37846) 2024-02-14T15:19:47.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7854fdd640 1 -- 172.21.15.17:0/3233013729 learned_addr learned my addr 172.21.15.17:0/3233013729 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:47.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f784f7fe640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7850101d20 0x7f785019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:47.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f784ffff640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7850100440 0x7f785019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:47.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7854fdd640 1 -- 172.21.15.17:0/3233013729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7850100440 msgr2=0x7f785019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:47.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7854fdd640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7850100440 0x7f785019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7854fdd640 1 -- 172.21.15.17:0/3233013729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7850101d20 msgr2=0x7f785019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:47.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.140+0000 7f7854fdd640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7850101d20 0x7f785019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.141+0000 7f7854fdd640 1 -- 172.21.15.17:0/3233013729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009560 con 0x7f78501026c0 2024-02-14T15:19:47.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.141+0000 7f7854fdd640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78501026c0 0x7f7850197790 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f784800b9e0 tx=0x7f784800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:47.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.141+0000 7f784d7fa640 1 -- 172.21.15.17:0/3233013729 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800caa0 con 0x7f78501026c0 2024-02-14T15:19:47.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.141+0000 7f784d7fa640 1 -- 172.21.15.17:0/3233013729 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f784800cc00 con 0x7f78501026c0 2024-02-14T15:19:47.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.141+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7850198530 con 0x7f78501026c0 2024-02-14T15:19:47.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.141+0000 7f784d7fa640 1 -- 172.21.15.17:0/3233013729 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848007670 con 0x7f78501026c0 2024-02-14T15:19:47.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.141+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78501a4680 con 0x7f78501026c0 2024-02-14T15:19:47.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.142+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7850101830 con 0x7f78501026c0 2024-02-14T15:19:47.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.143+0000 7f784d7fa640 1 -- 172.21.15.17:0/3233013729 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7848014440 con 0x7f78501026c0 2024-02-14T15:19:47.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:46 smithi050 ceph-mon[36512]: pgmap v517: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 447 KiB/s wr, 31 op/s 2024-02-14T15:19:47.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/408155919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3883115643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:46 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/521918784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.144+0000 7f784d7fa640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f78380a2230 0x7f78380a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:47.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.144+0000 7f784ffff640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f78380a2230 0x7f78380a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:47.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.144+0000 7f784d7fa640 1 -- 172.21.15.17:0/3233013729 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f78480ce290 con 0x7f78501026c0 2024-02-14T15:19:47.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.144+0000 7f784ffff640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f78380a2230 0x7f78380a46f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7840036040 tx=0x7f7840005fd0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:47.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.147+0000 7f784d7fa640 1 -- 172.21.15.17:0/3233013729 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7848096c20 con 0x7f78501026c0 2024-02-14T15:19:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:46 smithi076 ceph-mon[37057]: pgmap v517: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 447 KiB/s wr, 31 op/s 2024-02-14T15:19:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/408155919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3883115643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:46 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/521918784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.341+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7850198c70 con 0x7f78501026c0 2024-02-14T15:19:47.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.341+0000 7f784d7fa640 1 -- 172.21.15.17:0/3233013729 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v118) v1 ==== 81+0+1003 (secure 0 0 0) 0x7f7848023400 con 0x7f78501026c0 2024-02-14T15:19:47.342 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/420003070 2024-02-15T15:03:24.930590+0000 2024-02-14T15:19:47.342 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2092370502 2024-02-15T15:03:24.930590+0000 2024-02-14T15:19:47.343 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/1044675107 2024-02-15T14:53:42.569888+0000 2024-02-14T15:19:47.343 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3371941300 2024-02-15T14:53:15.465600+0000 2024-02-14T15:19:47.343 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/2869822133 2024-02-15T14:53:15.465600+0000 2024-02-14T15:19:47.343 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/2869822133 2024-02-15T14:53:15.465600+0000 2024-02-14T15:19:47.343 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3241397077 2024-02-15T14:53:42.569888+0000 2024-02-14T15:19:47.344 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2755965102 2024-02-15T15:03:24.930590+0000 2024-02-14T15:19:47.344 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1117796267 2024-02-15T14:53:15.465600+0000 2024-02-14T15:19:47.344 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/1044675107 2024-02-15T14:53:42.569888+0000 2024-02-14T15:19:47.344 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1369411962 2024-02-15T15:03:24.930590+0000 2024-02-14T15:19:47.344 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3756063081 2024-02-15T14:53:42.569888+0000 2024-02-14T15:19:47.344 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2319986884 2024-02-15T14:53:15.465600+0000 2024-02-14T15:19:47.345 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3179868836 2024-02-15T14:53:42.569888+0000 2024-02-14T15:19:47.345 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/3796717329 2024-02-15T15:03:24.930590+0000 2024-02-14T15:19:47.345 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/3796717329 2024-02-15T15:03:24.930590+0000 2024-02-14T15:19:47.345 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3775022683 2024-02-15T15:03:24.930590+0000 2024-02-14T15:19:47.346 INFO:teuthology.orchestra.run.smithi017.stderr:listed 17 entries 2024-02-14T15:19:47.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f78380a2230 msgr2=0x7f78380a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:47.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f78380a2230 0x7f78380a46f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7840036040 tx=0x7f7840005fd0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78501026c0 msgr2=0x7f7850197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:47.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78501026c0 0x7f7850197790 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f784800b9e0 tx=0x7f784800bea0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 shutdown_connections 2024-02-14T15:19:47.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f78380a2230 0x7f78380a46f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78501026c0 0x7f7850197790 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7850101d20 0x7f785019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 --2- 172.21.15.17:0/3233013729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7850100440 0x7f785019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 >> 172.21.15.17:0/3233013729 conn(0x7f7850077310 msgr2=0x7f7850103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 shutdown_connections 2024-02-14T15:19:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:47.343+0000 7f7856a40640 1 -- 172.21.15.17:0/3233013729 wait complete. 2024-02-14T15:19:47.354 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-14T15:19:47.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:19:47.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-14T15:19:47.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:19:47.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:19:47.711 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-14T15:19:47.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:19:47.729 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-02-14T15:19:47.790 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-14T15:19:47.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:19:47.791 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-14T15:19:47.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:19:47.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-02-14T15:19:47.950 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-14T15:19:47.950 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-02-14T15:19:47.950 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-14T15:19:47.962 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:47 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/205758791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:19:47.963 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:47 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3233013729' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:19:47.966 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2360 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/8b6734776db8aef51266f2a3d0160e888bd04b69e83d997f0b31f8a4cedcb4b3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/8b6734776db8aef51266f2a3d0160e888bd04b69e83d997f0b31f8a4cedcb4b3/diff,workdir=/var/lib/containers/storage/overlay/8b6734776db8aef51266f2a3d0160e888bd04b69e83d997f0b31f8a4cedcb4b3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d861576df424177cb8cdc863c94f9701ada21fe5141715de2dea90bb781e6154/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/d861576df424177cb8cdc863c94f9701ada21fe5141715de2dea90bb781e6154/diff,workdir=/var/lib/containers/storage/overlay/d861576df424177cb8cdc863c94f9701ada21fe5141715de2dea90bb781e6154/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/0ae06949177ecab0ecef87cec4209170f8d5c28ba2f7e634c5a6cb40cc7c3397/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/0ae06949177ecab0ecef87cec4209170f8d5c28ba2f7e634c5a6cb40cc7c3397/diff,workdir=/var/lib/containers/storage/overlay/0ae06949177ecab0ecef87cec4209170f8d5c28ba2f7e634c5a6cb40cc7c3397/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6408aba98fd886742fd814fca128d424f64488294446768d78578460008dbbdb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/6408aba98fd886742fd814fca128d424f64488294446768d78578460008dbbdb/diff,workdir=/var/lib/containers/storage/overlay/6408aba98fd886742fd814fca128d424f64488294446768d78578460008dbbdb/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4e9fe037f9ab03d830a6fdae261e5069fc336b2cf621fb68f8cc164f0ed07d46/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/4e9fe037f9ab03d830a6fdae261e5069fc336b2cf621fb68f8cc164f0ed07d46/diff,workdir=/var/lib/containers/storage/overlay/4e9fe037f9ab03d830a6fdae261e5069fc336b2cf621fb68f8cc164f0ed07d46/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2ff86006cb0ff0a4bff2d8ba8758700c1593c538c6cc4b4a28882bd157042932/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/2ff86006cb0ff0a4bff2d8ba8758700c1593c538c6cc4b4a28882bd157042932/diff,workdir=/var/lib/containers/storage/overlay/2ff86006cb0ff0a4bff2d8ba8758700c1593c538c6cc4b4a28882bd157042932/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b2ba981a99196d8d0ad5a545ddf127e148a9bda34e278ef7ac21cbbc5d408d10/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/b2ba981a99196d8d0ad5a545ddf127e148a9bda34e278ef7ac21cbbc5d408d10/diff,workdir=/var/lib/containers/storage/overlay/b2ba981a99196d8d0ad5a545ddf127e148a9bda34e278ef7ac21cbbc5d408d10/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b2cf7369bec566ea5e6aa214e613209caaa13d4039ec91ce165a4a27a777ed9c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/b2cf7369bec566ea5e6aa214e613209caaa13d4039ec91ce165a4a27a777ed9c/diff,workdir=/var/lib/containers/storage/overlay/b2cf7369bec566ea5e6aa214e613209caaa13d4039ec91ce165a4a27a777ed9c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4ea4454b7f04bb88c0089ee77463bb00245da1f898860442535e18f5fc143d18/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/4ea4454b7f04bb88c0089ee77463bb00245da1f898860442535e18f5fc143d18/diff,workdir=/var/lib/containers/storage/overlay/4ea4454b7f04bb88c0089ee77463bb00245da1f898860442535e18f5fc143d18/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0a56b646ee5b8ed0c5a636ceb609c05fd1c4a2c2a2d11313967f6ba76da383ed/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MZ2PHVXYZ55S7KRQN65F35KER7:/var/lib/containers/storage/overlay/l/Z7EYXTO37RJCR6GUT5MB5P7IMO,upperdir=/var/lib/containers/storage/overlay/0a56b646ee5b8ed0c5a636ceb609c05fd1c4a2c2a2d11313967f6ba76da383ed/diff,workdir=/var/lib/containers/storage/overlay/0a56b646ee5b8ed0c5a636ceb609c05fd1c4a2c2a2d11313967f6ba76da383ed/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-02-14T15:19:47.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:19:47.967 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-02-14T15:19:48.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-14T15:19:48.021 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-02-14T15:19:48.091 INFO:teuthology.orchestra.run.smithi017.stderr:Cannot find device "ceph-brx" 2024-02-14T15:19:48.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:19:48.093 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-14T15:19:48.103 INFO:tasks.cephadm:Teardown begin 2024-02-14T15:19:48.104 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-14T15:19:48.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:19:48.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:47 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/205758791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:19:48.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:47 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3233013729' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:19:48.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:47 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/205758791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-14T15:19:48.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:47 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3233013729' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T15:19:48.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 -- 172.21.15.17:0/2160900027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101df0 msgr2=0x7fc704105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 --2- 172.21.15.17:0/2160900027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101df0 0x7fc704105e30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009930 tx=0x7fc6ec02bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 -- 172.21.15.17:0/2160900027 shutdown_connections 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 --2- 172.21.15.17:0/2160900027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101df0 0x7fc704105e30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 --2- 172.21.15.17:0/2160900027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704106510 0x7fc70410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 --2- 172.21.15.17:0/2160900027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc704101440 0x7fc704101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 -- 172.21.15.17:0/2160900027 >> 172.21.15.17:0/2160900027 conn(0x7fc7040fcce0 msgr2=0x7fc7040ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 -- 172.21.15.17:0/2160900027 shutdown_connections 2024-02-14T15:19:48.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 -- 172.21.15.17:0/2160900027 wait complete. 2024-02-14T15:19:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.236+0000 7fc709e59640 1 Processor -- start 2024-02-14T15:19:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc709e59640 1 -- start start 2024-02-14T15:19:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc709e59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101440 0x7fc70419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc709e59640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704106510 0x7fc70419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc709e59640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc704197730 0x7fc704197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc709e59640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704198200 con 0x7fc704101440 2024-02-14T15:19:48.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc709e59640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704198370 con 0x7fc704106510 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc709e59640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704198530 con 0x7fc704197730 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101440 0x7fc70419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101440 0x7fc70419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37872/0 (socket says 172.21.15.17:37872) 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 -- 172.21.15.17:0/3862273322 learned_addr learned my addr 172.21.15.17:0/3862273322 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc702ffd640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704106510 0x7fc70419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc703fff640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc704197730 0x7fc704197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 -- 172.21.15.17:0/3862273322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc704197730 msgr2=0x7fc704197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc704197730 0x7fc704197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 -- 172.21.15.17:0/3862273322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704106510 msgr2=0x7fc70419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704106510 0x7fc70419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 -- 172.21.15.17:0/3862273322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec009560 con 0x7fc704101440 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc702ffd640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704106510 0x7fc70419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.237+0000 7fc7037fe640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101440 0x7fc70419d620 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400ea80 tx=0x7fc6f400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.238+0000 7fc700ff9640 1 -- 172.21.15.17:0/3862273322 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400ce40 con 0x7fc704101440 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.238+0000 7fc700ff9640 1 -- 172.21.15.17:0/3862273322 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f4005020 con 0x7fc704101440 2024-02-14T15:19:48.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.238+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7041a4480 con 0x7fc704101440 2024-02-14T15:19:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.238+0000 7fc700ff9640 1 -- 172.21.15.17:0/3862273322 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40116d0 con 0x7fc704101440 2024-02-14T15:19:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.238+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7041a4910 con 0x7fc704101440 2024-02-14T15:19:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.239+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc704102910 con 0x7fc704101440 2024-02-14T15:19:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.240+0000 7fc700ff9640 1 -- 172.21.15.17:0/3862273322 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc6f4005190 con 0x7fc704101440 2024-02-14T15:19:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.240+0000 7fc700ff9640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc6f00a2230 0x7fc6f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.240+0000 7fc700ff9640 1 -- 172.21.15.17:0/3862273322 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fc6f40cec00 con 0x7fc704101440 2024-02-14T15:19:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.240+0000 7fc702ffd640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc6f00a2230 0x7fc6f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.241+0000 7fc702ffd640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc6f00a2230 0x7fc6f00a46f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009900 tx=0x7fc6ec005fd0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:48.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.244+0000 7fc700ff9640 1 -- 172.21.15.17:0/3862273322 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6f40974d0 con 0x7fc704101440 2024-02-14T15:19:48.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.465+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc704109770 con 0x7fc704101440 2024-02-14T15:19:48.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.466+0000 7fc700ff9640 1 -- 172.21.15.17:0/3862273322 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11318 (secure 0 0 0) 0x7fc6f401f070 con 0x7fc704101440 2024-02-14T15:19:48.467 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:19:48.467 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":26764,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/349944697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":349944697},{"type":"v1","addr":"172.21.15.50:6837","nonce":349944697}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:19:33.893894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":77,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26947":{"gid":26947,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/1295210715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1295210715},{"type":"v1","addr":"172.21.15.76:6833","nonce":1295210715}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:19:48.467 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2024-02-14T15:19:48.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc6f00a2230 msgr2=0x7fc6f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc6f00a2230 0x7fc6f00a46f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009900 tx=0x7fc6ec005fd0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101440 msgr2=0x7fc70419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101440 0x7fc70419d620 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400ea80 tx=0x7fc6f400ef40 comp rx=0 tx=0).stop 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 shutdown_connections 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7fc6f00a2230 0x7fc6f00a46f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc704197730 0x7fc704197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704106510 0x7fc70419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 --2- 172.21.15.17:0/3862273322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc704101440 0x7fc70419d620 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 >> 172.21.15.17:0/3862273322 conn(0x7fc7040fcce0 msgr2=0x7fc7040fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.468+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 shutdown_connections 2024-02-14T15:19:48.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.469+0000 7fc709e59640 1 -- 172.21.15.17:0/3862273322 wait complete. 2024-02-14T15:19:48.480 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-14T15:19:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 -- 172.21.15.17:0/3650870776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 msgr2=0x7f1714100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 --2- 172.21.15.17:0/3650870776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f1714100620 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1708009930 tx=0x7f170802bd10 comp rx=0 tx=0).stop 2024-02-14T15:19:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 -- 172.21.15.17:0/3650870776 shutdown_connections 2024-02-14T15:19:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 --2- 172.21.15.17:0/3650870776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1714105640 0x7f1714105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 --2- 172.21.15.17:0/3650870776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1714100b60 0x7f1714105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 --2- 172.21.15.17:0/3650870776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f1714100620 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 -- 172.21.15.17:0/3650870776 >> 172.21.15.17:0/3650870776 conn(0x7f17140772b0 msgr2=0x7f17140776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 -- 172.21.15.17:0/3650870776 shutdown_connections 2024-02-14T15:19:48.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.615+0000 7f171a583640 1 -- 172.21.15.17:0/3650870776 wait complete. 2024-02-14T15:19:48.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.616+0000 7f171a583640 1 Processor -- start 2024-02-14T15:19:48.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.616+0000 7f171a583640 1 -- start start 2024-02-14T15:19:48.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f171a583640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f171419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f171a583640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1714100b60 0x7f171419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f171a583640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1714105640 0x7f17141a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f171a583640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141a2cd0 con 0x7f1714100240 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f171a583640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141a2e40 con 0x7f1714100b60 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f171a583640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141a3000 con 0x7f1714105640 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f1713fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f171419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f17137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1714100b60 0x7f171419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f1713fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f171419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37892/0 (socket says 172.21.15.17:37892) 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.617+0000 7f1713fff640 1 -- 172.21.15.17:0/3608314640 learned_addr learned my addr 172.21.15.17:0/3608314640 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f1713fff640 1 -- 172.21.15.17:0/3608314640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1714105640 msgr2=0x7f17141a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f1713fff640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1714105640 0x7f17141a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f1713fff640 1 -- 172.21.15.17:0/3608314640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1714100b60 msgr2=0x7f171419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f1713fff640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1714100b60 0x7f171419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f1713fff640 1 -- 172.21.15.17:0/3608314640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708009560 con 0x7f1714100240 2024-02-14T15:19:48.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f1713fff640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f171419de10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f170802c2a0 tx=0x7f17080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:48.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f17117fa640 1 -- 172.21.15.17:0/3608314640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170802dc00 con 0x7f1714100240 2024-02-14T15:19:48.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f17117fa640 1 -- 172.21.15.17:0/3608314640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f170802dd60 con 0x7f1714100240 2024-02-14T15:19:48.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.618+0000 7f17117fa640 1 -- 172.21.15.17:0/3608314640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708003bf0 con 0x7f1714100240 2024-02-14T15:19:48.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.619+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17141a3280 con 0x7f1714100240 2024-02-14T15:19:48.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.619+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17141aab60 con 0x7f1714100240 2024-02-14T15:19:48.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.620+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17140680b0 con 0x7f1714100240 2024-02-14T15:19:48.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.622+0000 7f17117fa640 1 -- 172.21.15.17:0/3608314640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1708003d50 con 0x7f1714100240 2024-02-14T15:19:48.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.623+0000 7f17117fa640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16fc0a2440 0x7f16fc0a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.623+0000 7f17117fa640 1 -- 172.21.15.17:0/3608314640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f17080f0d90 con 0x7f1714100240 2024-02-14T15:19:48.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.627+0000 7f17137fe640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16fc0a2440 0x7f16fc0a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.631+0000 7f17137fe640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16fc0a2440 0x7f16fc0a4900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17140699c0 tx=0x7f1704005f90 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:48.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.632+0000 7f17117fa640 1 -- 172.21.15.17:0/3608314640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17080b9790 con 0x7f1714100240 2024-02-14T15:19:48.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.868+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f171419f160 con 0x7f1714100240 2024-02-14T15:19:48.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.869+0000 7f17117fa640 1 -- 172.21.15.17:0/3608314640 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11318 (secure 0 0 0) 0x7f1708043020 con 0x7f1714100240 2024-02-14T15:19:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:19:48.870 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/763584482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":763584482},{"type":"v1","addr":"172.21.15.17:6835","nonce":763584482}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2485072299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2485072299},{"type":"v1","addr":"172.21.15.17:6837","nonce":2485072299}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14649,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/516562430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":516562430},{"type":"v1","addr":"172.21.15.17:6839","nonce":516562430}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3297312936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3297312936},{"type":"v1","addr":"172.21.15.17:6841","nonce":3297312936}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/277751222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":277751222},{"type":"v1","addr":"172.21.15.50:6833","nonce":277751222}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":26764,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/349944697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":349944697},{"type":"v1","addr":"172.21.15.50:6837","nonce":349944697}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-14T15:00:48.619843+0000","modified":"2024-02-14T15:19:33.893894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24502,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/3474801477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":3474801477},{"type":"v1","addr":"172.21.15.50:6835","nonce":3474801477}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/1787917010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1787917010},{"type":"v1","addr":"172.21.15.76:6835","nonce":1787917010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":77,"addr":"172.21.15.76:6837/3638854935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3638854935},{"type":"v1","addr":"172.21.15.76:6837","nonce":3638854935}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6839/666966898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":666966898},{"type":"v1","addr":"172.21.15.76:6839","nonce":666966898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/916171404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":916171404},{"type":"v1","addr":"172.21.15.50:6839","nonce":916171404}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26947":{"gid":26947,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6833/1295210715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1295210715},{"type":"v1","addr":"172.21.15.76:6833","nonce":1295210715}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-14T15:19:48.872 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2024-02-14T15:19:48.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.870+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16fc0a2440 msgr2=0x7f16fc0a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.870+0000 7f171a583640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16fc0a2440 0x7f16fc0a4900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17140699c0 tx=0x7f1704005f90 comp rx=0 tx=0).stop 2024-02-14T15:19:48.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.870+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 msgr2=0x7f171419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.870+0000 7f171a583640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f171419de10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f170802c2a0 tx=0x7f17080045e0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 shutdown_connections 2024-02-14T15:19:48.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f16fc0a2440 0x7f16fc0a4900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1714105640 0x7f17141a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1714100b60 0x7f171419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 --2- 172.21.15.17:0/3608314640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1714100240 0x7f171419de10 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 >> 172.21.15.17:0/3608314640 conn(0x7f17140772b0 msgr2=0x7f1714102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 shutdown_connections 2024-02-14T15:19:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.871+0000 7f171a583640 1 -- 172.21.15.17:0/3608314640 wait complete. 2024-02-14T15:19:48.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-14T15:19:48.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.984+0000 7f7c25b3e640 1 -- 172.21.15.17:0/2637305369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c20106520 msgr2=0x7f7c2010e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.984+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/2637305369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c20106520 0x7f7c2010e0f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009cd0 tx=0x7f7c1802c0f0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.984+0000 7f7c25b3e640 1 -- 172.21.15.17:0/2637305369 shutdown_connections 2024-02-14T15:19:48.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.984+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/2637305369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c20106520 0x7f7c2010e0f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.984+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/2637305369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c20101e00 0x7f7c20105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.984+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/2637305369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c20101450 0x7f7c20101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 -- 172.21.15.17:0/2637305369 >> 172.21.15.17:0/2637305369 conn(0x7f7c200fccd0 msgr2=0x7f7c200ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:48.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 -- 172.21.15.17:0/2637305369 shutdown_connections 2024-02-14T15:19:48.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 -- 172.21.15.17:0/2637305369 wait complete. 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 Processor -- start 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 -- start start 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c20101450 0x7f7c2019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c20106520 0x7f7c2019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c201975b0 0x7f7c20197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20197fa0 con 0x7f7c201975b0 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20198110 con 0x7f7c20106520 2024-02-14T15:19:48.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.985+0000 7f7c25b3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c201982d0 con 0x7f7c20101450 2024-02-14T15:19:48.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c201975b0 0x7f7c20197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c201975b0 0x7f7c20197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37906/0 (socket says 172.21.15.17:37906) 2024-02-14T15:19:48.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 -- 172.21.15.17:0/3022621756 learned_addr learned my addr 172.21.15.17:0/3022621756 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:48.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1f7fe640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c20101450 0x7f7c2019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1effd640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c20106520 0x7f7c2019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 -- 172.21.15.17:0/3022621756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c20101450 msgr2=0x7f7c2019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c20101450 0x7f7c2019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 -- 172.21.15.17:0/3022621756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c20106520 msgr2=0x7f7c2019d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:48.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c20106520 0x7f7c2019d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:48.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 -- 172.21.15.17:0/3022621756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c18009900 con 0x7f7c201975b0 2024-02-14T15:19:48.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1ffff640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c201975b0 0x7f7c20197a60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7c18037040 tx=0x7f7c18034d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:48.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1cff9640 1 -- 172.21.15.17:0/3022621756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18032ac0 con 0x7f7c201975b0 2024-02-14T15:19:48.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1cff9640 1 -- 172.21.15.17:0/3022621756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c18003e50 con 0x7f7c201975b0 2024-02-14T15:19:48.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c201a42c0 con 0x7f7c201975b0 2024-02-14T15:19:48.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c1cff9640 1 -- 172.21.15.17:0/3022621756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c180406b0 con 0x7f7c201975b0 2024-02-14T15:19:48.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.986+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c201a4740 con 0x7f7c201975b0 2024-02-14T15:19:48.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.987+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c20067820 con 0x7f7c201975b0 2024-02-14T15:19:48.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.988+0000 7f7c1cff9640 1 -- 172.21.15.17:0/3022621756 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7c1800a040 con 0x7f7c201975b0 2024-02-14T15:19:48.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.989+0000 7f7c1cff9640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7c080a2230 0x7f7c080a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:48.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.989+0000 7f7c1f7fe640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7c080a2230 0x7f7c080a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:48.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.989+0000 7f7c1cff9640 1 -- 172.21.15.17:0/3022621756 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f7c180f17a0 con 0x7f7c201975b0 2024-02-14T15:19:48.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.989+0000 7f7c1f7fe640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7c080a2230 0x7f7c080a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400abe0 tx=0x7f7c1400a2d0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:48.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:48.992+0000 7f7c1cff9640 1 -- 172.21.15.17:0/3022621756 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c180ba240 con 0x7f7c201975b0 2024-02-14T15:19:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:48 smithi050 ceph-mon[36512]: pgmap v518: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 464 KiB/s wr, 35 op/s 2024-02-14T15:19:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:48 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3862273322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:19:49.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:48 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3608314640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:19:49.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.177+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c20198ae0 con 0x7f7c201975b0 2024-02-14T15:19:49.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.180+0000 7f7c1cff9640 1 -- 172.21.15.17:0/3022621756 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+25220 (secure 0 0 0) 0x7f7c18044090 con 0x7f7c201975b0 2024-02-14T15:19:49.181 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-02-14T15:19:49.181 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":118,"fsid":"867ff6ea-cb48-11ee-95b9-87774f69a715","created":"2024-02-14T14:52:46.029095+0000","modified":"2024-02-14T15:19:15.817399+0000","last_up_change":"2024-02-14T14:59:58.558516+0000","last_in_change":"2024-02-14T14:59:38.552055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T14:56:08.919729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-14T15:00:47.196882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-14T15:00:47.810743+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-14T15:00:51.802704+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"423af2fd-89fb-4e39-913c-be53474fe012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6803","nonce":1615583912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6805","nonce":1615583912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6809","nonce":1615583912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1615583912},{"type":"v1","addr":"172.21.15.17:6807","nonce":1615583912}]},"public_addr":"172.21.15.17:6803/1615583912","cluster_addr":"172.21.15.17:6805/1615583912","heartbeat_back_addr":"172.21.15.17:6809/1615583912","heartbeat_front_addr":"172.21.15.17:6807/1615583912","state":["exists","up"]},{"osd":1,"uuid":"3f59afa4-e5b3-462c-89a0-e9552f262730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051103540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051103540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051103540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051103540},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051103540}]},"public_addr":"172.21.15.50:6801/2051103540","cluster_addr":"172.21.15.50:6803/2051103540","heartbeat_back_addr":"172.21.15.50:6807/2051103540","heartbeat_front_addr":"172.21.15.50:6805/2051103540","state":["exists","up"]},{"osd":2,"uuid":"0f1d2809-6bb1-46aa-9ba8-dd08192a591b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6801","nonce":2271935501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6803","nonce":2271935501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6807","nonce":2271935501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2271935501},{"type":"v1","addr":"172.21.15.76:6805","nonce":2271935501}]},"public_addr":"172.21.15.76:6801/2271935501","cluster_addr":"172.21.15.76:6803/2271935501","heartbeat_back_addr":"172.21.15.76:6807/2271935501","heartbeat_front_addr":"172.21.15.76:6805/2271935501","state":["exists","up"]},{"osd":3,"uuid":"6c1e9407-028a-4d4b-af15-2f0d9b05669b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6811","nonce":1293297743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6813","nonce":1293297743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6817","nonce":1293297743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1293297743},{"type":"v1","addr":"172.21.15.17:6815","nonce":1293297743}]},"public_addr":"172.21.15.17:6811/1293297743","cluster_addr":"172.21.15.17:6813/1293297743","heartbeat_back_addr":"172.21.15.17:6817/1293297743","heartbeat_front_addr":"172.21.15.17:6815/1293297743","state":["exists","up"]},{"osd":4,"uuid":"445cd928-d6fe-4815-8094-e407331ec2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6809","nonce":4058610796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6811","nonce":4058610796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6815","nonce":4058610796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":4058610796},{"type":"v1","addr":"172.21.15.50:6813","nonce":4058610796}]},"public_addr":"172.21.15.50:6809/4058610796","cluster_addr":"172.21.15.50:6811/4058610796","heartbeat_back_addr":"172.21.15.50:6815/4058610796","heartbeat_front_addr":"172.21.15.50:6813/4058610796","state":["exists","up"]},{"osd":5,"uuid":"f7c1a10c-7507-426e-b91a-9f788f8edd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6809","nonce":75527481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6811","nonce":75527481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6815","nonce":75527481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":75527481},{"type":"v1","addr":"172.21.15.76:6813","nonce":75527481}]},"public_addr":"172.21.15.76:6809/75527481","cluster_addr":"172.21.15.76:6811/75527481","heartbeat_back_addr":"172.21.15.76:6815/75527481","heartbeat_front_addr":"172.21.15.76:6813/75527481","state":["exists","up"]},{"osd":6,"uuid":"837621c6-44f2-4ede-83d8-031db03b8ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6819","nonce":2957627579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6821","nonce":2957627579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6825","nonce":2957627579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":2957627579},{"type":"v1","addr":"172.21.15.17:6823","nonce":2957627579}]},"public_addr":"172.21.15.17:6819/2957627579","cluster_addr":"172.21.15.17:6821/2957627579","heartbeat_back_addr":"172.21.15.17:6825/2957627579","heartbeat_front_addr":"172.21.15.17:6823/2957627579","state":["exists","up"]},{"osd":7,"uuid":"8ea6e493-7d1b-4570-a7d0-2f50d2adf530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6817","nonce":3407611050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6819","nonce":3407611050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6823","nonce":3407611050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3407611050},{"type":"v1","addr":"172.21.15.50:6821","nonce":3407611050}]},"public_addr":"172.21.15.50:6817/3407611050","cluster_addr":"172.21.15.50:6819/3407611050","heartbeat_back_addr":"172.21.15.50:6823/3407611050","heartbeat_front_addr":"172.21.15.50:6821/3407611050","state":["exists","up"]},{"osd":8,"uuid":"e4cf495a-56b2-45b3-9527-0ee4f829e144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6817","nonce":416823437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6819","nonce":416823437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6823","nonce":416823437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":416823437},{"type":"v1","addr":"172.21.15.76:6821","nonce":416823437}]},"public_addr":"172.21.15.76:6817/416823437","cluster_addr":"172.21.15.76:6819/416823437","heartbeat_back_addr":"172.21.15.76:6823/416823437","heartbeat_front_addr":"172.21.15.76:6821/416823437","state":["exists","up"]},{"osd":9,"uuid":"e2b22fd6-b721-41e5-b910-db93fa49a826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6827","nonce":2843045035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6829","nonce":2843045035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6833","nonce":2843045035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2843045035},{"type":"v1","addr":"172.21.15.17:6831","nonce":2843045035}]},"public_addr":"172.21.15.17:6827/2843045035","cluster_addr":"172.21.15.17:6829/2843045035","heartbeat_back_addr":"172.21.15.17:6833/2843045035","heartbeat_front_addr":"172.21.15.17:6831/2843045035","state":["exists","up"]},{"osd":10,"uuid":"d4f225cb-2077-48f1-ab76-3e94e0566653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6825","nonce":156762867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6827","nonce":156762867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6831","nonce":156762867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":156762867},{"type":"v1","addr":"172.21.15.50:6829","nonce":156762867}]},"public_addr":"172.21.15.50:6825/156762867","cluster_addr":"172.21.15.50:6827/156762867","heartbeat_back_addr":"172.21.15.50:6831/156762867","heartbeat_front_addr":"172.21.15.50:6829/156762867","state":["exists","up"]},{"osd":11,"uuid":"982cfd49-b3f7-4e98-a92a-7927c0aee762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6825","nonce":3113998070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6827","nonce":3113998070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6831","nonce":3113998070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3113998070},{"type":"v1","addr":"172.21.15.76:6829","nonce":3113998070}]},"public_addr":"172.21.15.76:6825/3113998070","cluster_addr":"172.21.15.76:6827/3113998070","heartbeat_back_addr":"172.21.15.76:6831/3113998070","heartbeat_front_addr":"172.21.15.76:6829/3113998070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:17.957517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:55:42.404149+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:06.154938+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:32.153579+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:56:57.216333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:21.923448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:57:47.834165+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:13.617377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:58:38.539265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:05.058351+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:31.017776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T14:59:57.154700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/420003070":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/2092370502":"2024-02-15T15:03:24.930590+0000","172.21.15.17:6800/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/3371941300":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6800/2869822133":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3241397077":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2755965102":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/1117796267":"2024-02-15T14:53:15.465600+0000","172.21.15.17:6801/1044675107":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/1369411962":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/3756063081":"2024-02-15T14:53:42.569888+0000","172.21.15.17:0/2319986884":"2024-02-15T14:53:15.465600+0000","172.21.15.17:0/3179868836":"2024-02-15T14:53:42.569888+0000","172.21.15.17:6800/3796717329":"2024-02-15T15:03:24.930590+0000","172.21.15.17:6801/3796717329":"2024-02-15T15:03:24.930590+0000","172.21.15.17:0/3775022683":"2024-02-15T15:03:24.930590+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T15:19:49.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7c080a2230 msgr2=0x7f7c080a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7c080a2230 0x7f7c080a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400abe0 tx=0x7f7c1400a2d0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c201975b0 msgr2=0x7f7c20197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c201975b0 0x7f7c20197a60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7c18037040 tx=0x7f7c18034d30 comp rx=0 tx=0).stop 2024-02-14T15:19:49.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 shutdown_connections 2024-02-14T15:19:49.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f7c080a2230 0x7f7c080a46f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c201975b0 0x7f7c20197a60 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c20106520 0x7f7c2019d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 --2- 172.21.15.17:0/3022621756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c20101450 0x7f7c2019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 >> 172.21.15.17:0/3022621756 conn(0x7f7c200fccd0 msgr2=0x7f7c200fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:49.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 shutdown_connections 2024-02-14T15:19:49.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.182+0000 7f7c25b3e640 1 -- 172.21.15.17:0/3022621756 wait complete. 2024-02-14T15:19:49.193 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:48 smithi017 ceph-mon[32549]: pgmap v518: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 464 KiB/s wr, 35 op/s 2024-02-14T15:19:49.193 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:48 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3862273322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:19:49.193 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:48 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3608314640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:19:49.196 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-14T15:19:49.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:48 smithi076 ceph-mon[37057]: pgmap v518: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 464 KiB/s wr, 35 op/s 2024-02-14T15:19:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:48 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3862273322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:19:49.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:48 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3608314640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-14T15:19:49.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 -- 172.21.15.17:0/1814276830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102450 msgr2=0x7f1a64102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/1814276830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102450 0x7f1a64102830 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009900 tx=0x7f1a5802bcd0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 -- 172.21.15.17:0/1814276830 shutdown_connections 2024-02-14T15:19:49.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/1814276830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6410b820 0x7f1a6410dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/1814276830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a64102d70 0x7f1a6410b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/1814276830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102450 0x7f1a64102830 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 -- 172.21.15.17:0/1814276830 >> 172.21.15.17:0/1814276830 conn(0x7f1a640772e0 msgr2=0x7f1a640776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:49.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 -- 172.21.15.17:0/1814276830 shutdown_connections 2024-02-14T15:19:49.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 -- 172.21.15.17:0/1814276830 wait complete. 2024-02-14T15:19:49.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 Processor -- start 2024-02-14T15:19:49.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.300+0000 7f1a6b77a640 1 -- start start 2024-02-14T15:19:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a6b77a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a64102450 0x7f1a6419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a6b77a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102d70 0x7f1a6419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a6b77a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6410b820 0x7f1a64197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a6b77a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64197df0 con 0x7f1a64102d70 2024-02-14T15:19:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a6b77a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64197f60 con 0x7f1a64102450 2024-02-14T15:19:49.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a6b77a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64198120 con 0x7f1a6410b820 2024-02-14T15:19:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102d70 0x7f1a6419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102d70 0x7f1a6419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37930/0 (socket says 172.21.15.17:37930) 2024-02-14T15:19:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a69516640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a64102450 0x7f1a6419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 -- 172.21.15.17:0/2516685233 learned_addr learned my addr 172.21.15.17:0/2516685233 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a69d17640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6410b820 0x7f1a64197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 -- 172.21.15.17:0/2516685233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6410b820 msgr2=0x7f1a64197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6410b820 0x7f1a64197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 -- 172.21.15.17:0/2516685233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a64102450 msgr2=0x7f1a6419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a64102450 0x7f1a6419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 -- 172.21.15.17:0/2516685233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a58009560 con 0x7f1a64102d70 2024-02-14T15:19:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a69516640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a64102450 0x7f1a6419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T15:19:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a68d15640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102d70 0x7f1a6419dae0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e9f0 tx=0x7f1a6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a567fc640 1 -- 172.21.15.17:0/2516685233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60009700 con 0x7f1a64102d70 2024-02-14T15:19:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a64198400 con 0x7f1a64102d70 2024-02-14T15:19:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a567fc640 1 -- 172.21.15.17:0/2516685233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6000cd80 con 0x7f1a64102d70 2024-02-14T15:19:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.301+0000 7f1a567fc640 1 -- 172.21.15.17:0/2516685233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60018920 con 0x7f1a64102d70 2024-02-14T15:19:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.302+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a641a4520 con 0x7f1a64102d70 2024-02-14T15:19:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.303+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a64068d30 con 0x7f1a64102d70 2024-02-14T15:19:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.304+0000 7f1a567fc640 1 -- 172.21.15.17:0/2516685233 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1a60018a80 con 0x7f1a64102d70 2024-02-14T15:19:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.304+0000 7f1a567fc640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a500a2230 0x7f1a500a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.304+0000 7f1a69516640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a500a2230 0x7f1a500a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.304+0000 7f1a567fc640 1 -- 172.21.15.17:0/2516685233 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f1a60014070 con 0x7f1a64102d70 2024-02-14T15:19:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.305+0000 7f1a69516640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a500a2230 0x7f1a500a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a58006010 tx=0x7f1a58005e40 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:49.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.313+0000 7f1a567fc640 1 -- 172.21.15.17:0/2516685233 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a60052d20 con 0x7f1a64102d70 2024-02-14T15:19:49.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.550+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1a64106910 con 0x7f1a64102d70 2024-02-14T15:19:49.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.552+0000 7f1a567fc640 1 -- 172.21.15.17:0/2516685233 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1a60097150 con 0x7f1a64102d70 2024-02-14T15:19:49.553 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-14T15:19:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.556+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a500a2230 msgr2=0x7f1a500a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.556+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a500a2230 0x7f1a500a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a58006010 tx=0x7f1a58005e40 comp rx=0 tx=0).stop 2024-02-14T15:19:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.556+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102d70 msgr2=0x7f1a6419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.556+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102d70 0x7f1a6419dae0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e9f0 tx=0x7f1a6000eeb0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 shutdown_connections 2024-02-14T15:19:49.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f1a500a2230 0x7f1a500a46f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a64102d70 0x7f1a6419dae0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6410b820 0x7f1a64197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 --2- 172.21.15.17:0/2516685233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a64102450 0x7f1a6419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 >> 172.21.15.17:0/2516685233 conn(0x7f1a640772e0 msgr2=0x7f1a640fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:49.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 shutdown_connections 2024-02-14T15:19:49.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.557+0000 7f1a6b77a640 1 -- 172.21.15.17:0/2516685233 wait complete. 2024-02-14T15:19:49.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-14T15:19:49.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.680+0000 7feb740d4640 1 -- 172.21.15.17:0/236865846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c101430 msgr2=0x7feb6c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.680+0000 7feb740d4640 1 --2- 172.21.15.17:0/236865846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c101430 0x7feb6c101810 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009930 tx=0x7feb5c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:49.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.680+0000 7feb740d4640 1 -- 172.21.15.17:0/236865846 shutdown_connections 2024-02-14T15:19:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.680+0000 7feb740d4640 1 --2- 172.21.15.17:0/236865846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb6c106500 0x7feb6c10e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.680+0000 7feb740d4640 1 --2- 172.21.15.17:0/236865846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb6c101de0 0x7feb6c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.680+0000 7feb740d4640 1 --2- 172.21.15.17:0/236865846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c101430 0x7feb6c101810 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.680+0000 7feb740d4640 1 -- 172.21.15.17:0/236865846 >> 172.21.15.17:0/236865846 conn(0x7feb6c0fccd0 msgr2=0x7feb6c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 -- 172.21.15.17:0/236865846 shutdown_connections 2024-02-14T15:19:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 -- 172.21.15.17:0/236865846 wait complete. 2024-02-14T15:19:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 Processor -- start 2024-02-14T15:19:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 -- start start 2024-02-14T15:19:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb6c101430 0x7feb6c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb6c106500 0x7feb6c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c1976e0 0x7feb6c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1981b0 con 0x7feb6c1976e0 2024-02-14T15:19:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c198320 con 0x7feb6c106500 2024-02-14T15:19:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb740d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1984e0 con 0x7feb6c101430 2024-02-14T15:19:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.681+0000 7feb72671640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c1976e0 0x7feb6c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c1976e0 0x7feb6c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37942/0 (socket says 172.21.15.17:37942) 2024-02-14T15:19:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 -- 172.21.15.17:0/3329193876 learned_addr learned my addr 172.21.15.17:0/3329193876 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb71e70640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb6c101430 0x7feb6c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb7166f640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb6c106500 0x7feb6c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 -- 172.21.15.17:0/3329193876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb6c101430 msgr2=0x7feb6c19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb6c101430 0x7feb6c19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 -- 172.21.15.17:0/3329193876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb6c106500 msgr2=0x7feb6c19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb6c106500 0x7feb6c19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 -- 172.21.15.17:0/3329193876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c009560 con 0x7feb6c1976e0 2024-02-14T15:19:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb72671640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c1976e0 0x7feb6c197b60 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7feb64009fd0 tx=0x7feb6400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb62ffd640 1 -- 172.21.15.17:0/3329193876 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6400bea0 con 0x7feb6c1976e0 2024-02-14T15:19:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb62ffd640 1 -- 172.21.15.17:0/3329193876 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb64009940 con 0x7feb6c1976e0 2024-02-14T15:19:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6c1a4430 con 0x7feb6c1976e0 2024-02-14T15:19:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb62ffd640 1 -- 172.21.15.17:0/3329193876 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640106c0 con 0x7feb6c1976e0 2024-02-14T15:19:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.682+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6c1a48c0 con 0x7feb6c1976e0 2024-02-14T15:19:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.683+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6c102900 con 0x7feb6c1976e0 2024-02-14T15:19:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.684+0000 7feb62ffd640 1 -- 172.21.15.17:0/3329193876 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feb64004240 con 0x7feb6c1976e0 2024-02-14T15:19:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.685+0000 7feb62ffd640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feb580a2230 0x7feb580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.685+0000 7feb71e70640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feb580a2230 0x7feb580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.685+0000 7feb62ffd640 1 -- 172.21.15.17:0/3329193876 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9645+0+0 (secure 0 0 0) 0x7feb640ce2e0 con 0x7feb6c1976e0 2024-02-14T15:19:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.685+0000 7feb71e70640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feb580a2230 0x7feb580a46f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb5c006010 tx=0x7feb5c000bc0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.689+0000 7feb62ffd640 1 -- 172.21.15.17:0/3329193876 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb64096ab0 con 0x7feb6c1976e0 2024-02-14T15:19:49.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32545]: 2024-02-14T15:19:49.551+0000 7ff3e405c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-14T15:19:49.886 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f[67290]: ignoring --setuser ceph since I am not root 2024-02-14T15:19:49.886 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f[67290]: ignoring --setgroup ceph since I am not root 2024-02-14T15:19:49.886 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f[67290]: starting mds.f at 2024-02-14T15:19:49.887 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b[67204]: ignoring --setuser ceph since I am not root 2024-02-14T15:19:49.887 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b[67204]: ignoring --setgroup ceph since I am not root 2024-02-14T15:19:49.887 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mds[67208]: -- 172.21.15.50:0/3774610995 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640b9f1f520 con 0x5640b9ef4800 2024-02-14T15:19:49.887 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b[67204]: starting mds.b at 2024-02-14T15:19:49.888 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: ignoring --setuser ceph since I am not root 2024-02-14T15:19:49.888 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: ignoring --setgroup ceph since I am not root 2024-02-14T15:19:49.888 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mds[67606]: -- 172.21.15.76:0/621336953 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b279607520 con 0x55b2795dd000 2024-02-14T15:19:49.888 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: starting mds.c at 2024-02-14T15:19:49.889 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e[66591]: ignoring --setuser ceph since I am not root 2024-02-14T15:19:49.889 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e[66591]: ignoring --setgroup ceph since I am not root 2024-02-14T15:19:49.889 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mds[66595]: -- 172.21.15.50:0/1971178564 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ebd64c5520 con 0x55ebd649a800 2024-02-14T15:19:49.889 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e[66591]: starting mds.e at 2024-02-14T15:19:49.890 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i[67909]: ignoring --setuser ceph since I am not root 2024-02-14T15:19:49.890 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i[67909]: ignoring --setgroup ceph since I am not root 2024-02-14T15:19:49.890 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mds[67913]: -- 172.21.15.76:0/858489945 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556a6169d520 con 0x556a61673000 2024-02-14T15:19:49.890 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i[67909]: starting mds.i at 2024-02-14T15:19:49.890 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[73142]: ignoring --setuser ceph since I am not root 2024-02-14T15:19:49.891 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[73142]: ignoring --setgroup ceph since I am not root 2024-02-14T15:19:49.891 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mds[73146]: -- 172.21.15.76:0/4195381826 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ea18381080 con 0x55ea1834b000 2024-02-14T15:19:49.891 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mds[73146]: -- 172.21.15.76:0/4195381826 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea18375520 con 0x55ea1834b000 2024-02-14T15:19:49.891 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[73142]: starting mds.l at 2024-02-14T15:19:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:49.918+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feb6c109730 con 0x7feb6c1976e0 2024-02-14T15:19:50.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:19:50.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3022621756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:50.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2516685233' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-14T15:19:50.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-14T15:19:50.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-14T15:19:50.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-14T15:19:50.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/2516685233' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-14T15:19:50.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: osdmap e119: 12 total, 12 up, 12 in 2024-02-14T15:19:50.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-14T15:19:50.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:19:50.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:19:50.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:49 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:19:50.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:19:50.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3022621756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:50.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2516685233' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-14T15:19:50.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-14T15:19:50.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-14T15:19:50.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-14T15:19:50.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/2516685233' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-14T15:19:50.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: osdmap e119: 12 total, 12 up, 12 in 2024-02-14T15:19:50.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-14T15:19:50.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:19:50.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:19:50.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:49 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:19:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T15:19:50.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3022621756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T15:19:50.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2516685233' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-14T15:19:50.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-14T15:19:50.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-14T15:19:50.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-02-14T15:19:50.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/2516685233' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-14T15:19:50.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: osdmap e119: 12 total, 12 up, 12 in 2024-02-14T15:19:50.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-14T15:19:50.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T15:19:50.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T15:19:50.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:49 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' 2024-02-14T15:19:50.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.554+0000 7feb62ffd640 1 -- 172.21.15.17:0/3329193876 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7feb6409a170 con 0x7feb6c1976e0 2024-02-14T15:19:50.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.556+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feb580a2230 msgr2=0x7feb580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.556+0000 7feb740d4640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feb580a2230 0x7feb580a46f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb5c006010 tx=0x7feb5c000bc0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.556+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c1976e0 msgr2=0x7feb6c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.556+0000 7feb740d4640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c1976e0 0x7feb6c197b60 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7feb64009fd0 tx=0x7feb6400b260 comp rx=0 tx=0).stop 2024-02-14T15:19:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 shutdown_connections 2024-02-14T15:19:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7feb580a2230 0x7feb580a46f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb6c1976e0 0x7feb6c197b60 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb6c106500 0x7feb6c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 --2- 172.21.15.17:0/3329193876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb6c101430 0x7feb6c19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 >> 172.21.15.17:0/3329193876 conn(0x7feb6c0fccd0 msgr2=0x7feb6c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 shutdown_connections 2024-02-14T15:19:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.557+0000 7feb740d4640 1 -- 172.21.15.17:0/3329193876 wait complete. 2024-02-14T15:19:50.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-14T15:19:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 -- 172.21.15.17:0/2465232911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 msgr2=0x7f5420069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 --2- 172.21.15.17:0/2465232911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f5420069fc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f541c009930 tx=0x7f541c02bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 -- 172.21.15.17:0/2465232911 shutdown_connections 2024-02-14T15:19:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 --2- 172.21.15.17:0/2465232911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420106570 0x7f542010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 --2- 172.21.15.17:0/2465232911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542006a590 0x7f5420105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 --2- 172.21.15.17:0/2465232911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f5420069fc0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 -- 172.21.15.17:0/2465232911 >> 172.21.15.17:0/2465232911 conn(0x7f5420077260 msgr2=0x7f5420077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:50.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 -- 172.21.15.17:0/2465232911 shutdown_connections 2024-02-14T15:19:50.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.685+0000 7f542766f640 1 -- 172.21.15.17:0/2465232911 wait complete. 2024-02-14T15:19:50.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 Processor -- start 2024-02-14T15:19:50.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 -- start start 2024-02-14T15:19:50.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f542019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:50.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420106570 0x7f542019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:50.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54201976b0 0x7f5420197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:50.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420198180 con 0x7f5420069be0 2024-02-14T15:19:50.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54201982f0 con 0x7f5420106570 2024-02-14T15:19:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542766f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54201984b0 con 0x7f54201976b0 2024-02-14T15:19:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f542019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f542019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56402/0 (socket says 172.21.15.17:56402) 2024-02-14T15:19:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 -- 172.21.15.17:0/1482399658 learned_addr learned my addr 172.21.15.17:0/1482399658 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f5425c0c640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54201976b0 0x7f5420197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f5424c0a640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420106570 0x7f542019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 -- 172.21.15.17:0/1482399658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54201976b0 msgr2=0x7f5420197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54201976b0 0x7f5420197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 -- 172.21.15.17:0/1482399658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420106570 msgr2=0x7f542019db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420106570 0x7f542019db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f542540b640 1 -- 172.21.15.17:0/1482399658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c009560 con 0x7f5420069be0 2024-02-14T15:19:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.686+0000 7f5424c0a640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420106570 0x7f542019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T15:19:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.687+0000 7f542540b640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f542019d5f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f541c009230 tx=0x7f541c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.687+0000 7f54167fc640 1 -- 172.21.15.17:0/1482399658 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c004350 con 0x7f5420069be0 2024-02-14T15:19:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.687+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54201a43f0 con 0x7f5420069be0 2024-02-14T15:19:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.687+0000 7f54167fc640 1 -- 172.21.15.17:0/1482399658 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f541c0044b0 con 0x7f5420069be0 2024-02-14T15:19:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.687+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54201a4920 con 0x7f5420069be0 2024-02-14T15:19:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.687+0000 7f54167fc640 1 -- 172.21.15.17:0/1482399658 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c003d10 con 0x7f5420069be0 2024-02-14T15:19:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.688+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5420102900 con 0x7f5420069be0 2024-02-14T15:19:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.689+0000 7f54167fc640 1 -- 172.21.15.17:0/1482399658 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f541c004df0 con 0x7f5420069be0 2024-02-14T15:19:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.689+0000 7f54167fc640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f540c0a2230 0x7f540c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.689+0000 7f5424c0a640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f540c0a2230 0x7f540c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.689+0000 7f54167fc640 1 -- 172.21.15.17:0/1482399658 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9645+0+0 (secure 0 0 0) 0x7f541c0f0620 con 0x7f5420069be0 2024-02-14T15:19:50.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.690+0000 7f5424c0a640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f540c0a2230 0x7f540c0a46f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5410005d00 tx=0x7f5410005ed0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:50.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.694+0000 7f54167fc640 1 -- 172.21.15.17:0/1482399658 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f541c0b8f00 con 0x7f5420069be0 2024-02-14T15:19:50.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:50.876+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5420109730 con 0x7f5420069be0 2024-02-14T15:19:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: pgmap v519: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 151 KiB/s wr, 17 op/s 2024-02-14T15:19:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3329193876' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T15:19:51.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: Cluster is now healthy 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3329193876' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6834/3100863911,v1:172.21.15.50:6838/3100863911] up:boot 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.50:6835/1543058142,v1:172.21.15.50:6839/1543058142] up:boot 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6837/4111243292,v1:172.21.15.76:6839/4111243292] up:boot 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6836/641352355,v1:172.21.15.76:6838/641352355] up:boot 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6832/3769481068,v1:172.21.15.76:6834/3769481068] up:boot 2024-02-14T15:19:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: mds.? [v2:172.21.15.76:6833/854024274,v1:172.21.15.76:6835/854024274] up:boot 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: fsmap 12 up:standby 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:19:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:50 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1482399658' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: pgmap v519: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 151 KiB/s wr, 17 op/s 2024-02-14T15:19:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3329193876' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T15:19:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-14T15:19:51.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: Cluster is now healthy 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3329193876' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6834/3100863911,v1:172.21.15.50:6838/3100863911] up:boot 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.50:6835/1543058142,v1:172.21.15.50:6839/1543058142] up:boot 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6837/4111243292,v1:172.21.15.76:6839/4111243292] up:boot 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6836/641352355,v1:172.21.15.76:6838/641352355] up:boot 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6832/3769481068,v1:172.21.15.76:6834/3769481068] up:boot 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: mds.? [v2:172.21.15.76:6833/854024274,v1:172.21.15.76:6835/854024274] up:boot 2024-02-14T15:19:51.235 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: fsmap 12 up:standby 2024-02-14T15:19:51.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:19:51.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:19:51.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:19:51.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:19:51.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:19:51.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:19:51.236 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:50 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1482399658' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: pgmap v519: 129 pgs: 129 active+clean; 850 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 151 KiB/s wr, 17 op/s 2024-02-14T15:19:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3329193876' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T15:19:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-14T15:19:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-14T15:19:51.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: Cluster is now healthy 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3329193876' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6834/3100863911,v1:172.21.15.50:6838/3100863911] up:boot 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.50:6835/1543058142,v1:172.21.15.50:6839/1543058142] up:boot 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6837/4111243292,v1:172.21.15.76:6839/4111243292] up:boot 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6836/641352355,v1:172.21.15.76:6838/641352355] up:boot 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6832/3769481068,v1:172.21.15.76:6834/3769481068] up:boot 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: mds.? [v2:172.21.15.76:6833/854024274,v1:172.21.15.76:6835/854024274] up:boot 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: fsmap 12 up:standby 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-14T15:19:51.267 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-14T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-14T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-14T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-14T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='mgr.15681 172.21.15.17:0/257207580' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-14T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:50 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1482399658' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:51.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.830+0000 7f54167fc640 1 -- 172.21.15.17:0/1482399658 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7f541c0bc5c0 con 0x7f5420069be0 2024-02-14T15:19:51.831 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-02-14T15:19:51.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f540c0a2230 msgr2=0x7f540c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:51.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f540c0a2230 0x7f540c0a46f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5410005d00 tx=0x7f5410005ed0 comp rx=0 tx=0).stop 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 msgr2=0x7f542019d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f542019d5f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f541c009230 tx=0x7f541c02c6c0 comp rx=0 tx=0).stop 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 shutdown_connections 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f540c0a2230 0x7f540c0a46f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54201976b0 0x7f5420197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5420106570 0x7f542019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 --2- 172.21.15.17:0/1482399658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5420069be0 0x7f542019d5f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.834+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 >> 172.21.15.17:0/1482399658 conn(0x7f5420077260 msgr2=0x7f54200fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.835+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 shutdown_connections 2024-02-14T15:19:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:51.835+0000 7f542766f640 1 -- 172.21.15.17:0/1482399658 wait complete. 2024-02-14T15:19:51.854 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 -- 172.21.15.17:0/4176731166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100a80 msgr2=0x7f92c8104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 --2- 172.21.15.17:0/4176731166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100a80 0x7f92c8104f40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009900 tx=0x7f92b002bd10 comp rx=0 tx=0).stop 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 -- 172.21.15.17:0/4176731166 shutdown_connections 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 --2- 172.21.15.17:0/4176731166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100a80 0x7f92c8104f40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 --2- 172.21.15.17:0/4176731166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c8105560 0x7f92c81059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 --2- 172.21.15.17:0/4176731166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c8100160 0x7f92c8100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 -- 172.21.15.17:0/4176731166 >> 172.21.15.17:0/4176731166 conn(0x7f92c80fbb30 msgr2=0x7f92c80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 -- 172.21.15.17:0/4176731166 shutdown_connections 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.010+0000 7f92cd348640 1 -- 172.21.15.17:0/4176731166 wait complete. 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 Processor -- start 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 -- start start 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100160 0x7f92c819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c8100a80 0x7f92c819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c8105560 0x7f92c81a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c81a2d00 con 0x7f92c8100160 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c81a2e70 con 0x7f92c8105560 2024-02-14T15:19:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92cd348640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c81a3030 con 0x7f92c8100a80 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92c6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100160 0x7f92c819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92c6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100160 0x7f92c819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56420/0 (socket says 172.21.15.17:56420) 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92c6ffd640 1 -- 172.21.15.17:0/3418921580 learned_addr learned my addr 172.21.15.17:0/3418921580 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92c77fe640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c8105560 0x7f92c81a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.011+0000 7f92bffff640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c8100a80 0x7f92c819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c6ffd640 1 -- 172.21.15.17:0/3418921580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c8100a80 msgr2=0x7f92c819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c6ffd640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c8100a80 0x7f92c819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c6ffd640 1 -- 172.21.15.17:0/3418921580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c8105560 msgr2=0x7f92c81a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c6ffd640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c8105560 0x7f92c81a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c6ffd640 1 -- 172.21.15.17:0/3418921580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0009560 con 0x7f92c8100160 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92bffff640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c8100a80 0x7f92c819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c6ffd640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100160 0x7f92c819de40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f92b800d950 tx=0x7f92b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c4ff9640 1 -- 172.21.15.17:0/3418921580 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8009c30 con 0x7f92c8100160 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c81a3310 con 0x7f92c8100160 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c4ff9640 1 -- 172.21.15.17:0/3418921580 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92b8009d90 con 0x7f92c8100160 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92c4ff9640 1 -- 172.21.15.17:0/3418921580 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b800f7e0 con 0x7f92c8100160 2024-02-14T15:19:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.012+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c81aab90 con 0x7f92c8100160 2024-02-14T15:19:52.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.013+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c8068500 con 0x7f92c8100160 2024-02-14T15:19:52.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.014+0000 7f92c4ff9640 1 -- 172.21.15.17:0/3418921580 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f92b8010ad0 con 0x7f92c8100160 2024-02-14T15:19:52.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.014+0000 7f92c4ff9640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f92b40a2230 0x7f92b40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:52.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.015+0000 7f92c4ff9640 1 -- 172.21.15.17:0/3418921580 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9091+0+0 (secure 0 0 0) 0x7f92b80cdf40 con 0x7f92c8100160 2024-02-14T15:19:52.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.015+0000 7f92bffff640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f92b40a2230 0x7f92b40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:52.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.015+0000 7f92bffff640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f92b40a2230 0x7f92b40a46f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f92c8104540 tx=0x7f92b002c650 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:52.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.018+0000 7f92c4ff9640 1 -- 172.21.15.17:0/3418921580 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92b8096990 con 0x7f92c8100160 2024-02-14T15:19:52.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.199+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92c81179b0 con 0x7f92c8100160 2024-02-14T15:19:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.888+0000 7f92c4ff9640 1 -- 172.21.15.17:0/3418921580 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) v1 ==== 169+0+0 (secure 0 0 0) 0x7f92b80200d0 con 0x7f92c8100160 2024-02-14T15:19:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-02-14T15:19:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.892+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f92b40a2230 msgr2=0x7f92b40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:52.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.892+0000 7f92cd348640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f92b40a2230 0x7f92b40a46f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f92c8104540 tx=0x7f92b002c650 comp rx=0 tx=0).stop 2024-02-14T15:19:52.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.892+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100160 msgr2=0x7f92c819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:52.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.892+0000 7f92cd348640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100160 0x7f92c819de40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f92b800d950 tx=0x7f92b800de10 comp rx=0 tx=0).stop 2024-02-14T15:19:52.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.892+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 shutdown_connections 2024-02-14T15:19:52.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.892+0000 7f92cd348640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f92b40a2230 0x7f92b40a46f0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f92c8104540 tx=0x7f92b002c650 comp rx=0 tx=0).stop 2024-02-14T15:19:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.893+0000 7f92cd348640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c8105560 0x7f92c81a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.893+0000 7f92cd348640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c8100a80 0x7f92c819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.893+0000 7f92cd348640 1 --2- 172.21.15.17:0/3418921580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92c8100160 0x7f92c819de40 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.893+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 >> 172.21.15.17:0/3418921580 conn(0x7f92c80fbb30 msgr2=0x7f92c80fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.893+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 shutdown_connections 2024-02-14T15:19:52.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:52.893+0000 7f92cd348640 1 -- 172.21.15.17:0/3418921580 wait complete. 2024-02-14T15:19:52.910 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-02-14T15:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3489958978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04069c20 msgr2=0x7f9b0406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3489958978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04069c20 0x7f9b0406a000 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009930 tx=0x7f9af402bc30 comp rx=0 tx=0).stop 2024-02-14T15:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3489958978 shutdown_connections 2024-02-14T15:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3489958978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b04106570 0x7f9b0410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3489958978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b0406a5d0 0x7f9b04105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3489958978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04069c20 0x7f9b0406a000 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3489958978 >> 172.21.15.17:0/3489958978 conn(0x7f9b040771d0 msgr2=0x7f9b040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3489958978 shutdown_connections 2024-02-14T15:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.032+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3489958978 wait complete. 2024-02-14T15:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 Processor -- start 2024-02-14T15:19:53.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 -- start start 2024-02-14T15:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b04069c20 0x7f9b0419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b04106570 0x7f9b0419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04197670 0x7f9b04197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04198140 con 0x7f9b04197670 2024-02-14T15:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041982b0 con 0x7f9b04069c20 2024-02-14T15:19:53.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b0a55c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04198470 con 0x7f9b04106570 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04197670 0x7f9b04197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04197670 0x7f9b04197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56442/0 (socket says 172.21.15.17:56442) 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 -- 172.21.15.17:0/3093803181 learned_addr learned my addr 172.21.15.17:0/3093803181 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b03fff640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b04069c20 0x7f9b0419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b037fe640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b04106570 0x7f9b0419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 -- 172.21.15.17:0/3093803181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b04106570 msgr2=0x7f9b0419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b04106570 0x7f9b0419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 -- 172.21.15.17:0/3093803181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b04069c20 msgr2=0x7f9b0419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:53.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b04069c20 0x7f9b0419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.033+0000 7f9b08af9640 1 -- 172.21.15.17:0/3093803181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4009560 con 0x7f9b04197670 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.034+0000 7f9b08af9640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04197670 0x7f9b04197af0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00eb10 tx=0x7f9afc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.034+0000 7f9b017fa640 1 -- 172.21.15.17:0/3093803181 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc004b20 con 0x7f9b04197670 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.034+0000 7f9b017fa640 1 -- 172.21.15.17:0/3093803181 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9afc004c80 con 0x7f9b04197670 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.034+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b041a4410 con 0x7f9b04197670 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.034+0000 7f9b017fa640 1 -- 172.21.15.17:0/3093803181 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc010650 con 0x7f9b04197670 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.034+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b041a48a0 con 0x7f9b04197670 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.035+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b04102900 con 0x7f9b04197670 2024-02-14T15:19:53.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.036+0000 7f9b017fa640 1 -- 172.21.15.17:0/3093803181 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9afc005020 con 0x7f9b04197670 2024-02-14T15:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.036+0000 7f9b017fa640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9aec0a2230 0x7f9aec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T15:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.037+0000 7f9b03fff640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9aec0a2230 0x7f9aec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T15:19:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.037+0000 7f9b017fa640 1 -- 172.21.15.17:0/3093803181 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8553+0+0 (secure 0 0 0) 0x7f9afc014070 con 0x7f9b04197670 2024-02-14T15:19:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.037+0000 7f9b03fff640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9aec0a2230 0x7f9aec0a46f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005ab0 tx=0x7f9af4005fd0 comp rx=0 tx=0).ready entity=mgr.15681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T15:19:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.041+0000 7f9b017fa640 1 -- 172.21.15.17:0/3093803181 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9afc0972e0 con 0x7f9b04197670 2024-02-14T15:19:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:52 smithi050 ceph-mon[36512]: pgmap v521: 129 pgs: 129 active+clean; 848 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 244 KiB/s wr, 21 op/s 2024-02-14T15:19:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:52 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/1482399658' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T15:19:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:52 smithi050 ceph-mon[36512]: osdmap e120: 12 total, 12 up, 12 in 2024-02-14T15:19:53.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:52 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3418921580' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:53.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.222+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9b041097d0 con 0x7f9b04197670 2024-02-14T15:19:53.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:52 smithi076 ceph-mon[37057]: pgmap v521: 129 pgs: 129 active+clean; 848 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 244 KiB/s wr, 21 op/s 2024-02-14T15:19:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:52 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/1482399658' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T15:19:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:52 smithi076 ceph-mon[37057]: osdmap e120: 12 total, 12 up, 12 in 2024-02-14T15:19:53.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:52 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3418921580' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:52 smithi017 ceph-mon[32549]: pgmap v521: 129 pgs: 129 active+clean; 848 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 244 KiB/s wr, 21 op/s 2024-02-14T15:19:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:52 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/1482399658' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T15:19:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:52 smithi017 ceph-mon[32549]: osdmap e120: 12 total, 12 up, 12 in 2024-02-14T15:19:53.266 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:52 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3418921580' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:53.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.894+0000 7f9b017fa640 1 -- 172.21.15.17:0/3093803181 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v122) v1 ==== 178+0+0 (secure 0 0 0) 0x7f9afc09a9a0 con 0x7f9b04197670 2024-02-14T15:19:53.895 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' removed 2024-02-14T15:19:53.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.897+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9aec0a2230 msgr2=0x7f9aec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.897+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9aec0a2230 0x7f9aec0a46f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005ab0 tx=0x7f9af4005fd0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.897+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04197670 msgr2=0x7f9b04197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.897+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04197670 0x7f9b04197af0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00eb10 tx=0x7f9afc00c2d0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 shutdown_connections 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:6800/877978339,v1:172.21.15.17:6801/877978339] conn(0x7f9aec0a2230 0x7f9aec0a46f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b04197670 0x7f9b04197af0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b04106570 0x7f9b0419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 --2- 172.21.15.17:0/3093803181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b04069c20 0x7f9b0419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T15:19:53.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 >> 172.21.15.17:0/3093803181 conn(0x7f9b040771d0 msgr2=0x7f9b040fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T15:19:53.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 shutdown_connections 2024-02-14T15:19:53.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-02-14T15:19:53.898+0000 7f9b0a55c640 1 -- 172.21.15.17:0/3093803181 wait complete. 2024-02-14T15:19:53.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T15:19:53.957 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T15:19:53.994 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T15:19:54.124 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-14T15:19:54.125 DEBUG:teuthology.orchestra.run.smithi017:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T15:19:54.139 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T15:19:54.156 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T15:19:54.171 INFO:tasks.cephadm:Stopping all daemons... 2024-02-14T15:19:54.171 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-14T15:19:54.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a 2024-02-14T15:19:54.202 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:53 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3418921580' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T15:19:54.202 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:53 smithi017 ceph-mon[32549]: osdmap e121: 12 total, 12 up, 12 in 2024-02-14T15:19:54.202 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:53 smithi017 ceph-mon[32549]: from='client.? 172.21.15.17:0/3093803181' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:54.233 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:53 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3418921580' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T15:19:54.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:53 smithi076 ceph-mon[37057]: osdmap e121: 12 total, 12 up, 12 in 2024-02-14T15:19:54.234 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:53 smithi076 ceph-mon[37057]: from='client.? 172.21.15.17:0/3093803181' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:54.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:53 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3418921580' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T15:19:54.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:53 smithi050 ceph-mon[36512]: osdmap e121: 12 total, 12 up, 12 in 2024-02-14T15:19:54.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:53 smithi050 ceph-mon[36512]: from='client.? 172.21.15.17:0/3093803181' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T15:19:54.492 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:54 smithi017 systemd[1]: Stopping Ceph mon.a for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:19:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32545]: 2024-02-14T15:19:54.491+0000 7ff3e9867700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:19:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a[32545]: 2024-02-14T15:19:54.491+0000 7ff3e9867700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-14T15:19:54.766 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:54 smithi017 podman[99321]: 2024-02-14 15:19:54.675603785 +0000 UTC m=+0.313372584 container died 5ea5e790fba548fc9618650355e2bc949b3b985166bcdb775684e5501e41cf2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:19:55.327 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:55 smithi017 podman[99321]: 2024-02-14 15:19:55.075707446 +0000 UTC m=+0.713476250 container cleanup 5ea5e790fba548fc9618650355e2bc949b3b985166bcdb775684e5501e41cf2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-02-14T15:19:55.327 INFO:journalctl@ceph.mon.a.smithi017.stdout:Feb 14 15:19:55 smithi017 bash[99321]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-a 2024-02-14T15:19:55.498 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.a.service' 2024-02-14T15:19:55.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:19:55.609 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-14T15:19:55.610 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-14T15:19:55.610 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.b 2024-02-14T15:19:55.934 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:55 smithi050 systemd[1]: Stopping Ceph mon.b for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:19:56.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-b[36508]: 2024-02-14T15:19:55.933+0000 7f4fee547700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:19:56.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-b[36508]: 2024-02-14T15:19:55.933+0000 7f4fee547700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-14T15:19:56.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:56 smithi050 podman[74969]: 2024-02-14 15:19:56.10635104 +0000 UTC m=+0.291735813 container died e78f83618c287769235beadc946b3fcd78c6ca926923f5d1420c4a02dfad3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-14T15:19:56.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:56 smithi050 podman[74969]: 2024-02-14 15:19:56.521936196 +0000 UTC m=+0.707320959 container cleanup e78f83618c287769235beadc946b3fcd78c6ca926923f5d1420c4a02dfad3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-14T15:19:56.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:56 smithi050 bash[74969]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-b 2024-02-14T15:19:57.031 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 14 15:19:56 smithi050 podman[74983]: 2024-02-14 15:19:56.774427032 +0000 UTC m=+0.665752916 container remove e78f83618c287769235beadc946b3fcd78c6ca926923f5d1420c4a02dfad3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:19:57.036 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.b.service' 2024-02-14T15:19:57.087 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:19:57.087 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-14T15:19:57.087 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-14T15:19:57.088 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.c 2024-02-14T15:19:57.484 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:57 smithi076 systemd[1]: Stopping Ceph mon.c for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:19:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-c[37037]: 2024-02-14T15:19:57.504+0000 7f2988521700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:19:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-c[37037]: 2024-02-14T15:19:57.504+0000 7f2988521700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-14T15:19:57.984 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:57 smithi076 podman[74426]: 2024-02-14 15:19:57.708791571 +0000 UTC m=+0.349937702 container died 8a2c4279921bffff61a05488fdeda40fda2cec5b6d3254403379f6e9308702cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-c, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131) 2024-02-14T15:19:58.306 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:58 smithi076 podman[74426]: 2024-02-14 15:19:58.105337483 +0000 UTC m=+0.746483664 container cleanup 8a2c4279921bffff61a05488fdeda40fda2cec5b6d3254403379f6e9308702cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-14T15:19:58.307 INFO:journalctl@ceph.mon.c.smithi076.stdout:Feb 14 15:19:58 smithi076 bash[74426]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mon-c 2024-02-14T15:19:58.515 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mon.c.service' 2024-02-14T15:19:58.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:19:58.566 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-14T15:19:58.566 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-14T15:19:58.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.x 2024-02-14T15:19:59.016 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:19:58 smithi017 systemd[1]: Stopping Ceph mgr.x for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:19:59.364 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:19:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x[32776]: 2024-02-14T15:19:59.030+0000 7f913796f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:19:59.365 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:19:59 smithi017 podman[99441]: 2024-02-14 15:19:59.061470627 +0000 UTC m=+0.291925641 container died 75150f7c5ceea6e130a14c8057c39357b00491a297bad7d2ab497769f017ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:19:59.657 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:19:59 smithi017 podman[99441]: 2024-02-14 15:19:59.447206671 +0000 UTC m=+0.677661683 container cleanup 75150f7c5ceea6e130a14c8057c39357b00491a297bad7d2ab497769f017ddb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-14T15:19:59.657 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Feb 14 15:19:59 smithi017 bash[99441]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-x 2024-02-14T15:19:59.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.x.service' 2024-02-14T15:19:59.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:19:59.832 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-14T15:19:59.832 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-14T15:19:59.832 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.y 2024-02-14T15:20:00.142 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:19:59 smithi050 systemd[1]: Stopping Ceph mgr.y for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:00.424 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:20:00 smithi050 podman[75090]: 2024-02-14 15:20:00.174054236 +0000 UTC m=+0.140914451 container died a8c20e957e06ecb987229afb400ade2bb7c10641ddb94cd6525197045cdeb26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-02-14T15:20:00.892 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:20:00 smithi050 podman[75090]: 2024-02-14 15:20:00.646698303 +0000 UTC m=+0.613558527 container cleanup a8c20e957e06ecb987229afb400ade2bb7c10641ddb94cd6525197045cdeb26e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-14T15:20:00.893 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Feb 14 15:20:00 smithi050 bash[75090]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-y 2024-02-14T15:20:01.147 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.y.service' 2024-02-14T15:20:01.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:01.191 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-14T15:20:01.191 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-14T15:20:01.191 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.z 2024-02-14T15:20:01.484 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:20:01 smithi076 systemd[1]: Stopping Ceph mgr.z for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:01.836 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:20:01 smithi076 podman[74561]: 2024-02-14 15:20:01.539297238 +0000 UTC m=+0.163296664 container died e8a8f5f084dc1ac4a96578086af5a6031f7c499e73875e08111de6e4073e0df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, io.buildah.version=1.33.2) 2024-02-14T15:20:02.230 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:20:01 smithi076 podman[74561]: 2024-02-14 15:20:01.936469991 +0000 UTC m=+0.560469459 container cleanup e8a8f5f084dc1ac4a96578086af5a6031f7c499e73875e08111de6e4073e0df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-14T15:20:02.230 INFO:journalctl@ceph.mgr.z.smithi076.stdout:Feb 14 15:20:01 smithi076 bash[74561]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mgr-z 2024-02-14T15:20:02.429 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mgr.z.service' 2024-02-14T15:20:02.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:02.469 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-14T15:20:02.469 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-14T15:20:02.469 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.0 2024-02-14T15:20:02.766 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:02 smithi017 systemd[1]: Stopping Ceph osd.0 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:03.266 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0[47600]: 2024-02-14T15:20:02.909+0000 7f4e3aad6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:20:03.266 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0[47600]: 2024-02-14T15:20:02.909+0000 7f4e3aad6700 -1 osd.0 122 *** Got signal Terminated *** 2024-02-14T15:20:03.267 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0[47600]: 2024-02-14T15:20:02.909+0000 7f4e3aad6700 -1 osd.0 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:20:08.686 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:08 smithi017 podman[99563]: 2024-02-14 15:20:08.403990005 +0000 UTC m=+5.739308250 container died 843518aa90b94161102c9f3bb95ef12477b408b3b3c44a6d1d70d3f83bfaf09e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-14T15:20:09.016 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:08 smithi017 podman[99563]: 2024-02-14 15:20:08.785084087 +0000 UTC m=+6.120402332 container cleanup 843518aa90b94161102c9f3bb95ef12477b408b3b3c44a6d1d70d3f83bfaf09e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-14T15:20:09.017 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:08 smithi017 bash[99563]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0 2024-02-14T15:20:09.413 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:09 smithi017 podman[99580]: 2024-02-14 15:20:09.054995308 +0000 UTC m=+0.650402274 container remove 843518aa90b94161102c9f3bb95ef12477b408b3b3c44a6d1d70d3f83bfaf09e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, ceph=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-14T15:20:09.707 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:09 smithi017 podman[99673]: 2024-02-14 15:20:09.315665453 +0000 UTC m=+0.018838415 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:20:09.708 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:09 smithi017 podman[99673]: 2024-02-14 15:20:09.454979771 +0000 UTC m=+0.158152765 container create e24c9749add2bd8d5a6ac67b755c686de749f435b3842f1d149171796db86832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, ceph=True) 2024-02-14T15:20:10.009 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:09 smithi017 podman[99673]: 2024-02-14 15:20:09.707385968 +0000 UTC m=+0.410558928 container init e24c9749add2bd8d5a6ac67b755c686de749f435b3842f1d149171796db86832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0-deactivate, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=) 2024-02-14T15:20:10.009 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:09 smithi017 podman[99673]: 2024-02-14 15:20:09.710801722 +0000 UTC m=+0.413974674 container start e24c9749add2bd8d5a6ac67b755c686de749f435b3842f1d149171796db86832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True) 2024-02-14T15:20:10.009 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:09 smithi017 podman[99673]: 2024-02-14 15:20:09.790046613 +0000 UTC m=+0.493219583 container attach e24c9749add2bd8d5a6ac67b755c686de749f435b3842f1d149171796db86832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-14T15:20:10.266 INFO:journalctl@ceph.osd.0.smithi017.stdout:Feb 14 15:20:10 smithi017 podman[99673]: 2024-02-14 15:20:10.009087009 +0000 UTC m=+0.712259970 container died e24c9749add2bd8d5a6ac67b755c686de749f435b3842f1d149171796db86832 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:20:10.958 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.0.service' 2024-02-14T15:20:11.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:11.008 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-14T15:20:11.008 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-14T15:20:11.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.1 2024-02-14T15:20:11.329 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:11 smithi050 systemd[1]: Stopping Ceph osd.1 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:11.642 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1[41151]: 2024-02-14T15:20:11.327+0000 7faa6e7d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:20:11.642 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1[41151]: 2024-02-14T15:20:11.327+0000 7faa6e7d0700 -1 osd.1 122 *** Got signal Terminated *** 2024-02-14T15:20:11.642 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1[41151]: 2024-02-14T15:20:11.327+0000 7faa6e7d0700 -1 osd.1 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:20:17.230 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:16 smithi050 podman[75226]: 2024-02-14 15:20:16.932826808 +0000 UTC m=+5.717113301 container died 9e82fe03358f0fefcbc481ed34c89f4cdc69541ef4ded42ab8712ed04c9c0422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:20:17.614 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:17 smithi050 podman[75226]: 2024-02-14 15:20:17.395616641 +0000 UTC m=+6.179903130 container cleanup 9e82fe03358f0fefcbc481ed34c89f4cdc69541ef4ded42ab8712ed04c9c0422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_CLEAN=True) 2024-02-14T15:20:17.614 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:17 smithi050 bash[75226]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1 2024-02-14T15:20:17.892 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:17 smithi050 podman[75243]: 2024-02-14 15:20:17.614057057 +0000 UTC m=+0.680932499 container remove 9e82fe03358f0fefcbc481ed34c89f4cdc69541ef4ded42ab8712ed04c9c0422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, ceph=True) 2024-02-14T15:20:18.285 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:18 smithi050 podman[75334]: 2024-02-14 15:20:17.929077666 +0000 UTC m=+0.018374471 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:20:18.285 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:18 smithi050 podman[75334]: 2024-02-14 15:20:18.048924542 +0000 UTC m=+0.138221334 container create 84fa60ed5a1668baff1287c403483c335eddc22bed707745fac609af49b5e04a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-14T15:20:18.594 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:18 smithi050 podman[75334]: 2024-02-14 15:20:18.28479772 +0000 UTC m=+0.374094514 container init 84fa60ed5a1668baff1287c403483c335eddc22bed707745fac609af49b5e04a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True) 2024-02-14T15:20:18.594 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:18 smithi050 podman[75334]: 2024-02-14 15:20:18.288565921 +0000 UTC m=+0.377862707 container start 84fa60ed5a1668baff1287c403483c335eddc22bed707745fac609af49b5e04a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-14T15:20:18.594 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:18 smithi050 podman[75334]: 2024-02-14 15:20:18.377097079 +0000 UTC m=+0.466393880 container attach 84fa60ed5a1668baff1287c403483c335eddc22bed707745fac609af49b5e04a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:20:18.892 INFO:journalctl@ceph.osd.1.smithi050.stdout:Feb 14 15:20:18 smithi050 podman[75334]: 2024-02-14 15:20:18.593735312 +0000 UTC m=+0.683032114 container died 84fa60ed5a1668baff1287c403483c335eddc22bed707745fac609af49b5e04a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:20:19.131 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.1.service' 2024-02-14T15:20:19.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:19.179 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-14T15:20:19.179 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-14T15:20:19.179 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.2 2024-02-14T15:20:19.484 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:19 smithi076 systemd[1]: Stopping Ceph osd.2 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:19.734 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2[42220]: 2024-02-14T15:20:19.481+0000 7f1ae784c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:20:19.734 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2[42220]: 2024-02-14T15:20:19.481+0000 7f1ae784c700 -1 osd.2 122 *** Got signal Terminated *** 2024-02-14T15:20:19.734 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2[42220]: 2024-02-14T15:20:19.481+0000 7f1ae784c700 -1 osd.2 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:20:25.308 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:25 smithi076 podman[74683]: 2024-02-14 15:20:25.003850714 +0000 UTC m=+5.636067435 container died b9c9def6367e77609af20e232cde922bb979fe1a371b0beed379a343cb1ac864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:20:25.667 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:25 smithi076 podman[74683]: 2024-02-14 15:20:25.416306144 +0000 UTC m=+6.048522858 container cleanup b9c9def6367e77609af20e232cde922bb979fe1a371b0beed379a343cb1ac864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:20:25.667 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:25 smithi076 bash[74683]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2 2024-02-14T15:20:25.976 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:25 smithi076 podman[74700]: 2024-02-14 15:20:25.667766316 +0000 UTC m=+0.657792992 container remove b9c9def6367e77609af20e232cde922bb979fe1a371b0beed379a343cb1ac864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-14T15:20:26.234 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:25 smithi076 podman[74794]: 2024-02-14 15:20:25.87925595 +0000 UTC m=+0.017258001 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:20:26.235 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:25 smithi076 podman[74794]: 2024-02-14 15:20:25.993865503 +0000 UTC m=+0.131867580 container create c37c06aed9f10639b9427e103e73cf298a0f412ae27ad171e25326bf62ee581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-14T15:20:26.734 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:26 smithi076 podman[74794]: 2024-02-14 15:20:26.250946045 +0000 UTC m=+0.388948102 container init c37c06aed9f10639b9427e103e73cf298a0f412ae27ad171e25326bf62ee581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0) 2024-02-14T15:20:26.734 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:26 smithi076 podman[74794]: 2024-02-14 15:20:26.254547105 +0000 UTC m=+0.392549148 container start c37c06aed9f10639b9427e103e73cf298a0f412ae27ad171e25326bf62ee581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:20:26.735 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:26 smithi076 podman[74794]: 2024-02-14 15:20:26.388487517 +0000 UTC m=+0.526489563 container attach c37c06aed9f10639b9427e103e73cf298a0f412ae27ad171e25326bf62ee581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=) 2024-02-14T15:20:27.092 INFO:journalctl@ceph.osd.2.smithi076.stdout:Feb 14 15:20:26 smithi076 podman[74794]: 2024-02-14 15:20:26.793498761 +0000 UTC m=+0.931500793 container died c37c06aed9f10639b9427e103e73cf298a0f412ae27ad171e25326bf62ee581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2) 2024-02-14T15:20:27.408 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.2.service' 2024-02-14T15:20:27.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:27.451 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-14T15:20:27.451 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-14T15:20:27.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.3 2024-02-14T15:20:27.766 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:27 smithi017 systemd[1]: Stopping Ceph osd.3 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:28.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:27 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T15:20:27.788+0000 7f8ca08b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:20:28.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:27 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T15:20:27.788+0000 7f8ca08b1700 -1 osd.3 122 *** Got signal Terminated *** 2024-02-14T15:20:28.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:27 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T15:20:27.788+0000 7f8ca08b1700 -1 osd.3 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:20:29.016 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:28 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T15:20:28.895+0000 7f8c9ceac700 -1 osd.3 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.731810+0000 front 2024-02-14T15:20:07.731867+0000 (oldest deadline 2024-02-14T15:20:28.732542+0000) 2024-02-14T15:20:29.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:29 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:29.465+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:30.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:29 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T15:20:29.938+0000 7f8c9ceac700 -1 osd.3 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.731810+0000 front 2024-02-14T15:20:07.731867+0000 (oldest deadline 2024-02-14T15:20:28.732542+0000) 2024-02-14T15:20:30.962 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:30 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:30.515+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:31.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:30 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:30.796+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:31.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:30 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T15:20:30.961+0000 7f8c9ceac700 -1 osd.3 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.731810+0000 front 2024-02-14T15:20:07.731867+0000 (oldest deadline 2024-02-14T15:20:28.732542+0000) 2024-02-14T15:20:31.936 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:31 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:31.530+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:32.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:31 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:31.796+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:32.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:31 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3[54574]: 2024-02-14T15:20:31.934+0000 7f8c9ceac700 -1 osd.3 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.731810+0000 front 2024-02-14T15:20:07.731867+0000 (oldest deadline 2024-02-14T15:20:28.732542+0000) 2024-02-14T15:20:32.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:32 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:32.501+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:33.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:32 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:32.767+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:33.516 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:33 smithi017 podman[99780]: 2024-02-14 15:20:33.216770906 +0000 UTC m=+5.558308862 container died 980f7349ee64d9cef55f86aaecb3bf497acdf99ce9a85c2abd18c40aa803b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-14T15:20:33.955 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:33 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:33.549+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:33.956 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:33 smithi017 podman[99780]: 2024-02-14 15:20:33.746046799 +0000 UTC m=+6.087584747 container cleanup 980f7349ee64d9cef55f86aaecb3bf497acdf99ce9a85c2abd18c40aa803b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD) 2024-02-14T15:20:33.956 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:33 smithi017 bash[99780]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3 2024-02-14T15:20:34.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:33 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:33.783+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:34.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:33 smithi017 podman[99808]: 2024-02-14 15:20:33.955625675 +0000 UTC m=+0.743208237 container remove 980f7349ee64d9cef55f86aaecb3bf497acdf99ce9a85c2abd18c40aa803b8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3, ceph=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-14T15:20:34.660 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:34 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:34.519+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:34.660 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:34 smithi017 podman[99897]: 2024-02-14 15:20:34.269389689 +0000 UTC m=+0.017975616 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:20:34.661 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:34 smithi017 podman[99897]: 2024-02-14 15:20:34.398232625 +0000 UTC m=+0.146818547 container create 587148cf5d2abbffbef62e89009be4ac881f57641fba7ed7988fa8e865ba9e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2) 2024-02-14T15:20:34.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:34.299+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:34.969 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:34 smithi017 podman[99897]: 2024-02-14 15:20:34.659034546 +0000 UTC m=+0.407620469 container init 587148cf5d2abbffbef62e89009be4ac881f57641fba7ed7988fa8e865ba9e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:20:34.969 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:34 smithi017 podman[99897]: 2024-02-14 15:20:34.662354134 +0000 UTC m=+0.410940054 container start 587148cf5d2abbffbef62e89009be4ac881f57641fba7ed7988fa8e865ba9e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-14T15:20:34.970 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:34 smithi017 podman[99897]: 2024-02-14 15:20:34.767426347 +0000 UTC m=+0.516012287 container attach 587148cf5d2abbffbef62e89009be4ac881f57641fba7ed7988fa8e865ba9e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:20:35.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:34 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:34.782+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:35.266 INFO:journalctl@ceph.osd.3.smithi017.stdout:Feb 14 15:20:34 smithi017 podman[99897]: 2024-02-14 15:20:34.968558377 +0000 UTC m=+0.717144304 container died 587148cf5d2abbffbef62e89009be4ac881f57641fba7ed7988fa8e865ba9e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux ) 2024-02-14T15:20:35.612 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:35 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:35.502+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:35.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.3.service' 2024-02-14T15:20:35.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:35.691 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-14T15:20:35.691 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-14T15:20:35.691 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.4 2024-02-14T15:20:35.733 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:35.271+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:36.010 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:35 smithi050 systemd[1]: Stopping Ceph osd.4 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:36.010 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:35 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:35.767+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:36.392 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:36 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4[48009]: 2024-02-14T15:20:36.008+0000 7f149d77a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:20:36.392 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:36 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4[48009]: 2024-02-14T15:20:36.008+0000 7f149d77a700 -1 osd.4 122 *** Got signal Terminated *** 2024-02-14T15:20:36.392 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:36 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4[48009]: 2024-02-14T15:20:36.008+0000 7f149d77a700 -1 osd.4 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:20:36.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:36.306+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:36.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:36 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:36.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:37.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:36 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:36.787+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:37.733 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:37.324+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:37.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:37 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:37.424+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:38.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:37 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:37.790+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:38.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:37 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:37.695+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:38.656 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:38 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:38.367+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:38.656 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:38 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:38.367+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:38.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:38 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:38.385+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:39.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:38 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:38.715+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:39.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:38 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:38.819+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:39.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:38 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:38.819+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:39.733 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:39 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:39.349+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:39.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:39 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:39.349+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:39.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:39 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:39.419+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:40.142 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:39 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4[48009]: 2024-02-14T15:20:39.828+0000 7f1495d6d700 -1 osd.4 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.939832+0000 front 2024-02-14T15:20:13.939794+0000 (oldest deadline 2024-02-14T15:20:39.239997+0000) 2024-02-14T15:20:40.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:39 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:39.679+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:40.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:39 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:39.796+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:40.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:39 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:39.796+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:40.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:40.009+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:40.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:40 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:40.319+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:40.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:40 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:40.319+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:40.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:40.432+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:40.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:40.432+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:40.892 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:40 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4[48009]: 2024-02-14T15:20:40.829+0000 7f1495d6d700 -1 osd.4 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.939832+0000 front 2024-02-14T15:20:13.939794+0000 (oldest deadline 2024-02-14T15:20:39.239997+0000) 2024-02-14T15:20:40.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:40 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:40.630+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:40.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:40 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:40.768+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:40.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:40 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:40.768+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:41.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:40 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:40.983+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:41.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:41 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:41.000+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:41.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:41 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:41.348+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:41.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:41 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:41.348+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:41.748 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:41 smithi050 podman[75466]: 2024-02-14 15:20:41.446900714 +0000 UTC m=+5.552067340 container died 3aaa2bd32fe8ae74f1733bb319b674bfe5b87d52294e7edd4ddb29ad530d80f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2) 2024-02-14T15:20:41.748 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:41 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:41.666+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:41.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:41 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:41.466+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:41.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:41 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:41.466+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:42.142 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:41 smithi050 podman[75466]: 2024-02-14 15:20:41.768277115 +0000 UTC m=+5.873443716 container cleanup 3aaa2bd32fe8ae74f1733bb319b674bfe5b87d52294e7edd4ddb29ad530d80f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:20:42.142 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:41 smithi050 bash[75466]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4 2024-02-14T15:20:42.142 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:41 smithi050 podman[75481]: 2024-02-14 15:20:41.969645884 +0000 UTC m=+0.518068711 container remove 3aaa2bd32fe8ae74f1733bb319b674bfe5b87d52294e7edd4ddb29ad530d80f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-02-14T15:20:42.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:41 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:41.782+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:42.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:41 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:41.782+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:42.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:41 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:41.985+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:42.357 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:42 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:42.016+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:42.633 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:42 smithi050 podman[75575]: 2024-02-14 15:20:42.379090707 +0000 UTC m=+0.109485491 container create 77b0df663b189c1e4c5e96c6e3178804dd1c6444e05c41b88f32b7d37e3b97b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-14T15:20:42.633 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:42 smithi050 podman[75575]: 2024-02-14 15:20:42.287438077 +0000 UTC m=+0.017832873 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:20:42.633 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:42 smithi050 podman[75575]: 2024-02-14 15:20:42.615164054 +0000 UTC m=+0.345558859 container init 77b0df663b189c1e4c5e96c6e3178804dd1c6444e05c41b88f32b7d37e3b97b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2) 2024-02-14T15:20:42.633 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:42 smithi050 podman[75575]: 2024-02-14 15:20:42.618545987 +0000 UTC m=+0.348940766 container start 77b0df663b189c1e4c5e96c6e3178804dd1c6444e05c41b88f32b7d37e3b97b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:20:42.668 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:42 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:42.356+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:42.668 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:42 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:42.356+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:42.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:42 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:42.425+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:42.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:42 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:42.425+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:42.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:42 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:42.632+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:42.892 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:42 smithi050 podman[75575]: 2024-02-14 15:20:42.715149252 +0000 UTC m=+0.445544055 container attach 77b0df663b189c1e4c5e96c6e3178804dd1c6444e05c41b88f32b7d37e3b97b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:20:42.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:42 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:42.786+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:42.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:42 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:42.786+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:42.972 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:42 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:42.668+0000 7f43a80f6700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.178663+0000 front 2024-02-14T15:20:14.178636+0000 (oldest deadline 2024-02-14T15:20:41.779351+0000) 2024-02-14T15:20:43.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:42 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:42.972+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:43.260 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 14 15:20:42 smithi050 podman[75575]: 2024-02-14 15:20:42.928570362 +0000 UTC m=+0.658965156 container died 77b0df663b189c1e4c5e96c6e3178804dd1c6444e05c41b88f32b7d37e3b97b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-14T15:20:43.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:42 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:42.985+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:43.653 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.4.service' 2024-02-14T15:20:43.687 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:43 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:43.373+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:43.688 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:43 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:43.373+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:43.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:43.701 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-14T15:20:43.701 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-14T15:20:43.701 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.5 2024-02-14T15:20:43.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:43 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:43.457+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:43.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:43 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:43.457+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:43.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:43 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:43.610+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:43.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:43 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:43.808+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:43.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:43 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:43.808+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:43.942 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:43 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:43.686+0000 7f43a80f6700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.178663+0000 front 2024-02-14T15:20:14.178636+0000 (oldest deadline 2024-02-14T15:20:41.779351+0000) 2024-02-14T15:20:43.942 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:43 smithi076 systemd[1]: Stopping Ceph osd.5 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:44.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:43 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:43.940+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:44.234 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:44 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:44.036+0000 7f43afb03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:20:44.234 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:44 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:44.036+0000 7f43afb03700 -1 osd.5 122 *** Got signal Terminated *** 2024-02-14T15:20:44.234 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:44 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:44.036+0000 7f43afb03700 -1 osd.5 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:20:44.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:43 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:43.973+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:44.698 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:44 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:44.338+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:44.698 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:44 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:44.338+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:44.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:44 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:44.412+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:44.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:44 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:44.412+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:44.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:44 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:44.619+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:44.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:44 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:44.814+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:44.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:44 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:44.814+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:44.984 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:44 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:44.697+0000 7f43a80f6700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.178663+0000 front 2024-02-14T15:20:14.178636+0000 (oldest deadline 2024-02-14T15:20:41.779351+0000) 2024-02-14T15:20:44.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:44 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:44.935+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:45.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:44 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:44.959+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:45.677 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:45 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:45.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:45.677 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:45 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:45.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:45.695 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:45 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:45.440+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:45.695 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:45 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:45.440+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:45.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:45 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:45.630+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:45.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:45 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:45.833+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:45.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:45 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:45.833+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:45.983 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:45 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:45.674+0000 7f43a80f6700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.178663+0000 front 2024-02-14T15:20:14.178636+0000 (oldest deadline 2024-02-14T15:20:41.779351+0000) 2024-02-14T15:20:45.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:45 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:45.916+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:45.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:45 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:45.916+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:46.016 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:45 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:45.924+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:46.664 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:46 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:46.316+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:46.664 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:46 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:46.316+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:46.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:46 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:46.441+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:46.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:46 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:46.441+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:46.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:46 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:46.441+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:46.983 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:46 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:46.663+0000 7f43a80f6700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.178663+0000 front 2024-02-14T15:20:14.178636+0000 (oldest deadline 2024-02-14T15:20:41.779351+0000) 2024-02-14T15:20:46.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:46 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:46.883+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:46.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:46 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:46.883+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:47.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:46 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:46.656+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:47.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:46 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:46.883+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:47.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:46 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:46.883+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:47.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:46 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:46.883+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:47.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:46 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:46.915+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:47.619 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:47 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:47.353+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:47.619 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:47 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:47.353+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:47.619 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:47 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:47.353+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:47.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:47 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:47.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:47.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:47 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:47.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:47.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:47 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:47.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:47.889 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:47 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:47.618+0000 7f43a80f6700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.178663+0000 front 2024-02-14T15:20:14.178636+0000 (oldest deadline 2024-02-14T15:20:41.779351+0000) 2024-02-14T15:20:47.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:47 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:47.634+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:48.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:47 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:47.889+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:48.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:47 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:47.889+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:48.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:47 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:47.937+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:48.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:47 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:47.908+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:48.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:47 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:47.908+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:48.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:47 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:47.908+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:48.637 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:48 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:48.359+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:48.637 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:48 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:48.359+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:48.637 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:48 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:48.359+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:48.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:48 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:48.452+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:48.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:48 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:48.452+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:48.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:48 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:48.452+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:48.984 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:48 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5[48902]: 2024-02-14T15:20:48.635+0000 7f43a80f6700 -1 osd.5 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.178663+0000 front 2024-02-14T15:20:14.178636+0000 (oldest deadline 2024-02-14T15:20:41.779351+0000) 2024-02-14T15:20:48.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:48 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:48.862+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:48.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:48 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:48.862+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:49.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:48 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:48.866+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:49.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:48 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:48.866+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:49.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:48 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:48.866+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:49.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:48 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:48.647+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:49.143 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:48 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:48.647+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:49.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:48 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:48.901+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:49.618 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:49.316+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:49.619 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:49.316+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:49.619 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:49.316+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:49.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:49.411+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:49.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:49.411+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:49.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:49.411+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:49.883 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:49 smithi076 podman[74911]: 2024-02-14 15:20:49.61836927 +0000 UTC m=+5.722198175 container died 48f57b09fd4a32977f109517f2f372b98172b665673bd1db25e41d8e6e160fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-02-14T15:20:49.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:49.629+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:49.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:49.629+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:50.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:49.882+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:50.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:49 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:49.882+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:50.234 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:50 smithi076 podman[74911]: 2024-02-14 15:20:50.004744987 +0000 UTC m=+6.108573875 container cleanup 48f57b09fd4a32977f109517f2f372b98172b665673bd1db25e41d8e6e160fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-14T15:20:50.234 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:50 smithi076 bash[74911]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5 2024-02-14T15:20:50.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:49.895+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:50.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:49.895+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:50.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:49.906+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:50.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:49.906+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:50.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:49 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:49.906+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:50.607 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:50 smithi076 podman[74928]: 2024-02-14 15:20:50.297923269 +0000 UTC m=+0.673035130 container remove 48f57b09fd4a32977f109517f2f372b98172b665673bd1db25e41d8e6e160fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:20:50.608 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:50 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:50.337+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:50.608 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:50 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:50.337+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:50.608 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:50 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:50.337+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:50.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:50 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:50.451+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:50.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:50 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:50.452+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:50.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:50 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:50.452+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:50.873 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:50 smithi076 podman[75020]: 2024-02-14 15:20:50.511278847 +0000 UTC m=+0.017598201 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:20:50.873 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:50 smithi076 podman[75020]: 2024-02-14 15:20:50.623899604 +0000 UTC m=+0.130218990 container create e48f967226a15338acbc88b792de894ee4f8f57febfe94e598363f12a8eaf9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-14T15:20:50.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:50 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:50.628+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:50.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:50 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:50.628+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:50.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:50 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:50.862+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:50.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:50 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:50.862+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:50.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:50 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:50.862+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:51.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:50 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:50.871+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:51.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:50 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:50.871+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:51.234 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:50 smithi076 podman[75020]: 2024-02-14 15:20:50.935557633 +0000 UTC m=+0.441876994 container init e48f967226a15338acbc88b792de894ee4f8f57febfe94e598363f12a8eaf9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:20:51.234 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:50 smithi076 podman[75020]: 2024-02-14 15:20:50.938769597 +0000 UTC m=+0.445088942 container start e48f967226a15338acbc88b792de894ee4f8f57febfe94e598363f12a8eaf9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-14T15:20:51.235 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:51 smithi076 podman[75020]: 2024-02-14 15:20:51.018709584 +0000 UTC m=+0.525028945 container attach e48f967226a15338acbc88b792de894ee4f8f57febfe94e598363f12a8eaf9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:20:51.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:50 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:50.922+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:51.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:50 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:50.922+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:51.689 INFO:journalctl@ceph.osd.5.smithi076.stdout:Feb 14 15:20:51 smithi076 podman[75020]: 2024-02-14 15:20:51.235414755 +0000 UTC m=+0.741734099 container died e48f967226a15338acbc88b792de894ee4f8f57febfe94e598363f12a8eaf9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0) 2024-02-14T15:20:51.689 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:51 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:51.334+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:51.689 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:51 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:51.334+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:51.689 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:51 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:51.334+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:51.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:51 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:51.480+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:51.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:51 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:51.480+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:51.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:51 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:51.480+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:51.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:51 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:51.581+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:51.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:51 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:51.581+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:51.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:51 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:51.815+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:51.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:51 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:51.815+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:51.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:51 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:51.815+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:51.983 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:51 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:51.837+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:51.984 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:51 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:51.837+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:52.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.5.service' 2024-02-14T15:20:52.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:20:52.054 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-14T15:20:52.054 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-14T15:20:52.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.6 2024-02-14T15:20:52.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:51 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:51.960+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:52.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:51 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:51.960+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:52.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:52 smithi017 systemd[1]: Stopping Ceph osd.6 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:20:52.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:52 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:52.294+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:52.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:52 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:52.294+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:52.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:52 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:52.294+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:52.766 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:52.371+0000 7f3e4af85700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:20:52.766 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:52.371+0000 7f3e4af85700 -1 osd.6 122 *** Got signal Terminated *** 2024-02-14T15:20:52.767 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:52.371+0000 7f3e4af85700 -1 osd.6 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:20:52.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:52.530+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:52.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:52.530+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:52.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:52.530+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:52.862 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:52 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:52.609+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:52.862 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:52 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:52.609+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:53.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:52 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:52.861+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:53.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:52 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:52.861+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:53.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:52 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:52.861+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:53.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:52 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:52.809+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:53.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:52 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:52.809+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:53.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:52.997+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:53.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:52 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:52.997+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:53.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:53.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:53.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:53.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:53.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:53.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:53.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:53 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:53.612+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:53.893 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:53 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:53.612+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:53.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:53 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:53.834+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:53.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:53 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:53.834+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:53.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:53 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:53.834+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:53.989 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:53.553+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:53.989 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:53.553+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:53.990 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:53.553+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:54.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:53.786+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:54.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:53.786+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:54.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:53 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:53.786+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:20:54.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:53.988+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:54.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:53 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:53.988+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:54.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:54.231+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:54.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:54.232+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:54.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:54.232+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:54.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:54 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:54.639+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:54.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:54 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:54.639+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:54.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:54 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:54.802+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:54.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:54 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:54.802+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:54.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:54 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:54.802+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:54.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:54 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:54.802+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:20:54.961 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:54.564+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:54.961 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:54.564+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:54.961 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:54.564+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:55.205 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:54.799+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:55.205 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:54.799+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:55.206 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:54 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:54.799+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:20:55.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:54.960+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:55.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:54.960+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:55.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:54.960+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:29.635102+0000 front 2024-02-14T15:20:29.635155+0000 (oldest deadline 2024-02-14T15:20:54.935380+0000) 2024-02-14T15:20:55.483 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:55.204+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:55.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:55.204+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:55.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:55.204+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:55.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:55.640+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:55.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:55.640+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:55.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:55.834+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:55.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:55.834+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:55.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:55.834+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:55.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:55 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:55.834+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:20:55.973 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:55 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:55.553+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:55.973 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:55 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:55.553+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:55.973 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:55 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:55.553+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:56.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:55.834+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:56.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:55.834+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:56.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:55 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:55.834+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:20:56.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:55 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:55.972+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:56.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:55 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:55.972+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:56.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:55 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:55.972+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:29.635102+0000 front 2024-02-14T15:20:29.635155+0000 (oldest deadline 2024-02-14T15:20:54.935380+0000) 2024-02-14T15:20:56.733 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:56.247+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:56.733 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:56.247+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:56.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:56.247+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:56.969 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:56.521+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:56.969 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:56.521+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:56.969 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:56.521+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:57.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:56.660+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:57.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:56.660+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:57.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:56.851+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:57.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:56.851+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:57.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:56.851+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:57.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:56 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:56.851+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:20:57.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:56.811+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:57.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:56.811+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:57.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:56 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:56.811+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:20:57.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:56.968+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.432805+0000 front 2024-02-14T15:20:14.432927+0000 (oldest deadline 2024-02-14T15:20:39.732901+0000) 2024-02-14T15:20:57.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:56.968+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.334957+0000 front 2024-02-14T15:20:24.334884+0000 (oldest deadline 2024-02-14T15:20:49.634709+0000) 2024-02-14T15:20:57.266 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:56 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6[61124]: 2024-02-14T15:20:56.968+0000 7f3e43578700 -1 osd.6 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:29.635102+0000 front 2024-02-14T15:20:29.635155+0000 (oldest deadline 2024-02-14T15:20:54.935380+0000) 2024-02-14T15:20:57.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:57.265+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:57.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:57.265+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:57.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:57.265+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:57.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:57 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:57.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:57.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:57 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:57.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:57.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:57 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:57.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:57.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:57 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:57.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:20:57.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:57.622+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:57.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:57.622+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:57.893 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:57.622+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:20:57.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:57.868+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:57.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:57.868+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:57.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:57.868+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:57.894 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:57 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:57.868+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:20:58.089 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:57 smithi017 podman[100003]: 2024-02-14 15:20:57.78103026 +0000 UTC m=+5.528004184 container died 30b22004a457540c3538be67e57dcab7bd2e1333ccdb376eb37689a05d291cba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-14T15:20:58.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:57.830+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:58.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:57.830+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:58.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:57 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:57.830+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:20:58.473 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:58 smithi017 podman[100003]: 2024-02-14 15:20:58.204865105 +0000 UTC m=+5.951839019 container cleanup 30b22004a457540c3538be67e57dcab7bd2e1333ccdb376eb37689a05d291cba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.2) 2024-02-14T15:20:58.473 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:58 smithi017 bash[100003]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6 2024-02-14T15:20:58.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:58.259+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:58.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:58.259+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:58.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:58.259+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:58.766 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:58 smithi017 podman[100019]: 2024-02-14 15:20:58.47294282 +0000 UTC m=+0.695855691 container remove 30b22004a457540c3538be67e57dcab7bd2e1333ccdb376eb37689a05d291cba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:20:58.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:58 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:58.505+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:58.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:58 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:58.505+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:58.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:58 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:58.505+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:58.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:58 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:58.505+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:20:59.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:58.665+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:59.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:58.665+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:59.143 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:58.665+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:20:59.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:58.833+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:59.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:58.833+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:59.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:58.833+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:59.144 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:58.833+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:20:59.185 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:58 smithi017 podman[100109]: 2024-02-14 15:20:58.777448126 +0000 UTC m=+0.017601967 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:20:59.185 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:58 smithi017 podman[100109]: 2024-02-14 15:20:58.898717226 +0000 UTC m=+0.138871055 container create dabe89f6fa15ccd1e76d966b8930f081bbf9a12d42e929bd9691123f6d7290eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:20:59.185 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:59 smithi017 podman[100109]: 2024-02-14 15:20:59.093129216 +0000 UTC m=+0.333283045 container init dabe89f6fa15ccd1e76d966b8930f081bbf9a12d42e929bd9691123f6d7290eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-14T15:20:59.185 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:59 smithi017 podman[100109]: 2024-02-14 15:20:59.096150739 +0000 UTC m=+0.336304568 container start dabe89f6fa15ccd1e76d966b8930f081bbf9a12d42e929bd9691123f6d7290eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-14T15:20:59.185 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:59 smithi017 podman[100109]: 2024-02-14 15:20:59.184587776 +0000 UTC m=+0.424741617 container attach dabe89f6fa15ccd1e76d966b8930f081bbf9a12d42e929bd9691123f6d7290eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-14T15:20:59.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:58.852+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:20:59.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:58.852+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:20:59.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:58 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:58.852+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:20:59.500 INFO:journalctl@ceph.osd.6.smithi017.stdout:Feb 14 15:20:59 smithi017 podman[100109]: 2024-02-14 15:20:59.389382895 +0000 UTC m=+0.629536724 container died dabe89f6fa15ccd1e76d966b8930f081bbf9a12d42e929bd9691123f6d7290eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131) 2024-02-14T15:20:59.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:59 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:59.250+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:20:59.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:59 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:59.250+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:20:59.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:20:59 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:20:59.250+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:20:59.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:59.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:20:59.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:59.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:20:59.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:59.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:20:59.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:20:59 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:20:59.499+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:20:59.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:59.620+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:20:59.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:59.620+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:20:59.893 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:20:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:20:59.620+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:20:59.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:59.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:20:59.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:59.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:20:59.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:59.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:20:59.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:20:59 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:20:59.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:20:59.976 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.6.service' 2024-02-14T15:21:00.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:00.018 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-14T15:21:00.018 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-14T15:21:00.018 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.7 2024-02-14T15:21:00.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:59 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:59.858+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:00.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:59 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:59.858+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:00.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:20:59 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:20:59.858+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:00.339 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:00 smithi050 systemd[1]: Stopping Ceph osd.7 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:00.642 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:00.337+0000 7f5198bd7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:00.642 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:00.337+0000 7f5198bd7700 -1 osd.7 122 *** Got signal Terminated *** 2024-02-14T15:21:00.642 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:00.337+0000 7f5198bd7700 -1 osd.7 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:21:00.643 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:00.583+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:21:00.643 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:00.583+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:21:00.643 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:00.583+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:21:00.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:00 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:00.257+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:00.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:00 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:00.257+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:00.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:00 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:00.257+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:00.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:00.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:00.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:00.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:00.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:00.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:00.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:00 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:00.471+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:01.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:00.848+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:01.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:00.848+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:01.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:00.848+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:01.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:00 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:00.848+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:01.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:00 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:00.861+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:01.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:00 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:00.861+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:01.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:00 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:00.861+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:01.627 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:01.560+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:21:01.627 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:01.560+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:21:01.628 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:01.560+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:21:01.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:01 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:01.247+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:01.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:01 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:01.247+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:01.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:01 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:01.247+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:01.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:01 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:01.463+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:01.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:01 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:01.463+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:01.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:01 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:01.463+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:01.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:01 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:01.463+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:01.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:01.865+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:01.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:01.865+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:01.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:01.865+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:01.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:01 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:01.865+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:02.229 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:01 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:01.847+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:02.230 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:01 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:01.847+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:02.230 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:01 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:01.847+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:02.230 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:01 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:01.847+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:02.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:02 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:02.228+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:02.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:02 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:02.229+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:02.485 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:02 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:02.229+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:02.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:02.418+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:02.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:02.418+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:02.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:02.418+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:02.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:02.418+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:02.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:02 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:02.418+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:02.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:02.819+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:02.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:02.819+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:02.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:02.819+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:02.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:02.819+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:02.893 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:02.600+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:21:02.893 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:02.600+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:21:02.894 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:02.600+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:21:02.894 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:02.600+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.181024+0000 front 2024-02-14T15:20:37.181062+0000 (oldest deadline 2024-02-14T15:21:01.781612+0000) 2024-02-14T15:21:03.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:02 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:02.835+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:03.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:02 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:02.835+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:03.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:02 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:02.835+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:03.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:02 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:02.835+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:03.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:03.258+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:03.736 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:03.258+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:03.736 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:03.258+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:03.736 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:03.258+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:03.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:03 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:03.378+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:03.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:03 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:03.378+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:03.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:03 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:03.378+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:03.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:03 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:03.378+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:03.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:03 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:03.378+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:03.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:03.804+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:03.894 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:03.804+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:03.894 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:03.804+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:03.894 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:03.804+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:03.894 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:03.804+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:03.895 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:03.597+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:21:03.895 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:03.597+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:21:03.895 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:03.597+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:21:03.895 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:03 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:03.597+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.181024+0000 front 2024-02-14T15:20:37.181062+0000 (oldest deadline 2024-02-14T15:21:01.781612+0000) 2024-02-14T15:21:04.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:03.820+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:04.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:03.820+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:04.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:03.820+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:04.235 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:03 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:03.820+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:04.512 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:04.298+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:04.512 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:04.298+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:04.512 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:04.298+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:04.512 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:04.299+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:04.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:04 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:04.361+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:04.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:04 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:04.361+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:04.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:04 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:04.361+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:04.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:04 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:04.361+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:04.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:04 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:04.361+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:04.844 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:04.574+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:12.076898+0000 front 2024-02-14T15:20:12.076922+0000 (oldest deadline 2024-02-14T15:20:37.377090+0000) 2024-02-14T15:21:04.844 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:04.574+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:22.678358+0000 front 2024-02-14T15:20:22.678266+0000 (oldest deadline 2024-02-14T15:20:47.978312+0000) 2024-02-14T15:21:04.844 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:04.574+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.980091+0000 front 2024-02-14T15:20:31.980125+0000 (oldest deadline 2024-02-14T15:20:56.680065+0000) 2024-02-14T15:21:04.844 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7[55026]: 2024-02-14T15:21:04.575+0000 7f51951d2700 -1 osd.7 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.181024+0000 front 2024-02-14T15:20:37.181062+0000 (oldest deadline 2024-02-14T15:21:01.781612+0000) 2024-02-14T15:21:05.142 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:04.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:05.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:04.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:05.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:04.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:05.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:04.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:05.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:04 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:04.844+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:05.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:04.848+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:05.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:04.848+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:05.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:04.848+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:05.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:04 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:04.848+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:05.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:05.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:05.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:05.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:05.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:05.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:05.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:05.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:05.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:05 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:05.407+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:05.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:05 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:05.407+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:05.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:05 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:05.407+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:05.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:05 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:05.407+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:05.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:05 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:05.407+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:06.142 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:05 smithi050 podman[75681]: 2024-02-14 15:21:05.854953762 +0000 UTC m=+5.634573891 container died 7f109cce6ef84c1bcae7dfc1061fd53d7ec7122d54055284bc588fcc489109f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-14T15:21:06.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:05 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:05.871+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:06.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:05 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:05.871+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:06.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:05 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:05.871+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:06.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:05 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:05.871+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:06.143 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:05 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:05.871+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:06.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:05.873+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:06.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:05.874+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:06.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:05.874+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:06.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:05 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:05.874+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:06.507 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:06 smithi050 podman[75681]: 2024-02-14 15:21:06.274201038 +0000 UTC m=+6.053821157 container cleanup 7f109cce6ef84c1bcae7dfc1061fd53d7ec7122d54055284bc588fcc489109f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-14T15:21:06.508 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:06 smithi050 bash[75681]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7 2024-02-14T15:21:06.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:06.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:06.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:06.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:06.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:06.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:06.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:06.308+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:06.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:06 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:06.381+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:06.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:06 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:06.381+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:06.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:06 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:06.381+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:06.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:06 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:06.381+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:06.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:06 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:06.381+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:06.892 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:06 smithi050 podman[75710]: 2024-02-14 15:21:06.50732567 +0000 UTC m=+0.655738160 container remove 7f109cce6ef84c1bcae7dfc1061fd53d7ec7122d54055284bc588fcc489109f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-14T15:21:07.211 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:06 smithi050 podman[75801]: 2024-02-14 15:21:06.856604025 +0000 UTC m=+0.018430058 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:21:07.211 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:06 smithi050 podman[75801]: 2024-02-14 15:21:06.966522805 +0000 UTC m=+0.128348821 container create eb54d81385dbb4a12c16f67fc5f3e16114157a3b6c35b27263c249a06db55c6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-14T15:21:07.212 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:06 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:06.897+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:07.212 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:06 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:06.897+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:07.212 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:06 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:06.897+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:07.212 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:06 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:06.897+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:07.213 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:06 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:06.897+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:07.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:06.897+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:07.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:06.897+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:07.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:06.897+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:07.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:06 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:06.897+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:07.514 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:07 smithi050 podman[75801]: 2024-02-14 15:21:07.211091651 +0000 UTC m=+0.372917683 container init eb54d81385dbb4a12c16f67fc5f3e16114157a3b6c35b27263c249a06db55c6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, io.buildah.version=1.33.2) 2024-02-14T15:21:07.514 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:07 smithi050 podman[75801]: 2024-02-14 15:21:07.214805194 +0000 UTC m=+0.376631218 container start eb54d81385dbb4a12c16f67fc5f3e16114157a3b6c35b27263c249a06db55c6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:21:07.514 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:07 smithi050 podman[75801]: 2024-02-14 15:21:07.294260783 +0000 UTC m=+0.456086816 container attach eb54d81385dbb4a12c16f67fc5f3e16114157a3b6c35b27263c249a06db55c6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, ceph=True) 2024-02-14T15:21:07.514 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:07 smithi050 podman[75801]: 2024-02-14 15:21:07.513977692 +0000 UTC m=+0.675803718 container died eb54d81385dbb4a12c16f67fc5f3e16114157a3b6c35b27263c249a06db55c6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux ) 2024-02-14T15:21:07.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:07.358+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:07.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:07.358+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:07.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:07.358+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:07.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:07.358+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:07.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:07 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:07.338+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:07.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:07 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:07.338+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:07.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:07 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:07.338+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:07.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:07 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:07.338+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:07.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:07 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:07.338+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:08.116 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 14 15:21:08 smithi050 podman[75827]: 2024-02-14 15:21:08.100428608 +0000 UTC m=+0.580817457 container remove eb54d81385dbb4a12c16f67fc5f3e16114157a3b6c35b27263c249a06db55c6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:21:08.117 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:07 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:07.929+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:08.117 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:07 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:07.929+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:08.117 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:07 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:07.929+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:08.117 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:07 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:07.929+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:08.118 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:07 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:07.929+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:08.122 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.7.service' 2024-02-14T15:21:08.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:08.172 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-14T15:21:08.173 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-14T15:21:08.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.8 2024-02-14T15:21:08.204 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:07.884+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:08.204 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:07.884+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:08.204 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:07.884+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:08.204 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:07 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:07.884+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:08.484 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 systemd[1]: Stopping Ceph osd.8 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:08.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:08.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:08.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:08.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:08.484 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:08.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:08.485 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:08.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:08.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:08 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:08.326+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:08.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:08 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:08.326+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:08.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:08 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:08.326+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:08.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:08 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:08.326+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:08.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:08 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:08.326+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:08.930 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:08.508+0000 7f1a01948700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:08.930 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:08.508+0000 7f1a01948700 -1 osd.8 122 *** Got signal Terminated *** 2024-02-14T15:21:08.930 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:08.508+0000 7f1a01948700 -1 osd.8 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:21:08.930 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:08.929+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:08.931 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:08.929+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:08.931 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:08.929+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:08.931 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:08 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:08.929+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:09.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:08 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:08.916+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:09.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:08 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:08.916+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:09.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:08 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:08.916+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:09.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:08 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:08.916+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:09.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:08 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:08.916+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:09.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:09.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:09.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:09.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:09.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:09.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:09.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:09.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:09.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:09.279+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:09.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:09.279+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:09.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:09.279+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:09.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:09.279+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:09.767 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:09 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:09.279+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:10.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:09.913+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:10.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:09.913+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:10.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:09.913+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:10.235 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:09 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:09.913+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:10.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:09 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:09.946+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:10.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:09 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:09.946+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:10.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:09 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:09.946+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:10.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:09 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:09.946+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:10.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:09 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:09.946+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:10.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:10.230+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:10.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:10.230+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:10.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:10.230+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:10.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:10.230+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:10.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:10 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:10.230+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:10.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:10.382+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:10.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:10.382+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:10.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:10.382+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:10.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:10.382+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:11.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:10.892+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:11.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:10.892+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:11.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:10.892+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:11.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:10 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:10.892+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:11.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:10 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:10.963+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:11.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:10 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:10.963+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:11.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:10 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:10.963+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:11.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:10 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:10.963+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:11.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:10 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:10.963+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:11.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:11 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:11.201+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:11.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:11 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:11.201+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:11.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:11 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:11.201+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:11.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:11 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:11.201+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:11.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:11 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:11.201+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:11.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:11 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:11.201+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:11.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:11.402+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:11.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:11.402+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:11.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:11.402+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:11.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:11.402+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:11.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:11.402+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:12.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:11.864+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:12.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:11.864+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:12.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:11.864+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:12.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:11 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:11.864+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:12.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:11.968+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:12.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:11.968+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:12.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:11.968+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:12.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:11.968+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:12.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:11.968+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:12.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:11 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:11.968+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:12.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:12.215+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:12.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:12.215+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:12.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:12.215+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:12.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:12.215+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:12.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:12.215+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:12.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:12 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:12.215+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:12.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:12.425+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:12.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:12.425+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:12.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:12.425+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:12.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:12.425+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:12.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:12.425+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:13.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:12.819+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.692883+0000 front 2024-02-14T15:20:15.692903+0000 (oldest deadline 2024-02-14T15:20:40.393039+0000) 2024-02-14T15:21:13.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:12.819+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.393479+0000 front 2024-02-14T15:20:20.393538+0000 (oldest deadline 2024-02-14T15:20:45.093780+0000) 2024-02-14T15:21:13.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:12.819+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:30.994940+0000 front 2024-02-14T15:20:30.994986+0000 (oldest deadline 2024-02-14T15:20:53.294659+0000) 2024-02-14T15:21:13.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:12.819+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.995668+0000 front 2024-02-14T15:20:37.995578+0000 (oldest deadline 2024-02-14T15:21:01.495502+0000) 2024-02-14T15:21:13.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:12 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8[55483]: 2024-02-14T15:21:12.819+0000 7f19fdf43700 -1 osd.8 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.497182+0000 front 2024-02-14T15:20:48.497211+0000 (oldest deadline 2024-02-14T15:21:11.898336+0000) 2024-02-14T15:21:13.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:13.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:13.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:13.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:13.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:13.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:13.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:13.225+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:13.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:13.225+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:13.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:13.225+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:13.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:13.225+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:13.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:13.225+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:13.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:13 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:13.225+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:13.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:13 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:13.403+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:13.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:13 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:13.403+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:13.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:13 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:13.403+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:13.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:13 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:13.403+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:13.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:13 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:13.403+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:14.359 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:13 smithi076 podman[75138]: 2024-02-14 15:21:13.996827528 +0000 UTC m=+5.626523860 container died 90d5d4249ed0d345574f9483961e2d902f8d37140eafaaa6c8963fcabe9ab110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-14T15:21:14.360 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:14 smithi076 podman[75138]: 2024-02-14 15:21:14.359382734 +0000 UTC m=+5.989079043 container cleanup 90d5d4249ed0d345574f9483961e2d902f8d37140eafaaa6c8963fcabe9ab110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-14T15:21:14.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.988+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:14.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.988+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:14.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.988+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:14.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.988+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:14.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.988+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:14.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:13 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:13.988+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:14.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:14 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:14.205+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:14.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:14 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:14.205+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:14.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:14 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:14.205+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:14.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:14 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:14.205+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:14.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:14 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:14.205+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:14.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:14 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:14.205+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:14.669 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:14 smithi076 bash[75138]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8 2024-02-14T15:21:14.670 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:14 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:14.409+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:14.670 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:14 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:14.409+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:14.670 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:14 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:14.409+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:14.670 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:14 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:14.409+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:14.671 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:14 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:14.409+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:14.929 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:14 smithi076 podman[75155]: 2024-02-14 15:21:14.66944455 +0000 UTC m=+0.665560590 container remove 90d5d4249ed0d345574f9483961e2d902f8d37140eafaaa6c8963fcabe9ab110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, ceph=True) 2024-02-14T15:21:15.233 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:14 smithi076 podman[75246]: 2024-02-14 15:21:14.882602211 +0000 UTC m=+0.017404662 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:21:15.234 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:15 smithi076 podman[75246]: 2024-02-14 15:21:15.003541631 +0000 UTC m=+0.138344073 container create 670b2708e496f97d49fff59a68e1678b8c7c7593f1df0fa2da74844a9865d007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-14T15:21:15.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:15.018+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:15.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:15.018+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:15.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:15.018+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:15.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:15.018+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:15.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:15.018+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:15.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:15 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:15.018+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:15.489 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:15 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:15.422+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:15.489 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:15 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:15.422+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:15.489 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:15 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:15.422+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:15.489 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:15 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:15.422+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:15.489 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:15 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:15.422+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:15.490 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:15 smithi076 podman[75246]: 2024-02-14 15:21:15.281269517 +0000 UTC m=+0.416071985 container init 670b2708e496f97d49fff59a68e1678b8c7c7593f1df0fa2da74844a9865d007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-14T15:21:15.490 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:15 smithi076 podman[75246]: 2024-02-14 15:21:15.284788542 +0000 UTC m=+0.419590973 container start 670b2708e496f97d49fff59a68e1678b8c7c7593f1df0fa2da74844a9865d007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-14T15:21:15.490 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:15 smithi076 podman[75246]: 2024-02-14 15:21:15.347557404 +0000 UTC m=+0.482359846 container attach 670b2708e496f97d49fff59a68e1678b8c7c7593f1df0fa2da74844a9865d007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-14T15:21:15.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:15.171+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:15.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:15.171+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:15.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:15.171+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:15.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:15.171+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:15.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:15.171+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:15.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:15 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:15.171+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:15.867 INFO:journalctl@ceph.osd.8.smithi076.stdout:Feb 14 15:21:15 smithi076 podman[75246]: 2024-02-14 15:21:15.585955987 +0000 UTC m=+0.720758418 container died 670b2708e496f97d49fff59a68e1678b8c7c7593f1df0fa2da74844a9865d007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-14T15:21:16.203 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.8.service' 2024-02-14T15:21:16.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:16.289 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-14T15:21:16.289 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-14T15:21:16.289 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.9 2024-02-14T15:21:16.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:16 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:16.040+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:16.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:16 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:16.040+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:16.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:16 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:16.040+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:16.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:16 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:16.040+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:16.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:16 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:16.040+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:16.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:16 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:16.040+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:16.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.145+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:16.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.145+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:16.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.145+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:16.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.145+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:16.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.145+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:16.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.145+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:16.518 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 systemd[1]: Stopping Ceph osd.9 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:16.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:16 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:16.449+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:16.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:16 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:16.449+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:16.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:16 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:16.449+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:16.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:16 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:16.449+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:16.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:16 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:16.449+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:17.016 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.598+0000 7f0ca6bc4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:17.016 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.598+0000 7f0ca6bc4700 -1 osd.9 122 *** Got signal Terminated *** 2024-02-14T15:21:17.016 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:16 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:16.598+0000 7f0ca6bc4700 -1 osd.9 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:21:17.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:17.010+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:17.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:17.010+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:17.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:17.010+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:17.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:17.010+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:17.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:17.010+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:17.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:17 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:17.010+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:17.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:17.127+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:17.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:17.127+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:17.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:17.127+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:17.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:17.127+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:17.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:17.127+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:17.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:17 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:17.127+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:17.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:17 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:17.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:17.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:17 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:17.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:17.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:17 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:17.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:17.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:17 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:17.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:17.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:17 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:17.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:18.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:18.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:18.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:18.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:18.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:18.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:18.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:18.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:18.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:18.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:18.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:18 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:18.013+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:18.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:18.119+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:18.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:18.119+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:18.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:18.119+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:18.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:18.119+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:18.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:18.119+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:18.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:18 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:18.119+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:18.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:18.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:18.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:18.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:18.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:18.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:18.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:18.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:18.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:18.400+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:19.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:19.056+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:19.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:19.056+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:19.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:19.056+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:19.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:19.056+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:19.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:19.056+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:19.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:19.056+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:19.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:19 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:19.056+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:19.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:19.082+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:19.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:19.082+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:19.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:19.082+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:19.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:19.082+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:19.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:19.082+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:19.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:19 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:19.082+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:19.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:19.392+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:19.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:19.392+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:19.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:19.392+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:19.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:19.392+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:19.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:19 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:19.392+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:20.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:20.072+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:20.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:20.072+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:20.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:20.072+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:20.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:20.072+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:20.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:20.072+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:20.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:20.072+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:20.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:20 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:20.072+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:20.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:20.078+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:20.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:20.078+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:20.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:20.078+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:20.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:20.078+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:20.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:20.078+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:20.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:20 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:20.078+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:20.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:20 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:20.360+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:20.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:20 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:20.360+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:20.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:20 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:20.360+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:20.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:20 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:20.360+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:20.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:20 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:20.360+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:21.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:21.045+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:21.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:21.045+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:21.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:21.045+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:21.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:21.045+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:21.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:21.045+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:21.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:21.045+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:21.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:21 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:21.045+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:21.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:21.029+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.606685+0000 front 2024-02-14T15:20:07.606619+0000 (oldest deadline 2024-02-14T15:20:29.306701+0000) 2024-02-14T15:21:21.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:21.029+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:13.908348+0000 front 2024-02-14T15:20:13.908294+0000 (oldest deadline 2024-02-14T15:20:40.309249+0000) 2024-02-14T15:21:21.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:21.029+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:20.310072+0000 front 2024-02-14T15:20:20.310053+0000 (oldest deadline 2024-02-14T15:20:46.210064+0000) 2024-02-14T15:21:21.516 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:21.029+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:32.612120+0000 front 2024-02-14T15:20:32.612147+0000 (oldest deadline 2024-02-14T15:20:57.312351+0000) 2024-02-14T15:21:21.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:21.029+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:37.312873+0000 front 2024-02-14T15:20:37.312973+0000 (oldest deadline 2024-02-14T15:21:02.012832+0000) 2024-02-14T15:21:21.517 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:21 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9[67756]: 2024-02-14T15:21:21.029+0000 7f0c9f1b7700 -1 osd.9 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:48.315156+0000 front 2024-02-14T15:20:48.315053+0000 (oldest deadline 2024-02-14T15:21:10.614996+0000) 2024-02-14T15:21:21.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:21.313+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:21.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:21.313+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:21.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:21.313+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:21.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:21.313+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:21.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:21.313+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:21.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:21 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:21.313+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:22.266 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:21 smithi017 podman[100214]: 2024-02-14 15:21:21.99040246 +0000 UTC m=+5.510964552 container died d25c302cc3b9518f56d2147e56542fb10e3874f823b6b4edb3a6440965d26ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2) 2024-02-14T15:21:22.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:22.054+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:22.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:22.054+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:22.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:22.054+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:22.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:22.054+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:22.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:22.054+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:22.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:22.054+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:22.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:22 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:22.054+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:22.733 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:22 smithi017 podman[100214]: 2024-02-14 15:21:22.45703122 +0000 UTC m=+5.977593358 container cleanup d25c302cc3b9518f56d2147e56542fb10e3874f823b6b4edb3a6440965d26ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-14T15:21:22.733 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:22 smithi017 bash[100214]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9 2024-02-14T15:21:22.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:22.320+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:22.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:22.320+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:22.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:22.320+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:22.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:22.320+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:22.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:22.320+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:22.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:22.320+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:23.016 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:22 smithi017 podman[100229]: 2024-02-14 15:21:22.733062732 +0000 UTC m=+0.741169068 container remove d25c302cc3b9518f56d2147e56542fb10e3874f823b6b4edb3a6440965d26ccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:21:23.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:23.091+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:23.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:23.091+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:23.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:23.091+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:23.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:23.091+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:23.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:23.091+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:23.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:23.091+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:23.394 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:23 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:23.091+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:23.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:23 smithi017 podman[100324]: 2024-02-14 15:21:22.977122995 +0000 UTC m=+0.017923123 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:21:23.395 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:23 smithi017 podman[100324]: 2024-02-14 15:21:23.092131124 +0000 UTC m=+0.132931236 container create 6c81a95f551e1f7332b4b8e6f51e461a2f95053e4c3ffdf66e8c9484832560a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-14T15:21:23.395 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:23 smithi017 podman[100324]: 2024-02-14 15:21:23.319898097 +0000 UTC m=+0.360698211 container init 6c81a95f551e1f7332b4b8e6f51e461a2f95053e4c3ffdf66e8c9484832560a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD) 2024-02-14T15:21:23.395 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:23 smithi017 podman[100324]: 2024-02-14 15:21:23.323105024 +0000 UTC m=+0.363905137 container start 6c81a95f551e1f7332b4b8e6f51e461a2f95053e4c3ffdf66e8c9484832560a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0) 2024-02-14T15:21:23.395 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:23 smithi017 podman[100324]: 2024-02-14 15:21:23.394270909 +0000 UTC m=+0.435071023 container attach 6c81a95f551e1f7332b4b8e6f51e461a2f95053e4c3ffdf66e8c9484832560a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-14T15:21:23.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:23.281+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:23.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:23.281+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:23.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:23.281+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:23.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:23.281+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:23.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:23.281+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:23.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:23 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:23.281+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:23.766 INFO:journalctl@ceph.osd.9.smithi017.stdout:Feb 14 15:21:23 smithi017 podman[100324]: 2024-02-14 15:21:23.613458561 +0000 UTC m=+0.654258684 container died 6c81a95f551e1f7332b4b8e6f51e461a2f95053e4c3ffdf66e8c9484832560a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-14T15:21:24.194 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.9.service' 2024-02-14T15:21:24.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:24.239 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-14T15:21:24.239 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-14T15:21:24.239 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.10 2024-02-14T15:21:24.392 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.138+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:24.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.138+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:24.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.138+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:24.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.138+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:24.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.138+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:24.393 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.138+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:24.394 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.138+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:24.394 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 systemd[1]: Stopping Ceph osd.10 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:24.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:24.285+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:24.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:24.285+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:24.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:24.285+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:24.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:24.285+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:24.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:24.285+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:24.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:24 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:24.285+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:24.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.594+0000 7f28e1c33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:24.892 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.594+0000 7f28e1c33700 -1 osd.10 122 *** Got signal Terminated *** 2024-02-14T15:21:24.893 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:24 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:24.594+0000 7f28e1c33700 -1 osd.10 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:21:25.642 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:25 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:25.179+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:25.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:25 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:25.179+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:25.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:25 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:25.179+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:25.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:25 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:25.179+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:25.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:25 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:25.179+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:25.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:25 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:25.179+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:25.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:25 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:25.179+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:25.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:25 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:25.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:25.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:25 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:25.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:25.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:25 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:25.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:25.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:25 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:25.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:25.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:25 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:25.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:25.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:25 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:25.272+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:26.642 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:26 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:26.147+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:26.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:26 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:26.147+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:26.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:26 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:26.147+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:26.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:26 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:26.147+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:26.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:26 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:26.147+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:26.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:26 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:26.147+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:26.644 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:26 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:26.147+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:26.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:26.289+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:26.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:26.289+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:26.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:26.289+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:26.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:26.289+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:26.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:26.289+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:26.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:26.289+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:27.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:27 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:27.148+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:27.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:27 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:27.148+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:27.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:27 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:27.148+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:27.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:27 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:27.148+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:27.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:27 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:27.148+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:27.644 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:27 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:27.148+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:27.644 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:27 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:27.148+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:27.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:27 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:27.290+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:27.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:27 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:27.290+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:27.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:27 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:27.290+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:27.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:27 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:27.290+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:27.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:27 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:27.290+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:27.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:27 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:27.290+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:28.642 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:28 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:28.197+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:28.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:28 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:28.197+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:28.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:28 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:28.197+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:28.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:28 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:28.197+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:28.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:28 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:28.197+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:28.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:28 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:28.197+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:28.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:28 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:28.197+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:28.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:28.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:28.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:28.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:28.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:28.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:28.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:28.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:28.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:28.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:28.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:28.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:28.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:28 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:28.311+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:29.642 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:29 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:29.183+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:02.103282+0000 front 2024-02-14T15:20:02.103359+0000 (oldest deadline 2024-02-14T15:20:30.303720+0000) 2024-02-14T15:21:29.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:29 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:29.183+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:15.505684+0000 front 2024-02-14T15:20:15.505654+0000 (oldest deadline 2024-02-14T15:20:38.405840+0000) 2024-02-14T15:21:29.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:29 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:29.183+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:21.307246+0000 front 2024-02-14T15:20:21.307286+0000 (oldest deadline 2024-02-14T15:20:46.607240+0000) 2024-02-14T15:21:29.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:29 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:29.183+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2024-02-14T15:20:31.109636+0000 front 2024-02-14T15:20:31.109805+0000 (oldest deadline 2024-02-14T15:20:54.609445+0000) 2024-02-14T15:21:29.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:29 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:29.183+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:38.710409+0000 front 2024-02-14T15:20:38.710380+0000 (oldest deadline 2024-02-14T15:21:03.410472+0000) 2024-02-14T15:21:29.644 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:29 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:29.183+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:47.312319+0000 front 2024-02-14T15:20:47.312373+0000 (oldest deadline 2024-02-14T15:21:11.412374+0000) 2024-02-14T15:21:29.644 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:29 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10[61962]: 2024-02-14T15:21:29.183+0000 7f28da226700 -1 osd.10 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:57.313513+0000 front 2024-02-14T15:20:57.313587+0000 (oldest deadline 2024-02-14T15:21:19.013512+0000) 2024-02-14T15:21:29.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:29 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:29.333+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:29.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:29 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:29.333+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:29.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:29 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:29.333+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:29.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:29 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:29.333+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:29.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:29 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:29.333+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:29.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:29 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:29.333+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:29.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:29 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:29.333+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:30.345 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:30 smithi050 podman[75906]: 2024-02-14 15:21:30.063579439 +0000 UTC m=+5.624946507 container died 23ec8f18b8a1aa6c4d32bd200ab16bee7b4e5264385ff03a6e0896ebb28a6b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-02-14T15:21:30.642 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:30 smithi050 podman[75906]: 2024-02-14 15:21:30.453029005 +0000 UTC m=+6.014396057 container cleanup 23ec8f18b8a1aa6c4d32bd200ab16bee7b4e5264385ff03a6e0896ebb28a6b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-14T15:21:30.642 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:30 smithi050 bash[75906]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10 2024-02-14T15:21:30.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:30 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:30.342+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:30.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:30 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:30.342+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:30.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:30 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:30.342+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:30.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:30 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:30.342+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:30.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:30 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:30.342+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:30.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:30 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:30.342+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:30.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:30 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:30.342+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:31.025 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:30 smithi050 podman[75923]: 2024-02-14 15:21:30.687799196 +0000 UTC m=+0.617623839 container remove 23ec8f18b8a1aa6c4d32bd200ab16bee7b4e5264385ff03a6e0896ebb28a6b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-14T15:21:31.276 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:31 smithi050 podman[76016]: 2024-02-14 15:21:30.928132936 +0000 UTC m=+0.018223511 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:21:31.276 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:31 smithi050 podman[76016]: 2024-02-14 15:21:31.038659293 +0000 UTC m=+0.128749896 container create e14555d97c3f055e6126b6acfeee1edf6efeaac5324257e436ede19826aaec46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-14T15:21:31.642 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:31 smithi050 podman[76016]: 2024-02-14 15:21:31.349804428 +0000 UTC m=+0.439895000 container init e14555d97c3f055e6126b6acfeee1edf6efeaac5324257e436ede19826aaec46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:21:31.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:31 smithi050 podman[76016]: 2024-02-14 15:21:31.353340311 +0000 UTC m=+0.443430867 container start e14555d97c3f055e6126b6acfeee1edf6efeaac5324257e436ede19826aaec46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-14T15:21:31.643 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:31 smithi050 podman[76016]: 2024-02-14 15:21:31.424118168 +0000 UTC m=+0.514208742 container attach e14555d97c3f055e6126b6acfeee1edf6efeaac5324257e436ede19826aaec46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-14T15:21:31.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:31 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:31.375+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:31.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:31 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:31.375+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:31.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:31 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:31.375+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:31.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:31 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:31.375+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:31.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:31 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:31.375+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:31.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:31 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:31.375+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:31.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:31 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:31.375+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:31.934 INFO:journalctl@ceph.osd.10.smithi050.stdout:Feb 14 15:21:31 smithi050 podman[76016]: 2024-02-14 15:21:31.652535357 +0000 UTC m=+0.742625915 container died e14555d97c3f055e6126b6acfeee1edf6efeaac5324257e436ede19826aaec46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:21:32.231 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.10.service' 2024-02-14T15:21:32.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:32.281 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-14T15:21:32.282 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-14T15:21:32.282 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.11 2024-02-14T15:21:32.604 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 systemd[1]: Stopping Ceph osd.11 for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:32.604 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:32.605 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:32.605 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:32.605 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:32.605 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:32.605 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:32.606 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.415+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:32.606 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.603+0000 7f25f0424700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:32.606 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.603+0000 7f25f0424700 -1 osd.11 122 *** Got signal Terminated *** 2024-02-14T15:21:32.606 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:32 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:32.603+0000 7f25f0424700 -1 osd.11 122 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T15:21:33.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:33 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:33.446+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:33.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:33 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:33.446+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:33.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:33 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:33.446+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:33.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:33 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:33.446+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:33.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:33 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:33.446+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:33.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:33 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:33.446+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:33.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:33 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:33.446+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:34.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:34.475+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:34.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:34.475+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:34.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:34.475+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:34.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:34.475+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:34.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:34.475+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:34.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:34.475+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:34.736 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:34 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:34.475+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:35.984 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:35.485+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:35.984 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:35.485+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:35.985 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:35.485+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:35.985 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:35.485+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:35.985 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:35.485+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:35.985 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:35.485+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:35.985 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:35 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:35.485+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:36.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:36.482+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:36.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:36.482+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:36.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:36.482+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:36.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:36.482+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:36.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:36.482+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:36.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:36.482+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:36.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:36 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:36.482+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:37.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2024-02-14T15:20:07.818390+0000 front 2024-02-14T15:20:07.818556+0000 (oldest deadline 2024-02-14T15:20:34.219129+0000) 2024-02-14T15:21:37.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-02-14T15:20:14.219577+0000 front 2024-02-14T15:20:14.219696+0000 (oldest deadline 2024-02-14T15:20:38.319702+0000) 2024-02-14T15:21:37.734 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6804 osd.2 since back 2024-02-14T15:20:24.220599+0000 front 2024-02-14T15:20:24.220812+0000 (oldest deadline 2024-02-14T15:20:47.021477+0000) 2024-02-14T15:21:37.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-02-14T15:20:39.625653+0000 front 2024-02-14T15:20:39.625682+0000 (oldest deadline 2024-02-14T15:21:02.525564+0000) 2024-02-14T15:21:37.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6812 osd.5 since back 2024-02-14T15:20:46.928288+0000 front 2024-02-14T15:20:46.928342+0000 (oldest deadline 2024-02-14T15:21:11.028411+0000) 2024-02-14T15:21:37.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2024-02-14T15:20:56.929210+0000 front 2024-02-14T15:20:56.929267+0000 (oldest deadline 2024-02-14T15:21:21.029197+0000) 2024-02-14T15:21:37.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-02-14T15:21:04.530498+0000 front 2024-02-14T15:21:04.530423+0000 (oldest deadline 2024-02-14T15:21:27.430039+0000) 2024-02-14T15:21:37.735 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11[62566]: 2024-02-14T15:21:37.453+0000 7f25eca1f700 -1 osd.11 122 heartbeat_check: no reply from 172.21.15.76:6820 osd.8 since back 2024-02-14T15:21:12.730908+0000 front 2024-02-14T15:21:12.730946+0000 (oldest deadline 2024-02-14T15:21:36.831008+0000) 2024-02-14T15:21:38.233 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:37 smithi076 podman[75350]: 2024-02-14 15:21:37.948561307 +0000 UTC m=+5.472309988 container died 02282b60983bda31adcb94f8c188ceb65ce772149ddef2acdd99ab985a269e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:21:38.571 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:38 smithi076 podman[75350]: 2024-02-14 15:21:38.320526353 +0000 UTC m=+5.844275041 container cleanup 02282b60983bda31adcb94f8c188ceb65ce772149ddef2acdd99ab985a269e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:21:38.571 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:38 smithi076 bash[75350]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11 2024-02-14T15:21:38.863 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:38 smithi076 podman[75367]: 2024-02-14 15:21:38.571542984 +0000 UTC m=+0.615659412 container remove 02282b60983bda31adcb94f8c188ceb65ce772149ddef2acdd99ab985a269e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-14T15:21:39.166 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:38 smithi076 podman[75457]: 2024-02-14 15:21:38.863894275 +0000 UTC m=+0.108551355 container create 5bbf427a433ddd8d29126955549f43449508a9bc918eda06411e21915161b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2) 2024-02-14T15:21:39.166 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:38 smithi076 podman[75457]: 2024-02-14 15:21:38.772686248 +0000 UTC m=+0.017343301 image pull 87bb2d90c3bfdbfc7753e96fe432d72f52eead7ee57204154cc1e27e7313fa90 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836 2024-02-14T15:21:39.166 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:39 smithi076 podman[75457]: 2024-02-14 15:21:39.083304761 +0000 UTC m=+0.327961798 container init 5bbf427a433ddd8d29126955549f43449508a9bc918eda06411e21915161b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True) 2024-02-14T15:21:39.166 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:39 smithi076 podman[75457]: 2024-02-14 15:21:39.086598421 +0000 UTC m=+0.331255450 container start 5bbf427a433ddd8d29126955549f43449508a9bc918eda06411e21915161b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:21:39.167 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:39 smithi076 podman[75457]: 2024-02-14 15:21:39.166349326 +0000 UTC m=+0.411006367 container attach 5bbf427a433ddd8d29126955549f43449508a9bc918eda06411e21915161b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:21:39.483 INFO:journalctl@ceph.osd.11.smithi076.stdout:Feb 14 15:21:39 smithi076 podman[75457]: 2024-02-14 15:21:39.374247959 +0000 UTC m=+0.618904997 container died 5bbf427a433ddd8d29126955549f43449508a9bc918eda06411e21915161b0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-14T15:21:40.004 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@osd.11.service' 2024-02-14T15:21:40.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:40.046 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-14T15:21:40.046 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-14T15:21:40.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.a 2024-02-14T15:21:40.384 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:21:40 smithi017 systemd[1]: Stopping Ceph mds.a for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:40.766 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:21:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a[74160]: 2024-02-14T15:21:40.383+0000 7f9083aaa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:40.766 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:21:40 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a[74160]: 2024-02-14T15:21:40.383+0000 7f9083aaa700 -1 mds.a *** got signal Terminated *** 2024-02-14T15:21:44.516 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:21:44 smithi017 podman[100440]: 2024-02-14 15:21:44.146707199 +0000 UTC m=+3.901715734 container died 5695e59a76e79d52b60b3d9b1865a1c6019bdfc0bad79085a25197c5acb3f485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2) 2024-02-14T15:21:45.016 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:21:44 smithi017 podman[100440]: 2024-02-14 15:21:44.566064194 +0000 UTC m=+4.321072725 container cleanup 5695e59a76e79d52b60b3d9b1865a1c6019bdfc0bad79085a25197c5acb3f485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2) 2024-02-14T15:21:45.016 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:21:44 smithi017 bash[100440]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a 2024-02-14T15:21:45.017 INFO:journalctl@ceph.mds.a.smithi017.stdout:Feb 14 15:21:44 smithi017 podman[100454]: 2024-02-14 15:21:44.783891967 +0000 UTC m=+0.636470498 container remove 5695e59a76e79d52b60b3d9b1865a1c6019bdfc0bad79085a25197c5acb3f485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:21:45.036 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.a.service' 2024-02-14T15:21:45.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:45.074 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-14T15:21:45.075 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-14T15:21:45.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.d 2024-02-14T15:21:45.395 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:21:45 smithi017 systemd[1]: Stopping Ceph mds.d for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:45.766 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:21:45 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d[73203]: 2024-02-14T15:21:45.394+0000 7ffa0228b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:45.767 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:21:45 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d[73203]: 2024-02-14T15:21:45.394+0000 7ffa0228b700 -1 mds.d *** got signal Terminated *** 2024-02-14T15:21:48.612 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:21:48 smithi017 podman[100561]: 2024-02-14 15:21:48.35672498 +0000 UTC m=+3.063995049 container died 0686cf100b386a546f422a715eafb9a99af52c65fb22060034d5cb9161cfd9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-14T15:21:49.016 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:21:48 smithi017 podman[100561]: 2024-02-14 15:21:48.694229326 +0000 UTC m=+3.401499401 container cleanup 0686cf100b386a546f422a715eafb9a99af52c65fb22060034d5cb9161cfd9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T15:21:49.017 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:21:48 smithi017 bash[100561]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d 2024-02-14T15:21:49.017 INFO:journalctl@ceph.mds.d.smithi017.stdout:Feb 14 15:21:48 smithi017 podman[100575]: 2024-02-14 15:21:48.853990166 +0000 UTC m=+0.492992788 container remove 0686cf100b386a546f422a715eafb9a99af52c65fb22060034d5cb9161cfd9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-d, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2) 2024-02-14T15:21:49.068 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.d.service' 2024-02-14T15:21:49.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:49.104 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-14T15:21:49.104 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-14T15:21:49.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.g 2024-02-14T15:21:49.448 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:21:49 smithi017 systemd[1]: Stopping Ceph mds.g for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:49.766 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:21:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g[73811]: 2024-02-14T15:21:49.447+0000 7f3207bc4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:49.766 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:21:49 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g[73811]: 2024-02-14T15:21:49.447+0000 7f3207bc4700 -1 mds.g *** got signal Terminated *** 2024-02-14T15:21:53.766 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:21:53 smithi017 podman[100681]: 2024-02-14 15:21:53.365265964 +0000 UTC m=+4.043081262 container died f8d1ceeaf43d4716689b0a1df871ba624495cb747b129383f895144b3df9df64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2) 2024-02-14T15:21:54.140 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:21:53 smithi017 podman[100681]: 2024-02-14 15:21:53.873037453 +0000 UTC m=+4.550852748 container cleanup f8d1ceeaf43d4716689b0a1df871ba624495cb747b129383f895144b3df9df64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-14T15:21:54.140 INFO:journalctl@ceph.mds.g.smithi017.stdout:Feb 14 15:21:53 smithi017 bash[100681]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-g 2024-02-14T15:21:54.293 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.g.service' 2024-02-14T15:21:54.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:54.327 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-14T15:21:54.327 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-14T15:21:54.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.j 2024-02-14T15:21:54.643 INFO:journalctl@ceph.mds.j.smithi017.stdout:Feb 14 15:21:54 smithi017 systemd[1]: Stopping Ceph mds.j for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:55.016 INFO:journalctl@ceph.mds.j.smithi017.stdout:Feb 14 15:21:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-j[72851]: 2024-02-14T15:21:54.642+0000 7f3cc107f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:55.016 INFO:journalctl@ceph.mds.j.smithi017.stdout:Feb 14 15:21:54 smithi017 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-j[72851]: 2024-02-14T15:21:54.642+0000 7f3cc107f700 -1 mds.j *** got signal Terminated *** 2024-02-14T15:21:57.766 INFO:journalctl@ceph.mds.j.smithi017.stdout:Feb 14 15:21:57 smithi017 podman[100802]: 2024-02-14 15:21:57.484328032 +0000 UTC m=+2.939830544 container died 8342a096b4218f2ee7d70f417f54e1bab364085320d9b0c53262109329ba8647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-j, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa) 2024-02-14T15:21:58.069 INFO:journalctl@ceph.mds.j.smithi017.stdout:Feb 14 15:21:57 smithi017 podman[100802]: 2024-02-14 15:21:57.859641229 +0000 UTC m=+3.315143750 container cleanup 8342a096b4218f2ee7d70f417f54e1bab364085320d9b0c53262109329ba8647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-j, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131) 2024-02-14T15:21:58.069 INFO:journalctl@ceph.mds.j.smithi017.stdout:Feb 14 15:21:57 smithi017 bash[100802]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-j 2024-02-14T15:21:58.207 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.j.service' 2024-02-14T15:21:58.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:21:58.243 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-14T15:21:58.243 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-14T15:21:58.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.b 2024-02-14T15:21:58.642 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:21:58 smithi050 systemd[1]: Stopping Ceph mds.b for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:21:58.642 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:21:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b[67204]: 2024-02-14T15:21:58.519+0000 7f84b81b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:21:58.642 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:21:58 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b[67204]: 2024-02-14T15:21:58.519+0000 7f84b81b3700 -1 mds.b *** got signal Terminated *** 2024-02-14T15:22:01.870 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:22:01 smithi050 podman[76120]: 2024-02-14 15:22:01.612557771 +0000 UTC m=+3.167958422 container died c345db9c2b627fd2aa4449387a971966f8416266865935517a0f3609fbd1c9b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:22:02.142 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:22:01 smithi050 podman[76120]: 2024-02-14 15:22:01.961342091 +0000 UTC m=+3.516742731 container cleanup c345db9c2b627fd2aa4449387a971966f8416266865935517a0f3609fbd1c9b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-14T15:22:02.142 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:22:01 smithi050 bash[76120]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b 2024-02-14T15:22:02.392 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.b.service' 2024-02-14T15:22:02.420 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:22:02 smithi050 podman[76134]: 2024-02-14 15:22:02.145920032 +0000 UTC m=+0.526523871 container remove c345db9c2b627fd2aa4449387a971966f8416266865935517a0f3609fbd1c9b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-b, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True) 2024-02-14T15:22:02.420 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:22:02 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.b.service: Deactivated successfully. 2024-02-14T15:22:02.421 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:22:02 smithi050 systemd[1]: Stopped Ceph mds.b for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:22:02.421 INFO:journalctl@ceph.mds.b.smithi050.stdout:Feb 14 15:22:02 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.b.service: Consumed 1min 41.188s CPU time. 2024-02-14T15:22:02.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:02.434 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-14T15:22:02.434 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-14T15:22:02.435 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.e 2024-02-14T15:22:02.757 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:02 smithi050 systemd[1]: Stopping Ceph mds.e for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:22:03.142 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e[66591]: 2024-02-14T15:22:02.756+0000 7fb7bfb3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:22:03.142 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:02 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e[66591]: 2024-02-14T15:22:02.756+0000 7fb7bfb3f700 -1 mds.e *** got signal Terminated *** 2024-02-14T15:22:05.892 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:05 smithi050 podman[76241]: 2024-02-14 15:22:05.61571483 +0000 UTC m=+2.956606208 container died 6b8e546e9b1e32438b403f6cafafb615f872dfbc1c05f734ee0d6cc636d049f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0) 2024-02-14T15:22:06.358 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:06 smithi050 podman[76241]: 2024-02-14 15:22:06.156847275 +0000 UTC m=+3.497738645 container cleanup 6b8e546e9b1e32438b403f6cafafb615f872dfbc1c05f734ee0d6cc636d049f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2) 2024-02-14T15:22:06.358 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:06 smithi050 bash[76241]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e 2024-02-14T15:22:06.596 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.e.service' 2024-02-14T15:22:06.624 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:06 smithi050 podman[76267]: 2024-02-14 15:22:06.357751241 +0000 UTC m=+0.732709300 container remove 6b8e546e9b1e32438b403f6cafafb615f872dfbc1c05f734ee0d6cc636d049f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2) 2024-02-14T15:22:06.625 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:06 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.e.service: Deactivated successfully. 2024-02-14T15:22:06.625 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:06 smithi050 systemd[1]: Stopped Ceph mds.e for 867ff6ea-cb48-11ee-95b9-87774f69a715. 2024-02-14T15:22:06.625 INFO:journalctl@ceph.mds.e.smithi050.stdout:Feb 14 15:22:06 smithi050 systemd[1]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.e.service: Consumed 1min 1.166s CPU time. 2024-02-14T15:22:06.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:06.638 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-14T15:22:06.638 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-14T15:22:06.638 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h 2024-02-14T15:22:06.892 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:22:06 smithi050 systemd[1]: Stopping Ceph mds.h for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:22:07.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:22:06 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[73351]: 2024-02-14T15:22:06.985+0000 7f412a8c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:22:07.392 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:22:06 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h[73351]: 2024-02-14T15:22:06.985+0000 7f412a8c6700 -1 mds.h *** got signal Terminated *** 2024-02-14T15:22:11.253 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:22:10 smithi050 podman[76372]: 2024-02-14 15:22:10.996954703 +0000 UTC m=+4.133761120 container died 9da18a661f5c618383223b23593080b8effb14b38d1a670d3850aa84e3012096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T15:22:11.520 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:22:11 smithi050 podman[76372]: 2024-02-14 15:22:11.343898296 +0000 UTC m=+4.480704706 container cleanup 9da18a661f5c618383223b23593080b8effb14b38d1a670d3850aa84e3012096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0) 2024-02-14T15:22:11.520 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:22:11 smithi050 bash[76372]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h 2024-02-14T15:22:11.771 INFO:journalctl@ceph.mds.h.smithi050.stdout:Feb 14 15:22:11 smithi050 podman[76385]: 2024-02-14 15:22:11.520090108 +0000 UTC m=+0.520050324 container remove 9da18a661f5c618383223b23593080b8effb14b38d1a670d3850aa84e3012096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-h, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True) 2024-02-14T15:22:11.775 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.h.service' 2024-02-14T15:22:11.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:11.815 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-14T15:22:11.815 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-14T15:22:11.815 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.k 2024-02-14T15:22:12.142 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:22:11 smithi050 systemd[1]: Stopping Ceph mds.k for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:22:12.642 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:22:12 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k[66266]: 2024-02-14T15:22:12.146+0000 7fe56b239700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:22:12.642 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:22:12 smithi050 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k[66266]: 2024-02-14T15:22:12.146+0000 7fe56b239700 -1 mds.k *** got signal Terminated *** 2024-02-14T15:22:14.948 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:22:14 smithi050 podman[76494]: 2024-02-14 15:22:14.680166204 +0000 UTC m=+2.635030674 container died bfcf9c1b0bed5e3d89451a09494b0890751a7fc020e3786cf44b82a70c4f8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-14T15:22:15.315 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:22:15 smithi050 podman[76494]: 2024-02-14 15:22:15.047237653 +0000 UTC m=+3.002102111 container cleanup bfcf9c1b0bed5e3d89451a09494b0890751a7fc020e3786cf44b82a70c4f8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-14T15:22:15.315 INFO:journalctl@ceph.mds.k.smithi050.stdout:Feb 14 15:22:15 smithi050 bash[76494]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-k 2024-02-14T15:22:15.476 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.k.service' 2024-02-14T15:22:15.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:15.515 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-14T15:22:15.515 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-14T15:22:15.515 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.c 2024-02-14T15:22:15.984 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:22:15 smithi076 systemd[1]: Stopping Ceph mds.c for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:22:15.984 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:22:15 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:22:15.785+0000 7f38999fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:22:15.984 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:22:15 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c[67602]: 2024-02-14T15:22:15.785+0000 7f38999fe700 -1 mds.c *** got signal Terminated *** 2024-02-14T15:22:17.895 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:22:17 smithi076 podman[75562]: 2024-02-14 15:22:17.639658422 +0000 UTC m=+1.944756505 container died 2323ce9ebc75637b95dbfbb00eb3e808160b8258d826201f4c3edfbaa7948c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-14T15:22:18.234 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:22:18 smithi076 podman[75562]: 2024-02-14 15:22:18.078057063 +0000 UTC m=+2.383155145 container cleanup 2323ce9ebc75637b95dbfbb00eb3e808160b8258d826201f4c3edfbaa7948c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-14T15:22:18.234 INFO:journalctl@ceph.mds.c.smithi076.stdout:Feb 14 15:22:18 smithi076 bash[75562]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-c 2024-02-14T15:22:18.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.c.service' 2024-02-14T15:22:18.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:18.527 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-14T15:22:18.528 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-14T15:22:18.528 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.f 2024-02-14T15:22:18.849 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:22:18 smithi076 systemd[1]: Stopping Ceph mds.f for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:22:19.234 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:22:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f[67290]: 2024-02-14T15:22:18.848+0000 7fa477972700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:22:19.234 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:22:18 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f[67290]: 2024-02-14T15:22:18.848+0000 7fa477972700 -1 mds.f *** got signal Terminated *** 2024-02-14T15:22:21.932 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:22:21 smithi076 podman[75695]: 2024-02-14 15:22:21.646959359 +0000 UTC m=+2.905462429 container died fbbd60dda08a5b307eaaf0f675463797033482397c13103e75e18e1004896835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-14T15:22:21.932 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:22:21 smithi076 podman[75695]: 2024-02-14 15:22:21.931933548 +0000 UTC m=+3.190436626 container cleanup fbbd60dda08a5b307eaaf0f675463797033482397c13103e75e18e1004896835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2) 2024-02-14T15:22:22.234 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:22:21 smithi076 bash[75695]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f 2024-02-14T15:22:22.234 INFO:journalctl@ceph.mds.f.smithi076.stdout:Feb 14 15:22:22 smithi076 podman[75709]: 2024-02-14 15:22:22.141354656 +0000 UTC m=+0.490630846 container remove fbbd60dda08a5b307eaaf0f675463797033482397c13103e75e18e1004896835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-f, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-14T15:22:22.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.f.service' 2024-02-14T15:22:22.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:22.308 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-14T15:22:22.309 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-14T15:22:22.309 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.i 2024-02-14T15:22:22.652 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:22:22 smithi076 systemd[1]: Stopping Ceph mds.i for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:22:22.983 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:22:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i[67909]: 2024-02-14T15:22:22.650+0000 7f21fa94a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:22:22.984 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:22:22 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i[67909]: 2024-02-14T15:22:22.650+0000 7f21fa94a700 -1 mds.i *** got signal Terminated *** 2024-02-14T15:22:25.960 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:22:25 smithi076 podman[75814]: 2024-02-14 15:22:25.644211997 +0000 UTC m=+3.110157414 container died 4a5faa0da1a299c92d6c186571cf0af68ef77b8e369a4f521f81b8b0f745e010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-14T15:22:25.961 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:22:25 smithi076 podman[75814]: 2024-02-14 15:22:25.960677651 +0000 UTC m=+3.426623062 container cleanup 4a5faa0da1a299c92d6c186571cf0af68ef77b8e369a4f521f81b8b0f745e010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:22:26.233 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:22:25 smithi076 bash[75814]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i 2024-02-14T15:22:26.234 INFO:journalctl@ceph.mds.i.smithi076.stdout:Feb 14 15:22:26 smithi076 podman[75828]: 2024-02-14 15:22:26.136781914 +0000 UTC m=+0.493803042 container remove 4a5faa0da1a299c92d6c186571cf0af68ef77b8e369a4f521f81b8b0f745e010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-14T15:22:26.357 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.i.service' 2024-02-14T15:22:26.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:26.394 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-14T15:22:26.394 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-14T15:22:26.394 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l 2024-02-14T15:22:26.657 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:22:26 smithi076 systemd[1]: Stopping Ceph mds.l for 867ff6ea-cb48-11ee-95b9-87774f69a715... 2024-02-14T15:22:26.983 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:22:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[73142]: 2024-02-14T15:22:26.721+0000 7f9cd9b4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T15:22:26.984 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:22:26 smithi076 ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l[73142]: 2024-02-14T15:22:26.721+0000 7f9cd9b4e700 -1 mds.l *** got signal Terminated *** 2024-02-14T15:22:29.932 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:22:29 smithi076 podman[75935]: 2024-02-14 15:22:29.64146033 +0000 UTC m=+3.035628353 container died 06398c205308b932ad9d6145c19ed4511f6e31e533eb48e16d1ce2d1b7d180fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-14T15:22:30.234 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:22:30 smithi076 podman[75935]: 2024-02-14 15:22:30.031348389 +0000 UTC m=+3.425516403 container cleanup 06398c205308b932ad9d6145c19ed4511f6e31e533eb48e16d1ce2d1b7d180fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:191d989ab4898b16a15a61605f682f0e88010be49063b30f29a026a527875836, name=ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240206.051942-a6fe3fa, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-14T15:22:30.234 INFO:journalctl@ceph.mds.l.smithi076.stdout:Feb 14 15:22:30 smithi076 bash[75935]: ceph-867ff6ea-cb48-11ee-95b9-87774f69a715-mds-l 2024-02-14T15:22:30.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-867ff6ea-cb48-11ee-95b9-87774f69a715@mds.l.service' 2024-02-14T15:22:30.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T15:22:30.435 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-14T15:22:30.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T15:22:30.632 INFO:teuthology.orchestra.run.smithi017.stdout:Deleting cluster with fsid: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T15:22:32.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T15:22:33.118 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T15:22:35.731 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T15:22:35.930 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T15:22:38.450 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T15:22:38.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T15:22:38.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T15:22:38.554 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-14T15:22:38.557 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi017/crash 2024-02-14T15:22:38.558 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash -- . 2024-02-14T15:22:38.603 INFO:teuthology.orchestra.run.smithi017.stderr:tar: /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T15:22:38.604 INFO:teuthology.orchestra.run.smithi017.stderr:tar: Error is not recoverable: exiting now 2024-02-14T15:22:38.606 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi050/crash 2024-02-14T15:22:38.607 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash -- . 2024-02-14T15:22:38.652 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T15:22:38.653 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-02-14T15:22:38.655 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi076/crash 2024-02-14T15:22:38.656 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash -- . 2024-02-14T15:22:38.696 INFO:teuthology.orchestra.run.smithi076.stderr:tar: /var/lib/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T15:22:38.696 INFO:teuthology.orchestra.run.smithi076.stderr:tar: Error is not recoverable: exiting now 2024-02-14T15:22:38.697 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-14T15:22:38.697 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-14T15:22:38.733 INFO:teuthology.orchestra.run.smithi017.stdout:2024-02-14T14:54:41.038982+0000 mon.a (mon.0) 187 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-14T15:22:38.733 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-14T15:22:38.733 INFO:tasks.cephadm:Compressing logs... 2024-02-14T15:22:38.733 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T15:22:38.776 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T15:22:38.779 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T15:22:38.807 INFO:teuthology.orchestra.run.smithi017.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-14T15:22:38.807 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T15:22:38.807 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-02-14T15:22:38.808 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.6.log 2024-02-14T15:22:38.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.a.log 2024-02-14T15:22:38.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T15:22:38.808 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.g.log 2024-02-14T15:22:38.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.a.log: /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.j.log 2024-02-14T15:22:38.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.3.log 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-02-14T15:22:38.808 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-02-14T15:22:38.809 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.x.log 2024-02-14T15:22:38.809 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.0.log 2024-02-14T15:22:38.810 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T15:22:38.810 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T15:22:38.814 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-14T15:22:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T15:22:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.10.log 2024-02-14T15:22:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T15:22:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.b.log 2024-02-14T15:22:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.4.log 2024-02-14T15:22:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.1.log 2024-02-14T15:22:38.815 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T15:22:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.k.log 2024-02-14T15:22:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.h.log 2024-02-14T15:22:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.1.log: /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.7.log 2024-02-14T15:22:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.e.log 2024-02-14T15:22:38.816 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.7.log: 90.2% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T15:22:38.820 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.y.log 2024-02-14T15:22:38.821 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.k.log.gz 2024-02-14T15:22:38.821 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T15:22:38.824 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.g.log.gz 2024-02-14T15:22:38.824 INFO:teuthology.orchestra.run.smithi017.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T15:22:38.824 INFO:teuthology.orchestra.run.smithi017.stderr: 94.6% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.j.log.gz 2024-02-14T15:22:38.824 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log 2024-02-14T15:22:38.824 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log: -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T15:22:38.824 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log: 83.9% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T15:22:38.824 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.9.log 2024-02-14T15:22:38.825 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.d.log 2024-02-14T15:22:38.825 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.9.log: /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log: 87.1% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T15:22:38.825 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.a.log 2024-02-14T15:22:38.825 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.d.log.gz 2024-02-14T15:22:38.826 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T15:22:38.831 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.y.log: 92.2% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.y.log.gz 2024-02-14T15:22:38.832 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log 2024-02-14T15:22:38.832 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T15:22:38.832 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.a.log.gz 2024-02-14T15:22:38.833 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T15:22:38.837 INFO:teuthology.orchestra.run.smithi076.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-14T15:22:38.837 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T15:22:38.837 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.l.log 2024-02-14T15:22:38.837 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T15:22:38.837 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.i.log 2024-02-14T15:22:38.837 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.11.log 2024-02-14T15:22:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log: 87.2% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T15:22:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.b.log 2024-02-14T15:22:38.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.z.log 2024-02-14T15:22:38.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.8.log 2024-02-14T15:22:38.843 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.5.log 2024-02-14T15:22:38.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.z.log: /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.f.log 2024-02-14T15:22:38.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T15:22:38.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log 2024-02-14T15:22:38.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.2.log 2024-02-14T15:22:38.844 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.c.log 2024-02-14T15:22:38.844 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T15:22:38.844 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.c.log 2024-02-14T15:22:38.844 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log: /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.c.log: 92.2% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.z.log.gz 2024-02-14T15:22:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log: /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T15:22:39.020 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T15:22:39.080 INFO:teuthology.orchestra.run.smithi076.stderr: 83.5% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T15:22:39.091 INFO:teuthology.orchestra.run.smithi076.stderr: 87.2% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T15:22:39.097 INFO:teuthology.orchestra.run.smithi076.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T15:22:39.102 INFO:teuthology.orchestra.run.smithi050.stderr: 93.8% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.h.log.gz 2024-02-14T15:22:39.120 INFO:teuthology.orchestra.run.smithi076.stderr: 90.2% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T15:22:39.219 INFO:teuthology.orchestra.run.smithi017.stderr: 89.9% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mgr.x.log.gz 2024-02-14T15:22:39.340 INFO:teuthology.orchestra.run.smithi076.stderr: 94.1% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.f.log.gz 2024-02-14T15:22:39.896 INFO:teuthology.orchestra.run.smithi017.stderr: 90.5% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.a.log.gz 2024-02-14T15:22:40.424 INFO:teuthology.orchestra.run.smithi050.stderr: 92.9% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.b.log.gz 2024-02-14T15:22:40.558 INFO:teuthology.orchestra.run.smithi076.stderr: 92.8% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.l.log.gz 2024-02-14T15:22:43.657 INFO:teuthology.orchestra.run.smithi076.stderr: 91.9% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mon.c.log.gz 2024-02-14T15:22:56.777 INFO:teuthology.orchestra.run.smithi050.stderr: 95.4% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.b.log.gz 2024-02-14T15:22:59.614 INFO:teuthology.orchestra.run.smithi050.stderr: 95.1% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.e.log.gz 2024-02-14T15:23:03.216 INFO:teuthology.orchestra.run.smithi017.stderr: 91.9% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.0.log.gz 2024-02-14T15:23:04.895 INFO:teuthology.orchestra.run.smithi050.stderr: 91.7% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.1.log.gz 2024-02-14T15:23:07.599 INFO:teuthology.orchestra.run.smithi050.stderr: 91.8% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.10.log.gz 2024-02-14T15:23:09.306 INFO:teuthology.orchestra.run.smithi017.stderr: 91.7% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.3.log.gz 2024-02-14T15:23:10.136 INFO:teuthology.orchestra.run.smithi050.stderr: 91.6% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.7.log.gz 2024-02-14T15:23:10.545 INFO:teuthology.orchestra.run.smithi017.stderr: 91.4% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.6.log.gz 2024-02-14T15:23:13.946 INFO:teuthology.orchestra.run.smithi017.stderr: 91.5% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.9.log.gz 2024-02-14T15:23:13.948 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-02-14T15:23:13.948 INFO:teuthology.orchestra.run.smithi017.stderr:real 0m35.158s 2024-02-14T15:23:13.948 INFO:teuthology.orchestra.run.smithi017.stderr:user 1m59.697s 2024-02-14T15:23:13.948 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m3.463s 2024-02-14T15:23:21.026 INFO:teuthology.orchestra.run.smithi050.stderr: 91.4% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.4.log.gz 2024-02-14T15:23:21.028 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-02-14T15:23:21.029 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m42.236s 2024-02-14T15:23:21.029 INFO:teuthology.orchestra.run.smithi050.stderr:user 2m44.101s 2024-02-14T15:23:21.029 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m4.593s 2024-02-14T15:25:11.442 INFO:teuthology.orchestra.run.smithi076.stderr: 95.9% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.i.log.gz 2024-02-14T15:25:32.424 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.8.log.gz 2024-02-14T15:25:44.695 INFO:teuthology.orchestra.run.smithi076.stderr: 91.7% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.11.log.gz 2024-02-14T15:26:37.994 INFO:teuthology.orchestra.run.smithi076.stderr: 91.6% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.2.log.gz 2024-02-14T15:27:25.908 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-osd.5.log.gz 2024-02-14T15:32:56.898 INFO:teuthology.orchestra.run.smithi076.stderr: 93.3% -- replaced with /var/log/ceph/867ff6ea-cb48-11ee-95b9-87774f69a715/ceph-mds.c.log.gz 2024-02-14T15:32:56.913 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-02-14T15:32:56.913 INFO:teuthology.orchestra.run.smithi076.stderr:real 10m18.105s 2024-02-14T15:32:56.913 INFO:teuthology.orchestra.run.smithi076.stderr:user 7m37.562s 2024-02-14T15:32:56.914 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m17.925s 2024-02-14T15:32:56.914 INFO:tasks.cephadm:Archiving logs... 2024-02-14T15:32:56.915 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi017/log 2024-02-14T15:32:56.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T15:33:15.848 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi050/log 2024-02-14T15:33:15.849 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T15:33:36.651 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi076/log 2024-02-14T15:33:36.652 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T15:34:47.358 INFO:tasks.cephadm:Removing cluster... 2024-02-14T15:34:47.359 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 --force 2024-02-14T15:34:47.556 INFO:teuthology.orchestra.run.smithi017.stdout:Deleting cluster with fsid: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T15:34:47.998 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 --force 2024-02-14T15:34:48.202 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T15:34:48.668 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 867ff6ea-cb48-11ee-95b9-87774f69a715 --force 2024-02-14T15:34:49.135 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: 867ff6ea-cb48-11ee-95b9-87774f69a715 2024-02-14T15:34:49.775 INFO:tasks.cephadm:Removing cephadm ... 2024-02-14T15:34:49.776 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T15:34:49.795 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T15:34:49.815 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T15:34:49.831 INFO:tasks.cephadm:Teardown complete 2024-02-14T15:34:49.831 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-14T15:34:49.846 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-14T15:34:49.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-14T15:34:49.850 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-14T15:34:49.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-14T15:34:49.980 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-14T15:34:49.991 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-14T15:34:49.992 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T15:34:49.995 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T15:34:49.998 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T15:34:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-02-14T15:34:50.012 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-02-14T15:34:50.013 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-02-14T15:34:50.014 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T15:34:50.015 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-02-14T15:34:50.015 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 -4172ns[-4172ns] +/- 103ms 2024-02-14T15:34:50.015 INFO:teuthology.orchestra.run.smithi017.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -470us[ -455us] +/- 55ms 2024-02-14T15:34:50.015 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +670us[ +670us] +/- 75ms 2024-02-14T15:34:50.015 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T15:34:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T15:34:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-02-14T15:34:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 -2727ns[-2727ns] +/- 103ms 2024-02-14T15:34:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv02.front.sepia.ceph.com 3 6 377 65 -452us[ -443us] +/- 54ms 2024-02-14T15:34:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +665us[ +665us] +/- 75ms 2024-02-14T15:34:50.018 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T15:34:50.114 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T15:34:50.114 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-02-14T15:34:50.114 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 +2502ns[+2502ns] +/- 104ms 2024-02-14T15:34:50.114 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -465us[ -475us] +/- 55ms 2024-02-14T15:34:50.115 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +656us[ +646us] +/- 75ms 2024-02-14T15:34:50.115 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T15:34:50.116 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-14T15:34:50.128 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-14T15:34:50.129 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-14T15:34:50.139 DEBUG:teuthology.orchestra.run.smithi017:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T15:34:50.230 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T15:34:50.320 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T15:34:50.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T15:34:50.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:34:50.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T15:34:50.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:34:50.496 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T15:34:50.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:34:50.527 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-14T15:34:50.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-14T15:34:50.550 INFO:teuthology.task.internal:Duration was 3004.372411 seconds 2024-02-14T15:34:50.550 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-14T15:34:50.561 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-14T15:34:50.562 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T15:34:50.564 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T15:34:50.567 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T15:34:50.607 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T15:34:50.618 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T15:34:50.639 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T15:34:51.043 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-14T15:34:51.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2024-02-14T15:34:51.044 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T15:34:51.065 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-02-14T15:34:51.066 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T15:34:51.096 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-02-14T15:34:51.096 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T15:34:51.120 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-14T15:34:51.120 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T15:34:51.123 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T15:34:51.139 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T15:34:51.209 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-14T15:34:51.210 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T15:34:51.213 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T15:34:51.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T15:34:51.632 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-14T15:34:51.646 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-14T15:34:51.646 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T15:34:51.673 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T15:34:51.705 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T15:34:51.775 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-14T15:34:51.787 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T15:34:51.790 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T15:34:51.792 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T15:34:51.812 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2024-02-14T15:34:51.819 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-02-14T15:34:51.834 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-02-14T15:34:51.989 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T15:34:52.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:34:52.003 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T15:34:52.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:34:52.044 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T15:34:52.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T15:34:52.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-14T15:34:52.069 INFO:teuthology.task.internal:Transferring archived files... 2024-02-14T15:34:52.070 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi017 2024-02-14T15:34:52.070 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T15:34:52.115 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi050 2024-02-14T15:34:52.115 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T15:34:52.158 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621/remote/smithi076 2024-02-14T15:34:52.158 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T15:34:52.197 INFO:teuthology.task.internal:Removing archive directory... 2024-02-14T15:34:52.197 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T15:34:52.199 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T15:34:52.202 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T15:34:52.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-14T15:34:52.258 INFO:teuthology.task.internal:Not uploading archives. 2024-02-14T15:34:52.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-14T15:34:52.269 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-14T15:34:52.269 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T15:34:52.272 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T15:34:52.274 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T15:34:52.283 INFO:teuthology.orchestra.run.smithi017.stdout: 266028 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 15:34 /home/ubuntu/cephtest 2024-02-14T15:34:52.287 INFO:teuthology.orchestra.run.smithi050.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 15:34 /home/ubuntu/cephtest 2024-02-14T15:34:52.300 INFO:teuthology.orchestra.run.smithi076.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 15:34 /home/ubuntu/cephtest 2024-02-14T15:34:52.301 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-14T15:34:52.312 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-14T15:34:52.335 INFO:teuthology.nuke:Checking targets against current locks 2024-02-14T15:34:52.361 DEBUG:teuthology.nuke:shortname: smithi017 2024-02-14T15:34:52.362 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T15:34:52.387 DEBUG:teuthology.nuke:shortname: smithi050 2024-02-14T15:34:52.387 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T15:34:52.391 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 14:35:57.969843', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAa8ezSIdIQKqK0GdLjLoauCo8pQ9mm8cyzgjq2oN8+LPnPb3qENtPalaXvAKivFrPGUKg4fVB7Wtr8yMbGOD04='} 2024-02-14T15:34:52.409 DEBUG:teuthology.nuke:shortname: smithi076 2024-02-14T15:34:52.409 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T15:34:52.412 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 14:35:57.971559', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgqXvaELHhVlHUxeAN7QyRbJFFiggi2/+PXlYCDY0LypmslC2KbdwnKVNsBInMbxDfU31Z5HKm2oEdEyghJdhk='} 2024-02-14T15:34:52.432 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559621', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 14:35:57.967825', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK7bWvqByIQoaYmNpH83DOBveIfzmZbDyd9okIeP4/1goaE2n/hmJNOEqbaruhiJqDad6bqkxwnecLP2avm6QtA='} 2024-02-14T15:34:52.442 INFO:teuthology.orchestra.console.smithi017:Power off 2024-02-14T15:34:52.442 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T15:34:52.467 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-02-14T15:34:52.468 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:34:52.483 INFO:teuthology.orchestra.console.smithi050:Power off 2024-02-14T15:34:52.483 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T15:34:52.494 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-14T15:34:52.505 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2024-02-14T15:34:52.505 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:34:52.519 INFO:teuthology.orchestra.console.smithi076:Power off 2024-02-14T15:34:52.520 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T15:34:52.531 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-14T15:34:52.540 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2024-02-14T15:34:52.540 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:34:52.562 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-02-14T15:34:56.495 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:34:56.531 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:34:56.563 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:34:56.607 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-14T15:34:56.642 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-14T15:34:56.677 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-02-14T15:35:00.613 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:35:00.643 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:35:00.677 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:35:00.723 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-02-14T15:35:00.753 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-14T15:35:00.786 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-02-14T15:35:00.824 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-02-14T15:35:04.757 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:35:04.787 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T15:35:04.870 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2024-02-14T15:35:04.898 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2024-02-14T15:35:04.971 INFO:teuthology.orchestra.console.smithi050:Power off completed 2024-02-14T15:35:04.998 INFO:teuthology.orchestra.console.smithi076:Power off completed 2024-02-14T15:35:05.106 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3004.3724114894867 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi017 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6fe3fa3adb3a69ed9af8a494c4670cfd0d57fe0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e555d8617d2541938f36130109b2d731 status: fail success: false 2024-02-14T15:35:05.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T15:35:05.211 INFO:teuthology.run:FAIL